INTERNET-DRAFT draft-jdobias-appsawg-swapi-00.txt Intended Status: Standards Track J. A. Dobias Category: Informational AsiaComtek Expires: April 2, 2014 September 2013 Simple Web Application Programming Interface -- SWAPI 2.1 Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." Copyright Notice Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. This document may not be modified, and derivative works of it may not be created, except to format it for publication as an RFC or to translate it into languages other than English. Abstract SWAPI is a non application specific protocol which operates on top of the standard HTTP or HTTPS application-level protocol. It is specifically designed to expose web services as a lightweight non language specific interface that is extremely easy to parse without the need of any external software libraries. The structure of the protocol is simple enough to allow client applications to make and receive SWAPI calls via most shell scripts. However, the structure also provides strict type casting hints on responses which make the protocol suitable for integration with software written in high and low level programming languages. Dobias Informational [Page 1] Internet-Draft SWAPI 2.1 September 2013 SWAPI is especially well suited to heterogeneous environments where connectivity to a web service from many different types of client may be required. As a design goal, the parsing of responses is completely linear and can be done with the standard string processing functions built into any programming language and most shell scripting emvironments. Table of Contents 1 Introduction ............................................... pg 3 1.1 Purpose ................................................ pg 4 1.2 Current Usage Examples ................................. pg 4 1.3 Terminology ............................................ pg 4 2 Justification .............................................. pg 5 3 Data Integrity ............................................. pg 6 3.1 Compatibility .......................................... pg 7 3.2 Signing Mode Keys ...................................... pg 7 3.3 Client Identification .................................. pg 7 4 Making API Calls ........................................... pg 8 4.1 Calling the URL ........................................ pg 8 4.2 Passing Arguments ...................................... pg 9 4.3 Sending Arrays ......................................... pg 9 4.4 Client Identification .................................. pg 10 4.5 Signing Mode API Calls ................................. pg 10 4.5.1 The Signing Key .................................... pg 10 4.5.2 Signing the API Call ............................... pg 10 4.5.3 Requesting the Response Type ....................... pg 11 4.6 Timeouts ............................................... pg 12 5 Using Return Values ........................................ pg 12 5.1 Null ................................................... pg 13 5.2 Strings ................................................ pg 13 5.2.1 Strings with new lines ............................. pg 13 5.3 Integers ............................................... pg 13 5.4 Floating Points ........................................ pg 13 5.5 Booleans ............................................... pg 14 5.6 Arrays ................................................. pg 14 5.6.1 Associative Arrays ................................. pg 15 5.7 Comments ............................................... pg 15 5.8 Errors ................................................. pg 16 5.8.1 Signing Mode Errors ................................ pg 16 5.9 Signing Mode Responses ................................. pg 17 5.9.1 Verifying a Signature .............................. pg 17 6 Creating a SWAPI Compliant Function ........................ pg 18 6.1 The SWAPI Function Files ............................... pg 18 6.2 HTTP Headers ........................................... pg 18 6.3 New Lines .............................................. pg 18 6.4 Avoiding Timeouts on Complex Functions ................. pg 19 6.5 Returning Null ......................................... pg 19 6.6 Returning Strings ...................................... pg 19 6.7 Returning Integers ..................................... pg 19 6.8 Returning Floating Points .............................. pg 20 Dobias Informational [Page 2] Internet-Draft SWAPI 2.1 September 2013 6.9 Returning Booleans ..................................... pg 20 6.10 Returning Arrays ...................................... pg 21 6.10.1 Associative Arrays ................................ pg 22 6.11 Showing Comments ...................................... pg 23 6.12 Receiving Arguments ................................... pg 24 6.13 Identifying the Client ................................ pg 24 6.14 Receiving Signing Mode Requests ....................... pg 25 6.15 Receiving Signed API Calls ............................ pg 25 6.15.1 Verifying a Signature ............................. pg 25 6.16 Responding in Signing Mode ............................ pg 26 6.16.1 Signing a Response ................................ pg 27 6.17 Implementing a Verbose Mode ........................... pg 28 6.18 Returning an Error .................................... pg 28 7 Security Considerations .................................... pg 28 8 IANA Considerations ........................................ pg 28 Appendix A) Allowable character sets ......................... pg 29 1 Introduction This document is the first submission of the SWAPI protocol for consideration as an internet standard. The protocol has matured over the past 10 years since initial development to a level that may be appropriate for acceptance within a relevent working group of the IETF and eventually published as an RFC. Version 2.1 is the first version submitted as an Internet Draft. The web API specification in this document is designed to facilitate a closed proprietary API call system for systems that do not require description, discovery or complex data types. The methodology for making API calls allows them to behave similar to standard programming language functions in that they return a value when called and may also receive arguments. SWAPI is designed as a simple alternative to other web service technologies such as XML/RPC or SOAP. Those protocols are designed for public web services and have a significant amount of memory and processor overhead involved in their processing. The aim of SWAPI is to provide a much lighter and simple interface with greater compatibility. The trade off is that there is no out-of-the-box solution for processing SWAPI calls like in the SOAP model and no automatic discovery and description (hence it is aimed at creating closed proprietary APIs). Therefore, the use of SWAPI requires the service provider to make a function reference guide available to service users. There is also no support for complex data types such as objects or structures. But it does support mixed type arrays and multidimensional arrays and associative arrays. SWAPI 2.1 also adds the null data type. The data signing feature new to SWAPI 2.0 is also very efficient. Developers can choose to implement data signing on API calls or server responses. Signing is also linked to another new feature in SWAPI 2.0 which provides client identification. Using secure HTTPS server-client connections with client identification and bilateral signing keys provides reasonable security and reliability. More Dobias Informational [Page 3] Internet-Draft SWAPI 2.1 September 2013 secure algorithms such as HMAC have been intentionally avoided to provide the widest possible range of implementations. This document describes both how to make API calls and how to respond to them. Most developers need only concern themselves with the former in order to connect to SWAPI systems. All the programming examples in this document are in PHP but the concepts apply equally to other languages such as C, Java, Action Script, Cold Fusion, Perl and any others you care to use. SWAPI supports the following data types in responses: o Null o Strings o Signed integers o Signed floating points o Booleans o Arrays o Associative Arrays o Comments o Errors 1.1 Purpose The purpose of SWAPI is to provide an API for web services with the greatest possible compatibility between hetrogeneous systems that is easy for a developer to implement and powerful enough to be used in place of more complex existing technologies such as SOAP. 1.2 Current Usage Examples SWAPI has been used effectively in the following applications: o Exposing web services for mobile clients (smartphones). o Remote control of Interactive Voice Response (IVR) systems. o Inter-host communication between nodes in a web server cluster. o Asynchronous Javascript And XML (AJAX) request and responses. 1.3 Terminology The following terms should be understood as unambiguous in the scope Dobias Informational [Page 4] Internet-Draft SWAPI 2.1 September 2013 of this document: o Call: is the execution of a SWAPI function that is exposed via a URL. The call includes the HTTP request URL and the URL encoded arguments. o Client: is a software program that has implemented support for making SWAPI function calls and parsing responses. o GET: is URL encoded arguments passed to a HTTP/S server in the URL. o POST: is the URL encoded arguments passed to a HTTP/S server in the HTTP "Post" header. o Response: is the payload of text data returned by a SWAPI server after receiving a function call by a client. o Request: is synonymous with the word "call". o Server: is a software program bound to an HTTP/S server that has implemented support for receiving SWAPI calls and returning responses. In addition, the words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD" and "SHOULD NOT" denote behaviours which are mandatory. The words "RECOMMENDED", "NOT RECOMMENDED", "MAY" and "OPTIONAL" denote behaviours which are valid but not mandatory. 2 Justification During the development of various interconnected systems and web services, it became clear that it was not possible to implement a common remote procedure call system using established protocols such as SOAP because these protocols were not consistently implemented accross different programming languages. To be more specific, the following problems needed to be addressed: 1. Different remote interfaces often need different mechanisms for communicating with the same web service. Some interfaces supported SOAP, while others do not. 2. Using existing technologies such as SOAP, the support for such technologies would change depending on environment and version and may require additional external software libraries to function. 3. Different programming had different requirements for variable type handling and character set encoding. Dobias Informational [Page 5] Internet-Draft SWAPI 2.1 September 2013 4. The simplest environments which run as shell scripts could not handle calls or response parsing for any established protocol. Although shell scripts can call simple HTTP URL requests, these types of simple applications needed to run seperate executables to handle this type of remote operation. 5. The workload overhead (processor, memory and network) for XML-based protocols can be heavy due to large XML request and response payloads, schema validation and XML parsing. Instead of reinventing new solutions for the same problem or attempting rationalisation of all interfaces to environments that all support SOAP with the same feature set, an entirely different protocol is proposed that resolves the problems described above by doing the following: 1. It should be simple enough to implement in all programming languages and most shell scripting environments. 2. It should be possible to make a client or server using very simple string handling functions and operators built-in to most programming languages or scripting environments. 3. The server should run over standard HTTP/S [RFC 2616] and the underlying HTTP server should require little or no configuration changes. 4. The responses should include variable type casting hints and string character set declarations which can be used or ignored depending on the requirements of the programming language. 5. The calls should resemble standard programming language function calls and support zero or more unnamed arguments in a specific order. 6. The responses should resemble standard programming language function return values. That means one single variable is returned. 3 Data Integrity SWAPI 2.0 improves on the original SWAPI specification by allowing for robust verification of communication. These features become more difficult to implement depending on the language being used to create the SWAPI client or server. The features provide: o Request signing & verification - using a shared secret key o Response signing & verification - using a shared secret key o Client identification - provides mechanism for a server to indentify the client Dobias Informational [Page 6] Internet-Draft SWAPI 2.1 September 2013 The use of data signing is asynchronous. This means that requests and responses may have different usage of signing and verification depending on how a developer implements the features. A request could be made with signed data and a response could be returned with no signing implemented at all. Whether implementing a SWAPI client or the server, the following two policies are available: o No data integrity: Data is sent as-is. Probably still the most suitable for backend communication over a trusted network. o Signing mode: Data transmissions are signed to ensure a faithful transmission using a key known to both the sender and receiver. The hash algorithm used is not defined by SWAPI. Therefore anything from a simple CRC for checking data reliability to SHA512 for anti-tamper assurance can be implemented. 3.1 Compatibility The SWAPI specification mandates that any type of client can speak to any type of server. With the introduction of the new signing mode, the possibility of feature mismatches arises. A client written in Action Script may not have access to cryptographic libraries used by a server written in C++ using signing mode. Compatibility is assured through the rule of safest assumption. This rule means that a server operating in signing mode must also accept standard requests which are not signed. Requests made in signing mode where the key is not known to the server are processed as if they had no security. Only an actual failed signature verification causes an error. The response returned by the server must be in the same mode as requested by the client or an error is returned if not available. This would happen if a client requested a signing mode response from a server that could not respond in signing mode. 3.2 Signing Mode Keys How the keys are generated and handled for signing mode is not defined by the SWAPI specification. It is up to the developer to decide whether they want a single key used for the server and all clients or different keys for the server and each client or unique key pairs for each client-server response. The only rule is that the keys should be comprised of standard ASCII characters and no longer than 128 bytes long. This allows client developers with basic needs to hard code a key into their application if necessary. 3.3 Client Identification Client identification provides a mechanism for the server to know who is sending requests. Each request identifies the sender so the server can accept or drop the request accordingly. Servers that use this feature should simply drop all requests from unknown clients. Dobias Informational [Page 7] Internet-Draft SWAPI 2.1 September 2013 This mechanism is mainly for controlling API subscriptions rather than ensuring data integrity. 4 Making API Calls Making API function calls is very simple. Each call is related to one URL with arguments passed as either in the URL as GET data (query string) or in the HTTP headers as POST data. In some cases a combination of the two may be used. 4.1 Calling the URL There are two types of functions: those that take arguments and those that do not. A function that takes no arguments is called just by the URL like this: https://api.examples.asiacomtek.com/ping.api However a function that takes arguments requires an optional GET parameter that indicates where all the following arguments are coming from. The parameter is called "data" and it can take the following arguments: 0 receive parameters from HTTP headers as POST POST receive parameters from HTTP headers as POST 1 receive parameters from URL as GET GET receive parameters from URL as GET So to call a function that takes arguments, a URL may look like this: https://api.examples.asiacomtek.com/join_strings.api?data=POST If the data argument is missing, the data source is assumed to be POST by default. There is also another optional argument that provides verbose output in human readable format. This is flagged by adding the "verbose=TRUE" GET parameter to the function's URL. https://api.examples.asiacomtek.com/ping.api?verbose=TRUE Because verbose output is commented out, it should not affect an client application provided that the application is able to ignore lines beginning with the hash (#) character. Verbose output should be used in development for debugging. For lower traffic systems verbose output can be left on even in a production environment. Verbose output can also be useful if logged to provide a human readable record of API transactions. However, because the format of the verbose output may change without notice, it should never be used as input for a parser. A client application should only be sensitive to the actual return values, which will be discussed later. Dobias Informational [Page 8] Internet-Draft SWAPI 2.1 September 2013 4.2 Passing Arguments Because all arguments sent via the URL as GET data or in the HTTP headers as POST data they are inherently strings. There are no data type definitions to worry about for the caller. SWAP function calls are similar to standard function calls where the argument order is critical but the argument name has no meaning. Each variable is passed as n1, n2, n3... and so on. It goes without saying that the system only allows variables to be passed by value and not by reference. Take the below PHP-styled function as an example: To call this function within PHP the code may look like this: Now suppose a web API function does the same thing. The two strings must be passed in the right order (shown split over multiple lines). https://api.examples.asiacomtek.com/join_strings.api? \ data=GET&n1=Hello+&n2=+World%21 returns S|UTF-8|Hello World! Finally, if sending arguments in the URL as GET data, you should always remember to URL encode the values. Either encoding as URL or raw URL is fine. 4.3 Sending Arrays Sending an array presents no special problem but does require array type notation in the form of [ ] to be appended to the variable name. The notation requires the element key and must not be blank. The following example shows an array with two elements being passed (shown split over multiple lines). https://api.examples.asiacomtek.com/send_names.api? \ data=GET&n1[0]=john+smith&n1[1]=Jenny+Jones Dobias Informational [Page 9] Internet-Draft SWAPI 2.1 September 2013 An associative array can also be supplied and it may even be required for some functions. 4.4 Client Identification A client can also identify itself to the server. Servers that require this will silently ignore requests that have no identification or are not authorised. Servers that do not require this will ignore identifications. Identification is made by adding the optional GET parameter "token=". In most cases this identity token will be supplied by the service provider. The following example shows the string concatenation function with client identification (shown split over multiple lines). https://api.examples.asiacomtek.com/join_strings.api? \ data=GET&token=J238JFJ493KD&n1=Hello& \ n2=+World%21 4.5 Signing Mode API Calls As well as sending your SWAPI calls over an HTTPS connection, data in the request can be signed to help the receiver ensure the data is reliable and was not altered en-route or corrupted. The first thing to do is to actually sign the arguments. Then attach the signature to the request and send it as part of the API call. 4.5.1 The Signing Key The signing key is any ASCII printable string of 128 bytes or less. This must be shared with the receiver. For example: J23kj48che48xdih(O'($#(Jidf94idiksjs4j8xd 4.5.2 Signing the API Call Signing the request is done on the combination of the function name and arguments. It is similar to a URL query string but with all the formatting characters removed and no URL encoding. The signing key is then appended to the string and the signature generated using the hash algorithm of choice. It must be done as the last action before making the call otherwise the signature will fail validation at the server end. Only signature related arguments (sig, sig_hash and sig_return) are not part of the signed string. The order of the parameters is also important. The parameter hierarchy is as follows: o data o token o verbose o n1... Dobias Informational [Page 10] Internet-Draft SWAPI 2.1 September 2013 Below is the URL created for the string concatenation function (shown split over multiple lines). https://api.examples.asiacomtek.com/join_strings.api? \ data=GET&token=J238JFJ493KD&n1=Hello& \ n2=+World%21 Now the signing version is created (there is no new line between the last character of the string and the key). join_strings.api?data=GET&token=J238JFJ493KD& \ n1=Hello&n2= World! \ J23kj48che48xdih(O'($#(Jidf94idiksjs4j8xd Then a hash of the string is calculated. The example here is a MD5 hash. f38cd314937f82e9eaf9d6726df047a4 Finally, the signature is attached to the real GET data using the parameters "sig" and "sig_hash" to tell the server the signature and the algorithm used to create it. https://api.examples.asiacomtek.com/join_strings.api? \ data=GET&token=J238JFJ493KD&n1=Hello& \ n2=+World%21&sig= \ a2933195ecd44701490ca55910b57d78&sig_hash=MD5 It is important to note that the function name starts immediately after the domain name and slash character. Therefore, if a function was called at the following URL: https://api.examples.asiacomtek.com:4567/basic/ping.api Then the reported function name used for singing would be: basic/ping.api 4.5.3 Requesting the Response Type The SWAPI server will not provide a signed response unless requested by the calling function. This is requested by adding the optional "sig_return=" GET parameter to the query string. The signed query will now look like this: https://api.examples.asiacomtek.com/join_strings.api? \ data=GET&token=J238JFJ493KD&n1=Hello& \ n2=+World%21&sig= \ a2933195ecd44701490ca55910b57d78& \ sig_hash=MD5&sig_return=MD5 Dobias Informational [Page 11] Internet-Draft SWAPI 2.1 September 2013 The service provider will provide the list of hash algorithms that may be requested in the "sig_return" parameter. 4.6 Timeouts When calling an API function, you must implement your own method for dealing with server timeouts or other network related errors. For most web CGI languages, this is built in but you may need to override this behaviour to enforce something more elegant than an error; such as a retry or silently caching the request for later. 5 Using Return Values SWAPI API functions return values in just the same way as regular program functions do. At their most basic level they can return true or false to indicate success or failure, or they can return some other expected value like a string, a number or an array. In section 3.2 we called a function called join_strings that, as the name suggests, joins strings and we got the following return value: S|UTF-8|Hello World! This indicates a string is being returned, encoded in UTF-8 and its value is "Hello World!". As mentioned earlier, everything is inherently a string so any desired types must be specifically included as part of the return value. It is up to the developer to make use of these type definitions (if at all), but they are present for every return value. The web API supports 5 variable types. They are: o Null o Strings o Integers o Floating points o Arrays (non indexed, indexed, single dimension, multidimensional, mixed type & associative) o Booleans To make a web API behave as much like a normal function call as possible, a function will return only one value. If receiving an array as a return value, each value is terminated by a single new line (LF). Carriage returns (CR) are not used in SWAPI web API's output (carriage returns have a special meaning within strings). Type declaration segments are always delineated using the pipe (|) character. Because the number of pipes in a type declaration is known, the pipe character is also valid in the body of the value itself. Dobias Informational [Page 12] Internet-Draft SWAPI 2.1 September 2013 5.1 Null Null values are returned with the "N" keyword. No other markup is required. Therefore a null response would simply look like this: N 5.2 Strings Strings are returned with the "S" keyword followed by a pipe (|); the string character encoding; another pipe (|) and then the string itself. S|| Examples: S|UTF-8|This is a string encoded in UTF8 S|iso-2022-jp|Valid and also has a third | which is okay. Your parser should treat everything from the second pipe (|) onwards as the string up until the linefeed (LF) character or the end of file (whichever comes first). The string encoding is also vitally important for most non-English applications. 5.2.1 Strings with new lines To avoid confusion with the linefeed character, all new lines are returned as carriage returns (CR). Even if you pass a string for processing that contains new lines as linefeeds (LF) they will be converted to carriage returns (CR) for the return value. Windows-style carriage return-linefeeds (CRLF) new lines are also returned as just single carriage returns (CR). Please bear this in mind and compensate for it if necessary in your application. Unexpected newline characters are a common cause of bugs. 5.3 Integers Integers are returned with the "I" keyword followed by a single pipe (|) character and then the integer itself. I| Examples: I|873458 I|-2342 5.4 Floating Points Floating point numbers are returned with the "F" keyword followed by Dobias Informational [Page 13] Internet-Draft SWAPI 2.1 September 2013 a single pipe (|) character and then the floating point number itself. F| Examples: F|2342.23 F|-0.5 F|0.0 5.5 Booleans Booleans represent the simplest return values. A boolean is returned with the "B" keyword followed by a single pipe (|) character and the boolean value which will always be either 0 (false) or 1 (true). B| Examples: B|0 B|1 5.6 Arrays An array is merely a list of values delineated by linefeed (LF) characters and preceded with the "A" or "K" keywords and suffixed by the "C" keyword. For single dimension arrays, the "C" keyword is optional so your parser should recognize both the last line of the output or the presence of "C" as the end of an array. SWAPI 2.0 adds indexing and association to arrays which changes the format slightly from other value types. An indexed array is started using the "A" keyword and an associative array is started with the "K" keyword. Multidimensional arrays are shown as blocks of values enclosed within "A" or "K" and "C" keywords that appear within the body of another array. SWAPI does not enforce a limit on the maximum length of an array or the maximum levels of dimensionality an array may have. A | [C] or for a multidimensional array A A | C [C] Examples: Dobias Informational [Page 14] Internet-Draft SWAPI 2.1 September 2013 A S|UTF-8|Monday S|UTF-8|Tuesday S|UTF-8|Wednesday S|UTF-8|Thursday S|UTF-8|Friday S|UTF-8|Saturday S|UTF-8|Sunday I|435 F|34.5 A S|UTF-8|Jenny Jones S|UTF-8|Dirk Bogart A S|UTF-8|Ziggy Stardust I|45 S|UTF-8|January 10, 1963 S|UTF-8|male C C 5.6.1 Associative Arrays Associative arrays have an additional pipe (|) delimited value containing an index string. This can even be part of another array declaration. The string must be no longer than 32 bytes and contain only ASCII alphanumeric and the following characters: -_. There must be no white space in this string. Clients that do not support associative arrays will ignore the keys so the use of these arrays should be kept to a minimum to ensure maximum compatibility. K || [C] Example: K name|S|UTF-8|John Doe age|I|43 occupation|S|UTF-8|Professional scuba diver phone|I|555123789 C 5.7 Comments A comment is any line intended to be readable by humans. Comments Dobias Informational [Page 15] Internet-Draft SWAPI 2.1 September 2013 are lines of text commented out in the same way as shell scripts using the hash (#) character. A parser should ignore any lines beginning with a hash. The return value of a function with comments may look like this: # Ping function called. # SNS API function by Joey Dobias (joey@alegria.co.jp). # If you are seeing this message and the time is also correct # then it means the web server is up and running and this # response is not cached. # The return array format is: # 0 = Web server Timestamp # 1 = Fileserver status # 2 = User account database status # 3 = Stress Check analysis database status # 4 = Blog database status # 5 = Messaging database status # 6 = Podcast database status # 7 = Admin database status # 8 = HTTP cluster ping test status # Standard function return values to follow. A S|UTF-8|2007-02-05 07:34:04 (GMT) S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK 5.8 Errors Occasionally a function may not complete successfully and will return an error instead of a normal return value. An error is basically a string that describes the error and uses the "E" keyword. E|| Example: E|UTF-8|Did not receive arguments from client. 5.8.1 Signing Mode Errors With the exception of signing mode, all errors are application specific. The following special signing mode errors may be returned as the only response to an API call: o SIG-FAIL - Indicates that the data received from the client failed verification. o SIG-NO-HASH - Indicates that the hash algorithm requested by Dobias Informational [Page 16] Internet-Draft SWAPI 2.1 September 2013 the client is not supported. In either of these scenarios, no other output (excluding preceding comments) will be returned. 5.9 Signing Mode Responses If an API function call requested that the response be signed using the GET parameter "sig_return" in the URL, then the server will provide a response in signing mode using the hash algorithm requested. This will result in the last line in the response (even after all comments) containing the special signature value. SIG|| Example: SIG|MD5|5765ada1c3974d0a0dde201742bf4999 5.9.1 Verifying a Signature Signature verification is optional, though clients that do not verify signatures will probably not request them. Verifying the signature requires that the client has the same key that the server used sign the data. The signature is verified by appending the key to the end of the response output (including all preceding comments), minus the line containing the signature itself and calculating the checksum. If the checksum matches the signature, the data is valid. Here is an example of a received response. #Ping function called. A S|UTF-8|2007-02-05 07:34:04 (GMT) S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK SIG|MD5|5765ada1c3974d0a0dde201742bf4999 The client then removes the signature line and appends the shared key. #Ping function called. A S|UTF-8|2007-02-05 07:34:04 (GMT) S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK Dobias Informational [Page 17] Internet-Draft SWAPI 2.1 September 2013 S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK J23kj48che48xdih(O'($#(Jidf94idiksjs4j8xd A MD5 checksum is calculated (the same algorithm as the received signature). 5765ada1c3974d0a0dde201742bf4999 In this example, the checksum matches the signature checksum and the data is confirmed as valid. 6 Creating a SWAPI Compliant Function Understanding how to call SWAPI functions provides almost enough knowledge for creating them. This section re-covers some of the concepts from sections 3 and 4 as well as introduces some other requirements for creating a SWAPI compliant function. 6.1 The SWAPI Function Files Regardless of what language your SWAPI function was written in, the extension should always be ".api". This allows developers to freely upgrade and change their SWAPI functions including the language they were written in without it having any effect on the clients that call the functions. This means that our join_strings.api function (see section 3.2), which was written in PHP, can be converted to JSP or Cold Fusion without the client applications that use it having to be changed or their developers ever having to know about it. To accomplish this, a web server needs to be configured to treat anything ending in .api as a CGI file of the programming language you are using. 6.2 HTTP Headers The only requirement for HTTP headers is that SWAPI functions should respond by declaring their content type as plain text. The header format is: Content-type: text/plain The encoding header is not relevant because SWAPI compliant parsers should be looking at the string declarations in the return output for their character encoding on a per string basis. The encoding header only affects the display of human readable comments, which are probably in English. 6.3 New Lines Dobias Informational [Page 18] Internet-Draft SWAPI 2.1 September 2013 New lines in the output must all be single linefeed (LF) characters with no carriage returns (CR) in the main body of the output. New lines in strings being returned should always be single carriage returns (CR). Therefore before returning strings that may contain new lines, it is very important to convert whatever newline characters the string uses into single carriage returns (CR) otherwise the parser will not be able to receive the string correctly. 6.4 Avoiding Timeouts on Complex Functions Some functions may take many seconds to process before returning. This can cause the client parser to fail with a timeout error. To avoid this, make sure some output is sent to the client right away. A single comment with the name and version of the function is a good way to achieve this. 6.5 Returning Null Null should be returned with the "N" keyword. No other markup is required. For example: N 6.6 Returning Strings Strings should be returned with the "S" keyword followed by a pipe (|); the string character encoding; another pipe (|) and then the string itself. S|| Examples: S|UTF-8|This is a string encoded in UTF8 S|iso-2022-jp|Valid and also has a third | which is okay. Bad examples: # The character set declaration has been omitted. S|This string is missing a pipe and a character set declaration. # This string also misses the character set declaration and the # second pipe makes the parser think the character set is " # This string misses out a". S|This string misses out a | which causes an error on next pipe. 6.7 Returning Integers Integers should be returned with the "I" keyword followed by a single pipe (|) character and then the integer itself. Dobias Informational [Page 19] Internet-Draft SWAPI 2.1 September 2013 I| Examples: I|873458 I|-2342 Bad examples: # A floating point where an integer should be. I|3453.3453 # A string where an integer should be. I|This isn't an integer 6.8 Returning Floating Points Floating point numbers should be returned with the "F" keyword followed by a single pipe (|) character and then the floating point number itself. F| Examples F|2342.23 F|-0.5 F|0.0 Bad examples: # Is an integer F|0 # Is a string F|This isn't a float 6.9 Returning Booleans A boolean should be returned with the "B" keyword followed by a single pipe (|) character and the boolean value which will always be either 0 (false) or 1 (true). B| Examples: B|0 Dobias Informational [Page 20] Internet-Draft SWAPI 2.1 September 2013 B|1 Bad examples: # Enumerated TRUE/FALSE strings are not valid booleans B|FALSE B|TRUE 6.10 Returning Arrays Arrays should be returned as a list of values delineated by linefeed (LF) characters and preceded with the "A" or "K" keywords and suffixed by the "C" keyword. For single dimension arrays, the "C" keyword is optional. Multidimensional arrays are returned as blocks of values enclosed within "A" or "K" and "C" keywords that appear within the body of another array. SWAPI does not enforce a limit on the maximum length of an array or the maximum levels of dimensionality an array may have. A | [C] or for a multidimensional array A A | C [C] Examples: A S|UTF-8|Monday S|UTF-8|Tuesday S|UTF-8|Wednesday S|UTF-8|Thursday S|UTF-8|Friday S|UTF-8|Saturday S|UTF-8|Sunday I|435 F|34.5 A S|UTF-8|Jenny Jones S|UTF-8|Dirk Bogart A S|UTF-8|Ziggy Stardust I|45 S|UTF-8|January 10, 1963 Dobias Informational [Page 21] Internet-Draft SWAPI 2.1 September 2013 S|UTF-8|male C C Bad examples: # ARRAY keyword must be on its own line A|S|UTF-8|Monday S|UTF-8|Tuesday S|UTF-8|Wednesday # Forget to use CLOSE-ARRAY keyword when describing a # multidimensional array. A I|435 F|34.5 A S|UTF-8|Index values for bookmark I|456 F|-0.001 C 6.10.1 Associative Arrays Associative arrays have an additional pipe (|) delimited value containing an index string. This can even be part of another array declaration. The string must be no longer than 32 bytes and contain only ASCII alphanumeric and the following characters: -_. There must be no white space in this string. Clients that do not support associative arrays will ignore the keys so the use of these arrays should be kept to a minimum to ensure maximum compatibility. Other rules are the same as for standard arrays. K || [C] Examples: K name|S|UTF-8|John Doe age|I|43 occupation|S|UTF-8|Professional scuba diver phone|I|555123789 C A K first_name|S|UTF-8|John Dobias Informational [Page 22] Internet-Draft SWAPI 2.1 September 2013 last_name|S|UTF-8|Doe age|I|43 C K first_name|S|UTF-8|Sue last_name|S|UTF-8|Pollard age|I|29 C C A 0|K first_name|S|UTF-8|John last_name|S|UTF-8|Doe age|I|43 C 1|K first_name|S|UTF-8|Sue last_name|S|UTF-8|Pollard age|I|29 C C 6.11 Showing Comments Comments can be inserted in much the same way as shell scripts using the hash (#) character. An example of a commented function may look like this: # Ping function called. # SNS API function by Joey Dobias (joey@alegria.co.jp). # If you are seeing this message and the time is also correct # then it means the web server is up and running and this # response is not cached. # The return array format is: # 0 = Web server Timestamp # 1 = Fileserver status # 2 = User account database status # 3 = Stress Check analysis database status # 4 = Blog database status # 5 = Messaging database status # 6 = Podcast database status # 7 = Admin database status # 8 = HTTP cluster ping test status #Standard function return values to follow. A S|UTF-8|2007-02-05 07:34:04 (GMT) S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK Dobias Informational [Page 23] Internet-Draft SWAPI 2.1 September 2013 S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK Comments can be placed on any line of the output to help in debugging and logging. However, if a response is sent is signing mode, no comments can be output after the signature. 6.12 Receiving Arguments Arguments may be received either in the URL query string as GET data or in the HTTP header as POST data. SWAPI functions should accept both as possible input sources with the argument location being explicitly declared in the URL GET parameter called "data". If the client calls a SWAPI function and does not declare where the arguments are located, your function should default to POST data in the HTTP header and if arguments are not found, should return an error. Valid input for the data parameter is: 0 receive parameters from HTTP headers as POST POST receive parameters from HTTP headers as POST 1 receive parameters from URL as GET GET receive parameters from URL as GET SWAPI functions can then parse the input in any way necessary. A function must ignore all variables that are not explicitly requested. So if a function which takes three arguments is called it must parse and validate only the first three arguments, even if the client passes extra arguments to the function. The arguments received will be labelled consecutively with the letter "n" following by a number stating at 1 (n1, n2, n3... etc.). Remember to remove slashes from escaped characters or decode URL encoded strings back into normal text before processing the arguments. 6.13 Identifying the Client A SWAPI 2.0 service provider can request that all clients identify themselves in the URL with the GET parameter "token=". The content of this parameter and how it is handled is up to the service provider to decide. The original intended usage was for the service provider to allocate a license code to each organisation that had a formal agreement to use the SWAPI API. A service provider could then drop requests from unlicensed clients or from clients with expired contracts. See the API call below: https://api.examples.asiacomtek.com/ping.api?token=PQNDHUOREN38 SWAPI servers that do not implement client identification should ignore this parameter. Dobias Informational [Page 24] Internet-Draft SWAPI 2.1 September 2013 6.14 Receiving Signing Mode Requests The client may request a response sent in signing mode by including the optional GET parameter "sig_return" in the URL. The value of the parameter should be a hash algorithm known to the server. See the API call below: https://api.examples.asiacomtek.com/ping.api?sig_return=SHA256 Handling the request should follow these rules: o If signing mode is not supported, the "sig_return" parameter is ignored. o If signing mode is supported but a signing key is not available (when signing keys are specific to the client), the "sig_return" parameter is ignored. If signing mode is supported but there the algorithm is not supported, a SIG-NO-HASH error is returned. No other output other than preceding comments are permitted. o If signing mode is supported and a signing key is available, a response is sent in signing mode. 6.15 Receiving Signed API Calls The client may send an API call in signing mode indicated by the GET parameter "sig" in the URL. If processed, this must be accompanied by the "sig_hash" parameter. See the API call below (shown on multiple lines): https://api.examples.asiacomtek.com/join_strings.api?data=GET& \ token=J238JFJ493KD&n1=Hello&n2=+World%21&sig= \ f38cd314937f82e9eaf9d6726df047a4&sig_hash=MD5 Handling the request should follow these rules: o If signing mode is not supported, the "sig" and "sig_hash" parameters are ignored. o If signing mode is supported but a signing key is not available, the "sig" and "sig_hash" parameters are ignored. o If signing mode is supported but the "sig_hash" parameter is missing or contains an unsupported algorithm, a SIG-NO-HASH error is returned. No other output other than preceding comments are permitted. o If signing mode is supported and a signing key is available, then the data signature is verified for authenticity. 6.15.1 Verifying a Signature Dobias Informational [Page 25] Internet-Draft SWAPI 2.1 September 2013 Verifying a request is done on the combination of the function name and arguments. It is similar to a URL query string but with all the formatting characters removed and no URL encoding. The signing key is then appended to the string and the signature generated using the checksum requested by the client. This requires that the server has the same signing key used by the client. Only signature related arguments (sig, sig_hash and sig_return) are not part of the signed string. The order of the parameters is also important. The parameter hierarchy is as follows: o data o token o verbose o n1... For example, the following API call is received (shown on multiple lines). https://api.examples.asiacomtek.com/join_strings.api?data=GET& \ token=J238JFJ493KD&n1=Hello&n2=+World%21&sig= \ f38cd314937f82e9eaf9d6726df047a4&sig_hash=MD5&sig_return=MD5 Now the signing version is created with the shared key (there is no new line between the last character of the string and the key). join_strings.api?data=GET&token=J238JFJ493KD&n1=Hello&n2=+World! \ J23kj48che48xdih(O'($#(Jidf94idiksjs4j8xd Then a hash of the string is calculated. The example here is a MD5 hash. f38cd314937f82e9eaf9d6726df047a4 Finally, the signature is compared to the value of the "sig" parameter. If the values are the same, then SWAPI server processes the request. Otherwise the error SIG-FAIL is returned as the only response other than comments. 6.16 Responding in Signing Mode A SWAPI server will provide a response in signing mode using the hash algorithm requested in the GET parameter "sig_return". This will result in the last line in the response (even after all comments) containing the special signature value. SIG|| Example: Dobias Informational [Page 26] Internet-Draft SWAPI 2.1 September 2013 SIG|MD5|5765ada1c3974d0a0dde201742bf4999 6.16.1 Signing a Response Signing the response is done by appending the key to the end of the response output (including all comments), minus the line containing the signature itself and calculating the checksum. Here is an example of a signing mode response. The response is generated #Ping function called. A S|UTF-8|2007-02-05 07:34:04 (GMT) S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK The server appends the signing key to a new line. #Ping function called. A S|UTF-8|2007-02-05 07:34:04 (GMT) S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK J23kj48che48xdih(O'($#(Jidf94idiksjs4j8xd An MD5 checksum is calculated (the same algorithm as the received signature). 5765ada1c3974d0a0dde201742bf4999 Then the signed response is sent to the client. #Ping function called. A S|UTF-8|2007-02-05 07:34:04 (GMT) S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK Dobias Informational [Page 27] Internet-Draft SWAPI 2.1 September 2013 S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK S|UTF-8|OK SIG|MD5|5765ada1c3974d0a0dde201742bf4999 6.17 Implementing a Verbose Mode Including a verbose mode into a function is not required for SWAPI but it is good practice. Because the server and client may be separated by great distance and time zones, SWAPI functions should help the client-side developers as much as possible by providing well documented output to their return values. A SWAPI function that supports verbose output should detect the presence of a GET perimeter called "verbose=TRUE". See the API call below: https://api.examples.asiacomtek.com/ping.api?verbose=TRUE 6.18 Returning an Error If for some reason a function fails, it should return an error instead of any other return value. An error is just a string with "E" keyword. The string that describes the error should be brief and easy to understand. E|| Example: E|UTF-8|Did not receive arguments from client. 7 Security Considerations The SWAPI protocol is inherently non secure by design. Although an optional data integrity assurance mechanism is provided, the payload of the calls and responses are (on the SWAPI level) in plain text. For this reason, the use of the security features of the underlying HTTP protocol in HTTPS mode or additional tunnelling is strongly encouraged. Using features such as standard HTTPS, client certificates, and additional tunnelling, the SWAPI exchanges can be made as secure as the underlying application layer. This document does not describe the usage of HTTPS or certificate-base authentication. Other security considerations relate to the implementation of the SWAPI service itself. The implement or of a web service has the responsibility to ensure intelligent design decisions when determining what functions to expose and how to validate function arguments. 8 IANA Considerations Dobias Informational [Page 28] Internet-Draft SWAPI 2.1 September 2013 At this time, no changes to the IANA registry are required for this purely informational draft. Appendix A) Allowable character sets A SWAPI application does not need to support more than one character set (our example code in this document only supports UTF-8). Whatever character sets it does support should conform to the notation of character sets exactly as show below. In other words, to declare UTF-8, you must use "UTF-8" and not "UTF8" or "UTF 8". Otherwise the client might not understand which character set you are declaring for a given string. o UTF-32 o UTF-32BE o UTF-32LE o UTF-16 o UTF-16BE o UTF-16LE o UTF-7 o UTF-8 o ASCII o EUC-JP o SJIS o ISO-2022-JP o JIS o ISO-8859-1 o ISO-8859-2 o ISO-8859-3 o ISO-8859-4 o ISO-8859-5 o ISO-8859-6 Dobias Informational [Page 29] Internet-Draft SWAPI 2.1 September 2013 o ISO-8859-7 o ISO-8859-8 o ISO-8859-9 o ISO-8859-10 o ISO-8859-13 o ISO-8859-14 o ISO-8859-15 o BASE64 o EUC-CN o CP936 o HZ o EUC-TW o BIG-5 o EUC-KR o ISO-2022-KR o KOI8-R References 1. [RFC 2616] Fielding, R. et al. "Hypertext Transfer Protocol -- HTTP/1.1" (June 1999) Author's Address Joey A. Dobias 905 1-16-5 Higashi-sakashita Itabashi-ku Tokyo, 174-0042 Japan E-mail: feeling_zenjp@yahoo.co.jp Document expires: April 2, 2014 Dobias Informational [Page 30]