Norbert Bollow Internet-Draft TenthNet Project Expires: May 22, 2006 November 19, 2005 QQP - Quick Queues Protocol Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on May 22, 2006. Copyright Notice Copyright (C) The Internet Society (2005). Abstract The QQP protocol specifies a method for using a single reliable, stream-oriented connection (i.e. a TCP connection which may optionally be encrypted by means of SSL or TLS) for transmitting multiple webservice requests and responses. QQP plays the role of transport protocol for the QRPC webservice protocol, similar to how XML-RPC and SOAP often use HTTP as transport protocol. 1. Introduction Over the past few years, "webservices" protocols such as for example SOAP or XML-RPC have started becoming popular. The fundamental idea of "webservices" is to access functionality on another computer by means of standardized multi-purpose protocols and a standardized extensible data format. For example, when using the "webservices" paradigm to specify a message transport system, you don't specify the protocol and data format from scratch like it is done for SMTP in [RFC2821] and [RFC2822]; instead you specify them in terms of a general-purpose data format (often XML) and a general-purpose protocol for transporting data between computers. QQP, the Quick Queues Protocol, is such a general data transport protocol, specifically for data in the SXDF format [SXDF]. As far as QQP is concerned, there is no difference between webservice requests and responses; QQP merely provides something that could be called a "full-duplex webservice data communication channel" between two Hosts, which both parties can use to transmit any number of webservice requests and responses. The data which is transmitted over QQP connections consists of "QQP resources" which are SXDF resources consisting of an element named Envelope (which contains QQP-related and routing information) and an element named Data (which can contain arbitrary payload). In particular, the Envelope contains an Element named Action which specifies what the receiver should do with the resource. The fundamental idea of QQP is that a single TCP [RFC793] connection can be used for all webservice requests and responses between a given pair of Hosts as long as the amount of data in each request or response is small. This is advantageous because of the various overheads incurred by creating a new connection, which include * The tree-way handshake for establishing a TCP connection; the time required for this is bounded from below by the network's Round-Trip Time. * The initially slow speed of TCP connections, which is necessary to dynamically maximize throughput without risking to overload the network. * For encrypted connections, the overhead of authentication and session key negotiation. On the other hand, for requests and responses which contain large amounts of data it is better to create a separate TCP connection so that the transfer of large QQP resources does not prevent the the concurrent transfer of other data via QQP. Alternatively to TCP connections, any other implementation of reliable data streams can be used. 2. Establishing the communication channel 2.1. Opening connections For an unencrypted QQP communication channel based on TCP, one of the parties opens a TCP connection which uses port 26 on both ends of the connection. As soon as the connection is established, both parties send a QQP resource called the "Greeting" which contains values for "QQPstreamID", "mLimit", "cLimit", "eLimit", "Capabilities" and "ExceptionsTo" (see below). This TCP connection is called the "Main QQP Connection". Each Host can start sending data over the Main QQP Connection as soon as it has received the Greeting of the other Host. In addition, after the Greeting has been received (either immediately at that time or at any later time as long as no Farewell has been sent or received) each Host may open one or more additional TCP connections to port 26 at the remote Host (with the originating port different from 26). Each such connections is called an "Extra QQP Connection". Extra QQP Connections are used only for data transfer in one direction, from the Host which initiated the connection to the Host where port 26 is used. (Allowing bidirectional data transfer over Extra QQP Connections could create a security problem.) For an encrypted QQP communication channel via the TLS Protocol [RFC2246] the process of establishing the communication channel is identical except that port 27 instead of port 26 is used. Similarly QQP communication channels can be established over any other protocol layer which allows to establish multiple reliable data streams between two Hosts. 2.2. The Greeting in the Main QQP Connection The Greeting is a QQP resource which contains values for "QQPstreamID", "mLimit", "eLimit", "cNum", "Capabilities" and "ExceptionsTo". The QQPstreamID element holds a string value which MUST be unique among all QQP connections originating from the same host. For example, a Universal Unique IDentifier (see [UUID]) MAY be used. The values for "mLimit", "eLimit" and "cNum" MUST be positive integer values, while the value for "Capabilities" MUST be a list of strings and the value for "ExceptionsTo" MUST be a qrpc: URL [QRPC]. The value for "mLimit" specifies a size limit for data items (which are QQP resources) that the Host which generated the Greeting will accept via the Main QQP Connection. The value for "eLimit" is the corresponding size limit for data items which will be accepted via Extra QQP Connections. The "eLimit" value SHOULD be significantly greater than the "mLimit" value. The size of a QQP resource is the value of the datasize field plus the number of digits of the datasize field plus two. ExceptionsTo SHOULD be a qrpc: URL where QRPC Exception resources can be received. If Sender detects wrong behavior of the QQP server, it SHOULD make an attempt to use this qrpc URL for reporting the error. If one Host attempts to send a QQP resource which is larger than the relevant limit which has been specified by the receiving Host, the receiving Host SHOULD send a Farewell and then the connection over which the oversized QQP resource is being sent. In addition, the receiving Host SHOULD send a QRPC Exception resource to the ExceptionsTo URL which was specified by the sender of the oversized resource in its Greeting. The value for "cNum" is an estimate on the number of concurrent Extra QQP Connections which the Host that generated the Greeting will be likely to accept from the same sender. This does not imply a promise to accept that many connections. Each Host may refuse Extra QQP Connections at its sole discretion. However, at least most of the time, at least one Extra QQP Connection SHOULD be accepted from any host from which QQP connections are accepted at all. Capabilities is an array of strings. For example, if Receiver is capable of processing resources where the Action element contains a qrpc: URL [QRPC], the Capabilities element should contain the string "qrpc". Further common capability strings are "tcp:26" and "tls:27" which indicate how Extra QQP Connections can be created. A request will be made to IANA to operate a registry for the possible string values in the Capabilities list in the Greeting, and their meaning. 2.3. The Greeting in Extra QQP Connections In Extra QQP Connections the Greeting does not provide values for "mLimit", "eLimit", "cNum", "Capabilities" or "ExceptionsTo"; the values from the Greeting of the Main QQP Connection apply. However a unique QQPstreamID value MUST be provided, and depending on circumstances Resumes and Position elements may also be required: If the purpose of the Extra QQP Connection is to resume transmission of data from a previous QQP connection which died for some reason (possibly due to network problems or due to a software crash at either end of the connection) the value of the Resumes element MUST be the QQPstreamID of the QQP data stream which is being continued, and Position indicates the position (in the sense of counting bytes, with the first byte in the data stream having number 1) from which transmission will start. In this case the data following this Greeting may not start with a valid QQP resource when viewed on its own; however together with the previously-transmitted bytes of the stream which is being resumed, the data stream MUST be a sequence of valid QQP resources. Otherwise (if the Greeting of the Extra QQP Connection does not have the Resumes and Position elements) the Greeting is followed by one or more complete QQP resources. 2.4. QQP_FORK Resources For each Extra QQP Connection, the sender MUST send, over the Main QQP Connection, a "QQP_FORK resource" which is a QQP resource with the string "QQP_FORK" (without the quotes) as the value of the Action field and the QQPstreamID of Extra QQP Connection as the value of the Data element. The QQP_FORK resource MAY be sent before the Extra QQP Connection is opened or it MAY be sent afterwards. However it MUST be sent before a QQP_EOT (see below) is sent. The purpose of the requirement to send a QQP_FORK resource for each Extra QQP Connection is that this avoids the possibility of a race condition which could result in data loss if an Extra QQP Connection is opened at roughly the same time as the other party attempts to close the communication channel. 3. Data Transport 3.1. Overview The purposes of the QQP protocol is to transport data, which is required to consist of QQP format data resources, between two Hosts. The data can be transferred over the Main QQP Connection (which allows data to be transferred in both directions) and over Extra QQP Connections (each of which allows data transfer in one direction only, with a greater size limit on the QQP resources that can be transmitted). Each QQP resource which is transmitted via QQP consists of two elements named Envelope and Contents: An element named Envelope MUST be present; it SHOULD contain at least an Action element and an ExceptionsTo element. It MAY also contain a ResponseTo element. An element named Contents SHOULD also be present; it MAY contain arbitrary QQP data. For example the Contents element MAY contain human-readable data in QQP format, or its value MAY be merely a string of length zero, or its value MAY be a string containing encrypted data. A digital signature as described in section 6 of [SXDF] MAY be present; in this case the _DATA element which holds the signed data SHOULD contain both the Envelope and the Contents elements. 3.2. Envelope Elements 3.2.1. Action Element Sender SHOULD ensure that each QQP resource which it transmits contains an Action element with a value which is a valid URL of a type which receiver has included in the Capabilities array in the Greeting. 3.2.2. ExceptionsTo Element The ExceptionsTo element specifies an qrpc: URL to which QRPC Exception resources related to the present QQP resource should be sent. 3.2.3. ResponseTo Element If present, the optional ResponseTo Element specifies the URL to which the specified Action will cause a response to be sent. If it is known that currently no data can be sent to this URL, the receiver of the resource MAY send a QRPC Exception resource to the ExceptionsTo URL instead of executing the requested Action. 3.3. QQP_ACK Resources As data is received over a QQP connection, the receiving host SHOULD periodically send a QQP_ACK resource to acknowledge that all the data up to a certain point in the data stream has been successfully received and safely stored or otherwise processed so that it can be deleted at the sending host without risk of data loss. In addition For these resources, the value of the Action element is the string "QQP_ACK" (without the quotes), while the Data element contains two elements, QQPstreamID and Position. QQPstreamID has a string value which identifies the QQP connection to which the acknowledgment refers, and Position has a positive integer value which indicates a position (in the sense of counting bytes, with the first byte in the data stream having number 1) up to which (inclusive) all the data in the stream has been processed so that it can be safely deleted on the sending host. In the case that the data stream (to which the QQP_ACK resource refers) is a continuation of a data stream from a previous QQP connection which died, the Position counter refers to the number of bytes since the beginning of the original data stream. 4. Closure and Death of Connections 4.1. Closing QQP Extra Connections QQP Extra Connections can be closed by simply closing the TCP connection after the transmission of a QQP resource has been completed. The receiving host SHOULD use the main QQP connection to send a QQP_ACK when the data has been safely stored or processed. 4.2. Orderly closure of the QQP Main Connection When one party of the connection wants to close the QQP Main Connection for any reason, it SHOULD send over this connection a QQP_EOT resource which is a resource where the value of the Action element is the string "QQP_EOT" (without the quotes) and the Data element contains two elements, Position and Comment. The value of the Position element is the positive integer which corresponds to the position of the first byte of the QQP_EOT resource. This information is actually redundant because the receiving host SHOULD be keeping track of the position in the data stream anyway so that it can send QQP_ACKs as described in section 3.3. above. The reason for sending this in the QQP_EOT resource is to make it easier to debug situations where QQP connections fail to close properly. If this debugging information disagrees with the value that the receiving host has computed, an Exception MAY be generated, but the Position information in the QQP_EOT resource SHOULD otherwise be ignored. With the exception of QQP_FORK and QQP_ACK resources, both hosts SHOULD NOT start the transmission of further QQP resources after one of the hosts has sent a QQP_EOT resource; however where transmission of QQP resources has been started already, these transmissions SHOULD be completed provided that this is within the power of the computer program which implements QQP. When one host has sent a QQP_EOT resource, the other host SHOULD also do this as soon as it is able to do so over the QQP Main Connection, i.e. the QQP_EOT resource SHOULD be sent immediately after the current QQP resource transmission (if any) over the QQP Main Connection is complete, and after any pending QQP_FORK resources have been sent. After the QQP_EOT resource has been sent, the QQP Main Connection MUST NOT be used for sending any data except QQP_ACK resources, and each QQP Extra Connection MUST be closed by the sending host as soon as the current SXDF resource transmission is complete. Each host will then use the QQP Main Connection only to send QQP_ACKs until all pending data has been so acknowledged. When that has happened, both hosts have the information to know that this is the case. Then each host SHOULD close the connection. 4.3. Connection Death and Re-sending of Data QQP implementations MUST be prepared to handle the possibility that for whatever reasons sometimes QQP connections will die without having been closed in an orderly manner as indicated above. In such situations, each host SHOULD, for each QQP data stream on which it has been sending data for which no QQP_ACK has arrived, at the next opportunity establish a QQP Extra Connection and resume sending the data, starting from right after the Position of the latest QQP_ACK. 5. Security Considerations Webservices typically act on untrusted data; they therefore need to be carefully designed and reviewed to prevent security breaches. When webservice requests are transmitted over an untrusted network, firewalls are RECOMMENDED as an additional line of defense. It is RECOMMENDED that each resource which is the sent should be digitally signed. The received MAY reject resources which do not have a valid signature from an authorized sender of webservice data. When potentially privacy-sensitive data is transmitted over untrusted networks, the use of encrypted connections (as provided e.g. by TLS) is RECOMMENDED. 6. IANA Considerations A request will be made to IANA to operate a registry for the possible string values in the Capabilities list in the Greeting, and their meaning. References Normative References [RFC793] Postel, J. (Ed.): Transmission Control Protocol, RFC 793, September 1981. [RFC1738] Berners-Lee, T., Masinter, L., McCahill, M., Eds., "Uniform Resource Locators", RFC 1738. [SXDF] Bollow, N., "SXDF - Simple Extensible Data Format", work in progress. [QRPC] Bollow, N., "QRPC - Queueable Remote Procedure Calls", work in progress. [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2440] Callas, J., Donnerhacke, L., Finney, H., Thayer, R. "OpenPGP Message Format", RFC 2440 Informative References [UUID] International Organization for Standardization: "Information technology - Open Systems Interconnection - Remote Procedure Call (RPC)", ISO/IEC 11578:1996. [RFC2821] Klensin, J., Ed., "Simple Mail Transfer Protocol", RFC 2821. [RFC2822] Resnick, P., Ed., "Internet message format", RFC 2822. Authors' Address Norbert Bollow Weidlistrasse 18 CH-8624 Gruet Phone: +41 44 972 2059 EMail: nb@bollow.ch Full Copyright Statement Copyright (C) The Internet Society (2005). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all his rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society.