Norbert Bollow Internet-Draft Tenthnet Project Expires: November 5, 2006 May 6, 2006 QQP - Quick Queues Protocol Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on November 5, 2006. Copyright Notice Copyright (C) The Internet Society (2006). Abstract The QQP protocol specifies a method for transmitting multiple webservice requests and responses between two Hosts. QQP plays the role of transport protocol for the QRPC webservice protocol, similar to how XML-RPC and SOAP often use HTTP over TCP as transport protocol. QQP is called the "Quick Queues Protocol" because it is faster than transmitting XML data via HTTP over TCP for several reasons. QQP is also designed to handle Internet Protocol multihoming, hiding IP address changes from application layer software without need for introducing an additional protocol layer for that purpose. 1. Introduction Over the past few years, "webservices" protocols such as for example SOAP or XML-RPC have started becoming popular. The fundamental idea of "webservices" is to access functionality on another computer by means of standardized multi-purpose protocols and a standardized extensible data format. For example, when using the "webservices" paradigm to specify a message transport system, you don't specify the protocol and data format from scratch like it is done for SMTP in [RFC2821] and [RFC2822]; instead you specify them in terms of a general-purpose data format (often XML) and a general-purpose protocol for transporting data between computers. QQP, the Quick Queues Protocol, is such a general data transport protocol, specifically for data in the SXDF format [SXDF]. As far as QQP is concerned, there is no difference between webservice requests and responses; QQP merely provides something that could be called a "full-duplex webservice data communication channel" between two Hosts, which both parties can use to transmit any number of webservice requests and responses. The data which is transmitted over QQP connections consists of "QQP resources" which are SXDF resources consisting of an element named Envelope (which contains QQP-related and routing information) and an element named Data (which can contain arbitrary payload). In particular, the Envelope contains an Element named Action which specifies what the receiver should do with the resource. One possible use of QQP is to use a single TCP [RFC793] connection can be used for all webservice requests and responses between a given pair of Hosts as long as the amount of data in each request or response is small. This is advantageous because of the various overheads incurred by creating a new connection, which include * The tree-way handshake for establishing a TCP connection; the time required for this is bounded from below by the network's Round-Trip Time. * The initially slow speed of TCP connections, which is necessary to dynamically maximize throughput without risking to overload the network. * For encrypted connections, the overhead of authentication and session key negotiation. The above-described method of using QQP over TCP is however only intended for situations where the preferred method of sending QQP data chunks directly as the payload of IPv6 packets is not available, either because one of the hosts is an IPv4 host behind NAT, or because the operating system of one of the hosts does not provide QQP support, and the application program works around this shortcoming by using QQP over one or more TCP connections. Creating several separate TCP connections between the same pair of hosts can be desirable when requests and responses which contain large amounts of data: With separate TCP connections the transfer of large QQP resources does not prevent the the concurrent transfer of other data via QQP. When several TCP connections are created, for each TCP connection there will be a "queue" of QQP resources which await transmission via that particular TCP connection. In the preferred situation, where the use of TCP is avoided and the QQP resources are transmitted as chunks as the payload of IPv6 packets [RFC2460], each of the hosts which communicates via QQP must nevertheless maintain a list of queues of QQP resources. In each queue, QQP resources are transmitted one-after-the-other. If the underlying network infrastructure supports QoS guarantees which are based on "flows" in the sense of [RFC2460], QQP implementations MAY make use of this functionality to provide QoS guarantees to QQP-using application programs. In this case, each [RFC2460] "flow" SHOULD be dedicated to one or more QQP queues, and as long as the QoS guarantees of that [RFC2460] flow are not violated, all data from the corresponding queues SHOULD be sent exclusively via that flow. Hosts which are not on IPv6 networks SHOULD work around that problem by means of one of the following three methods: (a) Encapsulating the QQP IPv6 packets in IPv4 packets by means of the 6to4 technique defined in [RFC3056]. This methods requires QQP support in the host's Internet Protocol Implementation, so that the special case of such IPv4 packets which contain IPv6 QQP packets is recognized and handled appropriately. (b) Encapsulating the QQP IPv6 packets in IPv4-based UDP datagrams, by means of the Teredo technique defined in [RFC4380]. This can work via a general-purpose Teredo gateway, or via a special-purpose gateway on a QQP server listing on UDP port 26. (c) Sending the data via a TCP connection to TCP port 26 on the QQP server. 2. Fundamental Definitions 2.1. QQP Hosts and FQDNs As far as the QQP protocol is concerned, internet hosts are identified by means of fully-qualified domain names (FQDNs). It is possible for a single physical machine to have more than one FQDN and to use multiple FQDNs at the same time for communicating via QQP, but as far as the QQP protocol is concerned, these different FQDNs correspond to different (logical) hosts, called "QQP hosts" in the following. 2.2. Communication Channels QQP defines a "communication channel" between two hosts which is identified by an unordered pair of fully-qualified domain names, with each of these FQDNs identifying one of the hosts participating in the communication channel. 2.3. ChannelID Each QQP communication channel is characterized by a unique ChannelID which is a 128-bit hash computed as follows from the FQDNs of the two hosts between whom the communication channel is defined: A string is computed from the two FQDNs by comparing the FQDNs alphabetically, putting that FQDN first which precedes the other in the alphabetical order, then an ASCII LF character, then the other FQDN, then another ASCII LF character. Then an MD5 hash is computed of this string which consists of the two FQDNs and the two LF characters. This MD5 hash is the ChannelID. 2.4. QQP Resources QQP achieves its purpose by transmitting SXDF resources between the two hosts between which the communication channel has been defined. These SXDF resources which are transmitted by means of QQP are also called "QQP resources". This specification distinguishes between two types of such QQP resources, namely "QQP control resources" and "QQP data resources". Here "QQP data resources" are payload such as webservice requests and responses, while "QQP control resources" are special SXDF resources defined in section 6 of this specification. 2.5. QQP Packets This specification defines a number of data transport methods for transmitting QQP resources. In most of these data transport methods, the QQP resources are embedded in IPv6 packets (which may in turn be embedded in IPv4 packets or IPv4 UDP datagrams). These IPv4 packets are called "QQP packets". The only exception to this rule is data transport by means of IPv4 TCP; only in that case QQP packets are not used because TCP allows to transmit QQP resources directly without additional encapsulation or flow control. 2.6. Valid Data Transport Methods The following methods of transmitting QQP resources are considered valid data transport methods: (a) For each "AAAA" DNS record which exists for a host's FQDN, it is a valid data transport method to send QQP packets to the IPv6 Internet Protocol number given in that "AAAA" DNS record. (b) For any host which is a mobile node using Mobile IPv6 as specified in [RFC3775], it is a valid data transport method to send QQP packets to any of its current care-of addresses. (c) For each "A" DNS record which exists for a host's FQDN, it is a valid data transport method to send QQP packets encapsulated in IPv4 packets by means of the 6to4 technique defined in [RFC3056] to the IPv4 Internet Protocol number given in that "A" DNS record. In this case the IPv6 destination address must be constructed using 0x0000 as SLA ID and 0x0000001A as Interface ID (see the illustration in section 2 of [RFC3056].) (d) For each "A" DNS record which exists for a host's FQDN, it is a valid data transport method to send QQP packets encapsulated in IPv4-based UDP datagrams by means of the Teredo technique defined in [RFC4380] to the IPv4 Internet Protocol number given in that "A" DNS record, using UDP port number 26. (e) If no "A" or "AAAA" DNS record exists for a host's FQDN, and QQP packages with a valid ChannelID have been received from an IPv4 IP number and UDP port number by means of data transport method (d), then it is a valid data transport method to send Teredo-encapsulated QQP packets to that IPv4 IP number and UDP port number. (f) For each "A" DNS record which exists for a host's FQDN, it is a valid data transport method to send QQP resources via a TCP connection to the IPv4 Internet Protocol number given in that "A" DNS record, using TCP port number 26. (g) If no "A" or "AAAA" DNS record exists for a host's FQDN, and the host has opened an IPv4 TCP connection to the local TCP port 26 and identified itself with a valid Greeting QQP control resource, then it is a valid data transport method to send QQP resources via that TCP connection. QQP resources SHOULD NOT be transmitted by any other method except the seven valid data transport methods listed above. Each QQP host SHOULD support all of the above-defined valid data transport methods which are applicable to its situation. This means in particular that the burden for ensuring interoperability between IPv4 hosts and IPv6 hosts is placed on IPv4 hosts. The complexity which is related to the protocol transition and the data transport methods (d), (e), (f) and (g) which provide interoperability with IPv4 hosts behind NAT (Network Address Translation) will disappear when the transition to IPv6 is complete. 2.7. QQP Queues and QIDs The QQP protocol allows the concurrent transmission of multiple QQP resources in a given direction. For this purpose, QQP data resources are organized into "queues", each of which is identified by an 8 bit unsigned integer number in the range from 1 to 250. This number is called the "Queue IDentifier" or "QID". The QID values of 0, 254 and 255 have special meanings as follows: The values of 0 and 254 are used for probes and responses to probes as described in section 5 below. The QID value of 255 is used for transmitting QQP control resources. The QID is specific to a logical flow of QQP resources in a given direction. The same QID MAY also be used to identify a QQP Queue for transmission in the other direction, that is however not required. Any given QQP resource is considered to be part of a single queue. Even if QQP packets are retransmitted because they are presumed to have been lost, or if transmission of a QQP resource is interrupted and resumed later, the same QID MUST be used for all QQP packets that contain data belonging to a given QQP resource. However if a QQP resource is retransmitted in its entirety, a new QID MAY be used. It is possible for multiple data transport methods to be used for QQP packets belonging to the same queue. The most significant bit of the QID indicates whether reliable transmission of QQP Packets should be ensured by means of acknowledgment of received packets and retransmission of lost packets. If this bit is 0, received packets are not acknowledged and lost packets are not retransmitted. If this bit is 1, received packets are acknowledged and lost packets are retransmitted. 2.8. QQP Packet Numbers Each QQP host generates packet numbers which are unsigned 32 bit integer values for all QQP packets which it sends. All QQP queues with QIDs from 128 to 255 share the same namespace of packet numbers. Packet number 0 is invalid and SHOULD never be used. Each QQP host SHOULD use packet number 1 for the first packet it sends (this packet contains a Greeting resource, see below). As soon as a QQP host starts sending a QQP resource, it allocates consecutive packet numbers for all QQP packets that will be used to transmit this QQP resource, starting with the lowest positive integer value that has not yet been allocated to any QQP resource in the QQP queues with QIDs from 128 to 255. In each of the QQP queues with QIDs from 1 to 127, the packets are simply numbered consecutively in the order in which they are sent, starting with 1. 3. Setup, Timeout and Refreshing of QQP Communication Channels 3.1. Establishing QQP Communication Channels The QQP communication channel is established by means of a three-way handshake. In the case of data transmission methods (f) and (g) described in section 2.6 above, this three-way handshake is carried out in the TCP protocol. Other wise the following procedure is used for the three-way handshake: Host A sends a QQP packet to host B which contains a Greeting without any acknowledgment. Then host B responds by sending a QQP packet to host A which contains host B's Greeting as well as the acknowledgment for host A's Greeting. Then host A starts sending QQP data resources which include an acknowledgment for host B's Greeting. Both parties MUST wait with the transmission of QQP data resources until until after sending their Greeting and receiving the other party's Greeting. Since Greetings are QQP control resources, they are always sent with a QID of 255. Since the most significant bit of this QID is 1, it is clear that the Greetings will be acknowledged. Each Greeting contains an ActiveQueue field containing a QID for a QQP queue which is established by means of the Greeting and its acknowledgment. Hence after the initial three-way handshake, there are two QQP queues for transmitting QQP data resources, one for each direction. If it should happen that both hosts simultaneously initiate the three-way handshake, two three-way handshakes occur simultaneously. This is not an error condition. The result is still a single QQP communication channel, because both hosts will have computed the same ChannelID. 3.2. Refusal of QQP Communication Instead of acknowledging a QQP packet which contains a Greeting, a QQP host may also respond with a QQP_NoChannel resource, see section 6.2 blow. In this case, no QQP communication channel is established. 3.3. Active Status of QQP Queues The sending host may explicitly put a queue into inactive state by sending a QQP_CloseQueue QQP control resource for that queue, see section 6.3 below. Also, Each QQP Greeting contains a TimeOut field with a value which SHOULD NOT be greater than 24 hours. A given QQP queue SHOULD be considered inactive if the time which has elapsed since sending the most recent QQP Greeting for which an acknowledgment has been exceeds the TimeOut value specified by the QQP host which is the receiving host for that queue. When a queue becomes inactive, the receiving host MAY discard all QQP packets pertaining to QQP data resources in that queue which have not been transmitted completely, even if they have already been acknowledged. If the sending host does not wish the receiving host to do this, it SHOULD send a new Greeting for that queue well before the timeout. 3.4. Active Status of QQP Communication Channels Each QQP host considers a given QQP queue to be "active" if there is at least one active QQP Queues. When a communication channel becomes inactive, QQP hosts MAY discard all state information pertaining to that communication channel. 3.5. Establishing Additional QQP Queues Additional QQP Queues may be established by sending additional Greeting resources. Since these additional Greetings will contain a non-zero acknowledgment field, they do not result in a repetition of the three-way handshake for establishing QQP communication channels. 4. QQP Packet Format QQP packets consist of a header and payload. The header consists of 256 bits as follows: 128 bit 32bit 16b 16b 32bit 32bit +-----------------------------+-----------------------------+ |ChannelID | PCKID|QID|PC | ACK | RTRQ | +-----------------------------+-----------------------------+ The meaning of the fields is as follows: ChannelID is as defined in section 2.3 above PCKID ("PaCKage IDentifier) is the package number defined in section 2.8 above. QID is as defined in section 2.7 above. PC is a "package count" of the QQP packages for the same QQP resource which will follow after the present package. If this is the only package or the last of a number of packages used to transmit a QQP resource, this field is zero. The ACK (ACKnowledgement) field gives the lowest package number up to which all of the other party's packages have been received. If the RTRQ (ReTransmission ReQuest) field is non-zero, that indicates a request to retransmit the package with the indicated package number. The size of the payload is determined from the payload size of the IPv6 packet in which the QQP packet is encapsulated. If multiple packages are used to transmit a single QQP resource, then with the possible exception of the last of these packages, they MUST all have the same size. The last of these packages MAY be shorter, but it MUST NOT be longer. 5. Probing This section is still mostly TODO. The basic idea is to send a test packet with QID of 254 multiple times to the recipient, using different paths and packet sizes. This test packet specifies how long the recipient should wait for further copies after the first copy has been received. Each copy of the packet has a unique identifier, say in the range from 0 to 255. Then the recipient of the test packets sends back an array indicating which of the test packets arrived, and if so how much later than the first copy of the test packet. 6. QQP Control Resources 6.1. The QQP_Greeting Resource The Greeting is a signed SXDF resource containing a single dictionary named QQP_Greeting with values for the following fields, all of which are required: TimeStamp - The current time, in seconds since the beginning of the year 1970. MySourceAddress - IP Address from which this Greeting is sent. MaxResourceSize - A limit on the size of data items (which are QQP resources) that the Host which generated the Greeting will accept. MaxBufferSize - The sending host should assume that no more than this amount of memory is available for buffering incompletely-transmitted packets until their transmission is complete. MaxQueues - The other host should not send more than this number of data streams simultaneously. MaxSustainedRate - Maximal sustained transmission data rate that the host which generated the Greeting expects to be able to process. MaxBurstRate - Maximal transmission data rate that the host which generated the Greeting expects to be able to process during bursts of data. MaxBurstSize - Maximal size of such bursts, in bytes. MinMaxBurstPause - Minimal pause between two bursts of maximal size TimeOut - The TimeOut value discussed in section 3.3 above ActiveQueue - A QID for a queue which is made or kept active, see section 3.1 above. ExceptionsTo - A qrpc: URL where QRPC Exception resources with information about error conditions will be received. See section 8 for information on the format of such Exception resources. If the QQP host which receives this Greeting trusts the authenticity of the key that the Greeting is signed with, it SHOULD accept QQP packages coming from the IP address given in MySourceAddress until 24 hours after the time given in the TimeStamp. The security implications of this mechanism are discussed in section 9 below. The relationship between the size of bursts of non-maximal size and the pause which is required between such bursts is intended to be calculated by means of the token bucket algorithm with parameters corresponding to the values of MaxSustainedRate, MaxBurstSize and MinMaxBurstPause. 6.2. The QQP_NoChannel Resource The NoChannel resource is a QQP resource containing a single integer value named QQP_NoChannel. The value indicates a timespan in seconds which the other host should wait before attempting again to establish a QQP communication channel. 6.3. The QQP_CloseQueue Resource The QQP_CloseQueue resource is a QQP resource containing a single integer value named QQP_CloseQueue. Its value is a QID of a queue which is being closed; by sending this control resource, the sending host for this queue indicates that it intends to send no more data with the specified QID in the foreseeable future. 7. Format of QQP Data Resources QQP data resources resources consist of two elements named Envelope and Contents: An element named Envelope MUST be present; it SHOULD contain at least an Action element and an ExceptionsTo element. It MAY also contain a ResponseTo element. An element named Contents SHOULD also be present; it may contain arbitrary SXDF data. For example the Contents element MAY contain human-readable data in SXDF format, or its value MAY be merely a string of length zero, or its value MAY be a string containing encrypted data. A digital signature as described in section 6 of [SXDF] MAY be present; in this case the _DATA element which holds the signed data SHOULD contain both the Envelope and the Contents elements. 7.1. Envelope Elements 7.1.1. Action Element Sender SHOULD ensure that each QQP resource which it transmits contains an Action element with a value which is a valid URL. 7.1.2. ExceptionsTo Element The ExceptionsTo element specifies an qrpc: URL to which QRPC Exception resources related to the present QQP resource should be sent. 7.1.3. ResponseTo Element If present, the optional ResponseTo Element specifies the URL to which the specified Action will cause a response to be sent. If it is known that currently no data can be sent to this URL, the receiver of the resource MAY send a QRPC Exception resource to the ExceptionsTo URL instead of executing the requested Action. 7.2. Contents Element The precise format of the Contents element is not specified here, as it is irrelevant to QQP implementations. In fact the Contents element MAY contain encrypted binary data. 8. Format of Exception resources. This section is still TODO. 9. Security Considerations Of course digital signature SHOULD be used in all QQP Data Resources where protecting the integrity of the data is important. However this does not protect against the possible disruption of QQP communication channels by hostile third parties. The QQP protocol relies on IP addresses for protecting QQP communication channels against disruption or manipulation by third parties. These network segments SHOULD therefore be protected by means of packet filters which prevent unauthorized third parties from injecting forged QQP packets. Also, these IP addresses are trusted for 24 hours after the time of the TimeStamp in the Greeting. Therefore, the use of IP addresses which could possibly come under the control of hostile third parties before 24 hours have elapsed SHOULD be avoided. 10. IANA Considerations This specification has no requests for IANA. References Normative References [RFC793] POSTEL, J. (Ed.): Transmission Control Protocol RFC 793, September 1981. [RFC1738] Berners-Lee, T., Masinter, L., McCahill, M., Eds., "Uniform Resource Locators", RFC 1738. [SXDF] Bollow, N., "SXDF - Simple Extensible Data Format", work in progress. [QRPC] Bollow, N., "QRPC - Queueable Remote Procedure Calls", work in progress. [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2440] Callas, J., Donnerhacke, L., Finney, H., Thayer, R. "OpenPGP Message Format", RFC 2440 [RFC3056] Carpenter, B., Moore, K.: Connection of IPv6 Domains via IPv4 Clouds, RFC 3056, February 2001. [RFC2460] Deering, S., Hinden, R.: Internet Protocol, Version 6 (IPv6) Specification, RFC 2460, December 1998. [RFC4380] Huitema, C.: Teredo: Tunneling IPv6 over UDP through Network Address Translations (NATs). RFC 4380, February 2006. Informative References [RFC3775] Johnson, D., Perkins, C., Arkko, J.: Mobility Support in IPv6, RFC 3775, June 2004. [RFC2821] Klensin, J., Ed., "Simple Mail Transfer Protocol", RFC 2821. [RFC2822] Resnick, P., Ed., "Internet message format", RFC 2822. Authors' Address Norbert Bollow Weidlistrasse 18 CH-8624 Gruet Phone: +41 44 972 2059 EMail: nb@bollow.ch Full Copyright Statement Copyright (C) The Internet Society (2006). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society.