Internet DRAFT - draft-bollow-qqp
draft-bollow-qqp
Norbert Bollow
Internet-Draft Tenthnet Project
Expires: November 5, 2006 May 6, 2006
QQP - Quick Queues Protocol
<draft-bollow-qqp-02.txt>
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on November 5, 2006.
Copyright Notice
Copyright (C) The Internet Society (2006).
Abstract
The QQP protocol specifies a method for transmitting multiple
webservice requests and responses between two Hosts. QQP plays
the role of transport protocol for the QRPC webservice protocol,
similar to how XML-RPC and SOAP often use HTTP over TCP as
transport protocol.
QQP is called the "Quick Queues Protocol" because it is faster
than transmitting XML data via HTTP over TCP for several reasons.
QQP is also designed to handle Internet Protocol multihoming,
hiding IP address changes from application layer software without
need for introducing an additional protocol layer for that purpose.
1. Introduction
Over the past few years, "webservices" protocols such as for example
SOAP or XML-RPC have started becoming popular. The fundamental idea
of "webservices" is to access functionality on another computer by
means of standardized multi-purpose protocols and a standardized
extensible data format. For example, when using the "webservices"
paradigm to specify a message transport system, you don't specify
the protocol and data format from scratch like it is done for SMTP in
[RFC2821] and [RFC2822]; instead you specify them in terms of a
general-purpose data format (often XML) and a general-purpose
protocol for transporting data between computers.
QQP, the Quick Queues Protocol, is such a general data transport
protocol, specifically for data in the SXDF format [SXDF]. As far
as QQP is concerned, there is no difference between webservice
requests and responses; QQP merely provides something that could be
called a "full-duplex webservice data communication channel" between
two Hosts, which both parties can use to transmit any number of
webservice requests and responses.
The data which is transmitted over QQP connections consists of
"QQP resources" which are SXDF resources consisting of an element
named Envelope (which contains QQP-related and routing information)
and an element named Data (which can contain arbitrary payload).
In particular, the Envelope contains an Element named Action which
specifies what the receiver should do with the resource.
One possible use of QQP is to use a single TCP [RFC793] connection
can be used for all webservice requests and responses between a given
pair of Hosts as long as the amount of data in each request or
response is small.
This is advantageous because of the various overheads incurred by
creating a new connection, which include
* The tree-way handshake for establishing a TCP connection; the
time required for this is bounded from below by the network's
Round-Trip Time.
* The initially slow speed of TCP connections, which is necessary
to dynamically maximize throughput without risking to overload
the network.
* For encrypted connections, the overhead of authentication and
session key negotiation.
The above-described method of using QQP over TCP is however only
intended for situations where the preferred method of sending
QQP data chunks directly as the payload of IPv6 packets is not
available, either because one of the hosts is an IPv4 host behind
NAT, or because the operating system of one of the hosts does not
provide QQP support, and the application program works around this
shortcoming by using QQP over one or more TCP connections.
Creating several separate TCP connections between the same pair
of hosts can be desirable when requests and responses which contain
large amounts of data: With separate TCP connections the transfer
of large QQP resources does not prevent the the concurrent transfer
of other data via QQP. When several TCP connections are created,
for each TCP connection there will be a "queue" of QQP resources
which await transmission via that particular TCP connection.
In the preferred situation, where the use of TCP is avoided and
the QQP resources are transmitted as chunks as the payload of IPv6
packets [RFC2460], each of the hosts which communicates via QQP
must nevertheless maintain a list of queues of QQP resources. In
each queue, QQP resources are transmitted one-after-the-other.
If the underlying network infrastructure supports QoS guarantees
which are based on "flows" in the sense of [RFC2460], QQP
implementations MAY make use of this functionality to provide
QoS guarantees to QQP-using application programs. In this case,
each [RFC2460] "flow" SHOULD be dedicated to one or more QQP
queues, and as long as the QoS guarantees of that [RFC2460]
flow are not violated, all data from the corresponding queues
SHOULD be sent exclusively via that flow.
Hosts which are not on IPv6 networks SHOULD work around that
problem by means of one of the following three methods:
(a) Encapsulating the QQP IPv6 packets in IPv4 packets by means of
the 6to4 technique defined in [RFC3056]. This methods requires
QQP support in the host's Internet Protocol Implementation, so
that the special case of such IPv4 packets which contain IPv6
QQP packets is recognized and handled appropriately.
(b) Encapsulating the QQP IPv6 packets in IPv4-based UDP datagrams,
by means of the Teredo technique defined in [RFC4380]. This
can work via a general-purpose Teredo gateway, or via a
special-purpose gateway on a QQP server listing on UDP port 26.
(c) Sending the data via a TCP connection to TCP port 26 on the
QQP server.
2. Fundamental Definitions
2.1. QQP Hosts and FQDNs
As far as the QQP protocol is concerned, internet hosts are
identified by means of fully-qualified domain names (FQDNs).
It is possible for a single physical machine to have more than one
FQDN and to use multiple FQDNs at the same time for communicating
via QQP, but as far as the QQP protocol is concerned, these
different FQDNs correspond to different (logical) hosts, called
"QQP hosts" in the following.
2.2. Communication Channels
QQP defines a "communication channel" between two hosts which is
identified by an unordered pair of fully-qualified domain names,
with each of these FQDNs identifying one of the hosts participating
in the communication channel.
2.3. ChannelID
Each QQP communication channel is characterized by a unique
ChannelID which is a 128-bit hash computed as follows from the
FQDNs of the two hosts between whom the communication channel is
defined:
A string is computed from the two FQDNs by comparing the FQDNs
alphabetically, putting that FQDN first which precedes the other
in the alphabetical order, then an ASCII LF character, then the
other FQDN, then another ASCII LF character. Then an MD5 hash
is computed of this string which consists of the two FQDNs and
the two LF characters. This MD5 hash is the ChannelID.
2.4. QQP Resources
QQP achieves its purpose by transmitting SXDF resources between
the two hosts between which the communication channel has been
defined. These SXDF resources which are transmitted by means
of QQP are also called "QQP resources". This specification
distinguishes between two types of such QQP resources, namely
"QQP control resources" and "QQP data resources". Here "QQP
data resources" are payload such as webservice requests and
responses, while "QQP control resources" are special SXDF
resources defined in section 6 of this specification.
2.5. QQP Packets
This specification defines a number of data transport methods
for transmitting QQP resources. In most of these data transport
methods, the QQP resources are embedded in IPv6 packets (which
may in turn be embedded in IPv4 packets or IPv4 UDP datagrams).
These IPv4 packets are called "QQP packets". The only exception
to this rule is data transport by means of IPv4 TCP; only in that
case QQP packets are not used because TCP allows to transmit QQP
resources directly without additional encapsulation or flow control.
2.6. Valid Data Transport Methods
The following methods of transmitting QQP resources are
considered valid data transport methods:
(a) For each "AAAA" DNS record which exists for a host's FQDN, it
is a valid data transport method to send QQP packets to the IPv6
Internet Protocol number given in that "AAAA" DNS record.
(b) For any host which is a mobile node using Mobile IPv6 as
specified in [RFC3775], it is a valid data transport method to
send QQP packets to any of its current care-of addresses.
(c) For each "A" DNS record which exists for a host's FQDN, it
is a valid data transport method to send QQP packets encapsulated
in IPv4 packets by means of the 6to4 technique defined in
[RFC3056] to the IPv4 Internet Protocol number given in that "A"
DNS record. In this case the IPv6 destination address must be
constructed using 0x0000 as SLA ID and 0x0000001A as Interface ID
(see the illustration in section 2 of [RFC3056].)
(d) For each "A" DNS record which exists for a host's FQDN, it
is a valid data transport method to send QQP packets encapsulated
in IPv4-based UDP datagrams by means of the Teredo technique
defined in [RFC4380] to the IPv4 Internet Protocol number given
in that "A" DNS record, using UDP port number 26.
(e) If no "A" or "AAAA" DNS record exists for a host's FQDN, and
QQP packages with a valid ChannelID have been received from an
IPv4 IP number and UDP port number by means of data transport
method (d), then it is a valid data transport method to send
Teredo-encapsulated QQP packets to that IPv4 IP number and UDP
port number.
(f) For each "A" DNS record which exists for a host's FQDN, it
is a valid data transport method to send QQP resources via a TCP
connection to the IPv4 Internet Protocol number given in that "A"
DNS record, using TCP port number 26.
(g) If no "A" or "AAAA" DNS record exists for a host's FQDN, and the
host has opened an IPv4 TCP connection to the local TCP port 26
and identified itself with a valid Greeting QQP control resource,
then it is a valid data transport method to send QQP resources
via that TCP connection.
QQP resources SHOULD NOT be transmitted by any other method except
the seven valid data transport methods listed above. Each QQP host
SHOULD support all of the above-defined valid data transport methods
which are applicable to its situation. This means in particular
that the burden for ensuring interoperability between IPv4 hosts and
IPv6 hosts is placed on IPv4 hosts. The complexity which is related
to the protocol transition and the data transport methods (d), (e),
(f) and (g) which provide interoperability with IPv4 hosts behind NAT
(Network Address Translation) will disappear when the transition to
IPv6 is complete.
2.7. QQP Queues and QIDs
The QQP protocol allows the concurrent transmission of multiple
QQP resources in a given direction. For this purpose, QQP data
resources are organized into "queues", each of which is identified by
an 8 bit unsigned integer number in the range from 1 to 250. This
number is called the "Queue IDentifier" or "QID". The QID values of
0, 254 and 255 have special meanings as follows: The values of 0 and
254 are used for probes and responses to probes as described in
section 5 below. The QID value of 255 is used for transmitting
QQP control resources.
The QID is specific to a logical flow of QQP resources in a given
direction. The same QID MAY also be used to identify a QQP Queue
for transmission in the other direction, that is however not
required.
Any given QQP resource is considered to be part of a single queue.
Even if QQP packets are retransmitted because they are presumed to
have been lost, or if transmission of a QQP resource is interrupted
and resumed later, the same QID MUST be used for all QQP packets
that contain data belonging to a given QQP resource. However if a
QQP resource is retransmitted in its entirety, a new QID MAY be used.
It is possible for multiple data transport methods to be used for
QQP packets belonging to the same queue.
The most significant bit of the QID indicates whether reliable
transmission of QQP Packets should be ensured by means of
acknowledgment of received packets and retransmission of lost
packets. If this bit is 0, received packets are not acknowledged
and lost packets are not retransmitted. If this bit is 1, received
packets are acknowledged and lost packets are retransmitted.
2.8. QQP Packet Numbers
Each QQP host generates packet numbers which are unsigned 32 bit
integer values for all QQP packets which it sends.
All QQP queues with QIDs from 128 to 255 share the same namespace
of packet numbers. Packet number 0 is invalid and SHOULD never be
used. Each QQP host SHOULD use packet number 1 for the first packet
it sends (this packet contains a Greeting resource, see below).
As soon as a QQP host starts sending a QQP resource, it allocates
consecutive packet numbers for all QQP packets that will be used to
transmit this QQP resource, starting with the lowest positive integer
value that has not yet been allocated to any QQP resource in the QQP
queues with QIDs from 128 to 255.
In each of the QQP queues with QIDs from 1 to 127, the packets are
simply numbered consecutively in the order in which they are sent,
starting with 1.
3. Setup, Timeout and Refreshing of QQP Communication Channels
3.1. Establishing QQP Communication Channels
The QQP communication channel is established by means of a
three-way handshake. In the case of data transmission methods (f)
and (g) described in section 2.6 above, this three-way handshake
is carried out in the TCP protocol. Other wise the following
procedure is used for the three-way handshake:
Host A sends a QQP packet to host B which contains a Greeting
without any acknowledgment. Then host B responds by sending a
QQP packet to host A which contains host B's Greeting as well
as the acknowledgment for host A's Greeting. Then host A starts
sending QQP data resources which include an acknowledgment for
host B's Greeting.
Both parties MUST wait with the transmission of QQP data resources
until until after sending their Greeting and receiving the other
party's Greeting.
Since Greetings are QQP control resources, they are always sent with
a QID of 255. Since the most significant bit of this QID is 1, it is
clear that the Greetings will be acknowledged.
Each Greeting contains an ActiveQueue field containing a QID for a
QQP queue which is established by means of the Greeting and its
acknowledgment. Hence after the initial three-way handshake, there
are two QQP queues for transmitting QQP data resources, one for each
direction.
If it should happen that both hosts simultaneously initiate the
three-way handshake, two three-way handshakes occur simultaneously.
This is not an error condition. The result is still a single QQP
communication channel, because both hosts will have computed the
same ChannelID.
3.2. Refusal of QQP Communication
Instead of acknowledging a QQP packet which contains a Greeting, a
QQP host may also respond with a QQP_NoChannel resource, see section
6.2 blow. In this case, no QQP communication channel is established.
3.3. Active Status of QQP Queues
The sending host may explicitly put a queue into inactive state
by sending a QQP_CloseQueue QQP control resource for that queue,
see section 6.3 below.
Also, Each QQP Greeting contains a TimeOut field with a value which
SHOULD NOT be greater than 24 hours. A given QQP queue SHOULD be
considered inactive if the time which has elapsed since sending the
most recent QQP Greeting for which an acknowledgment has been
exceeds the TimeOut value specified by the QQP host which is the
receiving host for that queue.
When a queue becomes inactive, the receiving host MAY discard all
QQP packets pertaining to QQP data resources in that queue which have
not been transmitted completely, even if they have already been
acknowledged.
If the sending host does not wish the receiving host to do this, it
SHOULD send a new Greeting for that queue well before the timeout.
3.4. Active Status of QQP Communication Channels
Each QQP host considers a given QQP queue to be "active" if there
is at least one active QQP Queues.
When a communication channel becomes inactive, QQP hosts MAY discard
all state information pertaining to that communication channel.
3.5. Establishing Additional QQP Queues
Additional QQP Queues may be established by sending additional
Greeting resources. Since these additional Greetings will contain
a non-zero acknowledgment field, they do not result in a repetition
of the three-way handshake for establishing QQP communication
channels.
4. QQP Packet Format
QQP packets consist of a header and payload. The header consists
of 256 bits as follows:
128 bit 32bit 16b 16b 32bit 32bit
+-----------------------------+-----------------------------+
|ChannelID | PCKID|QID|PC | ACK | RTRQ |
+-----------------------------+-----------------------------+
The meaning of the fields is as follows:
ChannelID is as defined in section 2.3 above
PCKID ("PaCKage IDentifier) is the package number defined in
section 2.8 above.
QID is as defined in section 2.7 above.
PC is a "package count" of the QQP packages for the same
QQP resource which will follow after the present package.
If this is the only package or the last of a number of
packages used to transmit a QQP resource, this field is zero.
The ACK (ACKnowledgement) field gives the lowest package number
up to which all of the other party's packages have been received.
If the RTRQ (ReTransmission ReQuest) field is non-zero, that
indicates a request to retransmit the package with the
indicated package number.
The size of the payload is determined from the payload size of
the IPv6 packet in which the QQP packet is encapsulated.
If multiple packages are used to transmit a single QQP resource,
then with the possible exception of the last of these packages,
they MUST all have the same size. The last of these packages
MAY be shorter, but it MUST NOT be longer.
5. Probing
This section is still mostly TODO. The basic idea is to send a
test packet with QID of 254 multiple times to the recipient, using
different paths and packet sizes. This test packet specifies how
long the recipient should wait for further copies after the first
copy has been received. Each copy of the packet has a unique
identifier, say in the range from 0 to 255. Then the recipient of
the test packets sends back an array indicating which of the test
packets arrived, and if so how much later than the first copy of
the test packet.
6. QQP Control Resources
6.1. The QQP_Greeting Resource
The Greeting is a signed SXDF resource containing a single dictionary
named QQP_Greeting with values for the following fields, all of which
are required:
TimeStamp - The current time, in seconds since the beginning
of the year 1970.
MySourceAddress - IP Address from which this Greeting is sent.
MaxResourceSize - A limit on the size of data items (which are QQP
resources) that the Host which generated the
Greeting will accept.
MaxBufferSize - The sending host should assume that no more than
this amount of memory is available for buffering
incompletely-transmitted packets until their
transmission is complete.
MaxQueues - The other host should not send more than this
number of data streams simultaneously.
MaxSustainedRate - Maximal sustained transmission data rate that
the host which generated the Greeting expects to
be able to process.
MaxBurstRate - Maximal transmission data rate that the host
which generated the Greeting expects to be able
to process during bursts of data.
MaxBurstSize - Maximal size of such bursts, in bytes.
MinMaxBurstPause - Minimal pause between two bursts of maximal size
TimeOut - The TimeOut value discussed in section 3.3 above
ActiveQueue - A QID for a queue which is made or kept active,
see section 3.1 above.
ExceptionsTo - A qrpc: URL where QRPC Exception resources
with information about error conditions will be
received. See section 8 for information on the
format of such Exception resources.
If the QQP host which receives this Greeting trusts the authenticity
of the key that the Greeting is signed with, it SHOULD accept QQP
packages coming from the IP address given in MySourceAddress until 24
hours after the time given in the TimeStamp. The security
implications of this mechanism are discussed in section 9 below.
The relationship between the size of bursts of non-maximal size and
the pause which is required between such bursts is intended to be
calculated by means of the token bucket algorithm with parameters
corresponding to the values of MaxSustainedRate, MaxBurstSize and
MinMaxBurstPause.
6.2. The QQP_NoChannel Resource
The NoChannel resource is a QQP resource containing a single integer
value named QQP_NoChannel. The value indicates a timespan in seconds
which the other host should wait before attempting again to establish
a QQP communication channel.
6.3. The QQP_CloseQueue Resource
The QQP_CloseQueue resource is a QQP resource containing a single
integer value named QQP_CloseQueue. Its value is a QID of a queue
which is being closed; by sending this control resource, the sending
host for this queue indicates that it intends to send no more data
with the specified QID in the foreseeable future.
7. Format of QQP Data Resources
QQP data resources resources consist of two elements named Envelope
and Contents:
An element named Envelope MUST be present; it SHOULD contain at least
an Action element and an ExceptionsTo element. It MAY also contain a
ResponseTo element.
An element named Contents SHOULD also be present; it may contain
arbitrary SXDF data. For example the Contents element MAY contain
human-readable data in SXDF format, or its value MAY be merely a
string of length zero, or its value MAY be a string containing
encrypted data.
A digital signature as described in section 6 of [SXDF] MAY be
present; in this case the _DATA element which holds the signed data
SHOULD contain both the Envelope and the Contents elements.
7.1. Envelope Elements
7.1.1. Action Element
Sender SHOULD ensure that each QQP resource which it transmits
contains an Action element with a value which is a valid URL.
7.1.2. ExceptionsTo Element
The ExceptionsTo element specifies an qrpc: URL to which QRPC
Exception resources related to the present QQP resource should be
sent.
7.1.3. ResponseTo Element
If present, the optional ResponseTo Element specifies the URL to
which the specified Action will cause a response to be sent. If it
is known that currently no data can be sent to this URL, the receiver
of the resource MAY send a QRPC Exception resource to the
ExceptionsTo URL instead of executing the requested Action.
7.2. Contents Element
The precise format of the Contents element is not specified here,
as it is irrelevant to QQP implementations. In fact the Contents
element MAY contain encrypted binary data.
8. Format of Exception resources.
This section is still TODO.
9. Security Considerations
Of course digital signature SHOULD be used in all QQP Data Resources
where protecting the integrity of the data is important. However
this does not protect against the possible disruption of QQP
communication channels by hostile third parties.
The QQP protocol relies on IP addresses for protecting QQP
communication channels against disruption or manipulation by
third parties. These network segments SHOULD therefore be protected
by means of packet filters which prevent unauthorized third parties
from injecting forged QQP packets.
Also, these IP addresses are trusted for 24 hours after the
time of the TimeStamp in the Greeting. Therefore, the use of
IP addresses which could possibly come under the control of hostile
third parties before 24 hours have elapsed SHOULD be avoided.
10. IANA Considerations
This specification has no requests for IANA.
References
Normative References
[RFC793] POSTEL, J. (Ed.): Transmission Control Protocol
RFC 793, September 1981.
[RFC1738] Berners-Lee, T., Masinter, L., McCahill, M.,
Eds., "Uniform Resource Locators", RFC 1738.
[SXDF] Bollow, N., "SXDF - Simple Extensible Data
Format", work in progress.
<http://SXDF.org/>
[QRPC] Bollow, N., "QRPC - Queueable Remote Procedure
Calls", work in progress.
<http://QRPC.org/>
[KEYWORDS] Bradner, S., "Key words for use in RFCs to
Indicate Requirement Levels", BCP 14, RFC 2119,
March 1997.
[RFC2440] Callas, J., Donnerhacke, L., Finney, H.,
Thayer, R. "OpenPGP Message Format", RFC 2440
[RFC3056] Carpenter, B., Moore, K.: Connection of IPv6
Domains via IPv4 Clouds, RFC 3056, February 2001.
[RFC2460] Deering, S., Hinden, R.: Internet Protocol,
Version 6 (IPv6) Specification, RFC 2460,
December 1998.
[RFC4380] Huitema, C.: Teredo: Tunneling IPv6 over UDP
through Network Address Translations (NATs).
RFC 4380, February 2006.
Informative References
[RFC3775] Johnson, D., Perkins, C., Arkko, J.:
Mobility Support in IPv6, RFC 3775, June 2004.
[RFC2821] Klensin, J., Ed., "Simple Mail Transfer
Protocol", RFC 2821.
[RFC2822] Resnick, P., Ed., "Internet message format",
RFC 2822.
Authors' Address
Norbert Bollow
Weidlistrasse 18
CH-8624 Gruet
Phone: +41 44 972 2059
EMail: nb@bollow.ch
Full Copyright Statement
Copyright (C) The Internet Society (2006).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on
an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT
THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR
ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A
PARTICULAR PURPOSE.
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Acknowledgment
Funding for the RFC Editor function is currently provided by the
Internet Society.
