Network Working Group J. Ibbotson Internet-Draft R. King Expires: May 13, 2002 IBM November 12, 2001 Requirements for reliable message delivery draft-ibbotson-reliable-messaging-reqts-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on May 13, 2002. Copyright Notice Copyright (C) The Internet Society (2001). All Rights Reserved. Abstract This memo identifies a set of requirements for the reliable delivery of messages over the Internet. It defines what is meant by reliable delivery between two software agents and what requirements this places on an inter-agent protocol to ensure reliable message delivery between them. Ibbotson & King Expires May 13, 2002 [Page 1] Internet-Draft Requirements for reliable message delivery November 2001 1. Reliable delivery of messages The use of HTTP/1.1 as defined by RFC2616 is standard within the Internet. In addition to interactions between a browser and a web server, it is increasingly being used as the protocol for message exchanges as part of business transactions in both business-to- consumer and business-to-business scenarios. However, HTTP/1.1 is not a reliable protocol; the failure or non-operation of a software component at either end of the network link or failure of the link itself will result in a client being in doubt as to whether an interaction completed successfully or not. (The reliability of TCP [RFC793], over which HTTP runs, does not eliminate the possibility of connection failure, let alone failures in the client, in the server, or in any proxies or other intermediaries between the two.) If the failure occurred after the request was processed by the server, but before the acknowledgement could reach the client, then the client must not resend the request, as that would cause duplicate processing. On the other hand, if the failure occurred before the server received a complete, intact request, then the client must resend the request, as otherwise it will be lost. In a browser-based interaction driven by a human, where most actions are idempotent, recovery actions may be managed by the human operator and doubts may be resolved by resubmitting the transaction or manually performing a query to see if the transaction completed. In automated, business-to-business interactions, correct recovery becomes critical and has to be managed by application-level software which may be part of a distributed business process. Thus, if a fault occurs during the delivery of a single message or batch of messages, the client and server must be able to re-synchronise automatically so that any in-doubt states can be resolved. The requirements for reliable messaging are: 1.1 Message Delivery In addition to ensuring that no messages are lost, reliable message delivery must also ensure that no duplicates are received at the server. For reliable delivery, exactly-once semantics must be supported. 1.2 Message Ordering When a sequence of messages is sent between a client and server, the order of the messages must be preserved at the server. Ibbotson & King Expires May 13, 2002 [Page 2] Internet-Draft Requirements for reliable message delivery November 2001 1.3 Multiple Hops In addition to requiring reliable message delivery over a single hop, applications may be connected by a multi-hop environment. Any single-hop protocol must be capable of operating within a multi-hop environment where the multiple hops are reduced to cascaded single hops. It is assumed that each intermediary will store and forward the message reliably between each hop. 1.4 Compatibility, Migration and Coexistence A natural evolutionary path must be offered for entities that currently send and receive requests (i.e., messages) only using HTTP. The existing network structure, with its firewalls and proxies must be taken into account. The agents engaged in reliable messaging must coexist with servers and services that are already in place, so that both the old and new messaging mechanisms can operate without interference. Ibbotson & King Expires May 13, 2002 [Page 3] Internet-Draft Requirements for reliable message delivery November 2001 2. Operations for reliable delivery In messaging systems, the terms client and server are rarely used since a peer to peer relationship between the endpoints is more appropriate. In this document, a client will be the software agent that initiates a reliable message delivery. The server will respond to the client's request. This usage of the terms client and server is similar to HTTP/1.1. For reliable delivery of messages between a client and server we require: 1. A description of the state machines operating at the client and server. 2. A protocol which allows state information to be moved between the client and server allowing the state machines to cooperate. These may be described in an abstract manner as a set of operations that are required for reliable message delivery to take place between a client and server. 2.1 Capability Negotiation The client and server must be aware of each others capabilities to support reliable delivery. These capabilities govern the way messages are exchanged between the client and server. They indicate limits on timeouts, message and batch sizes, and which operations are permitted for message transmission. For example, some servers may be configured to only receive messages, so that only messages pushed from a client will be accepted. A set of default values can be assigned for the capabilities. A negotiation operation that allows a client and server to agree on their delivery capabilities is required before any reliable message exchange can take place between parties that cannot support the default values, or that would, for example, show a performance benefit from other values. 2.2 Message Push The message push operation allows a client to reliably send a single or batch of messages to a server. Each batch of messages should be uniquely identified with a transaction identifier to allow resynchronization of the delivery in the case of system or network failure. Ibbotson & King Expires May 13, 2002 [Page 4] Internet-Draft Requirements for reliable message delivery November 2001 2.3 Message Pull The message pull operation allows a client to ask a server to send it any messages waiting for delivery to applications located at the client. The response to a message pull operation may be empty if the server has nothing waiting for delivery to the client or a batch of one or more messages. The batch of messages returned to the client is uniquely identified with a transaction identifier. 2.4 Message Exchange The message exchange operation combines a push and pull into a single operation. It allows a client to push a batch of messages to the server and then invites any messages waiting on the server to be returned to the client. Both outgoing and returning message batches are uniquely identified with transaction identifiers generated by their respective sources. 2.5 State Reporting The state reporting operation allows a client to report to a server exactly which messages it has received and stored, and to ask the server to send back similar information about messages the client had pushed to the server. For example, when a system or network failure interrupts a message pull operation, the client would report its current state, thereby allowing the server to determine which messages had arrived safely at the client. This allows the client to resynchronize and resend any messages not received by the server due to the failure. For exactly-once delivery it permits a client to rollback its state on messages it has sent but (according to the state resolution) have not been received and stored persistently by the server. Ibbotson & King Expires May 13, 2002 [Page 5] Internet-Draft Requirements for reliable message delivery November 2001 3. Further requirements for reliable delivery The following set of requirements have also been identified for a reliable message delivery protocol. 3.1 Relationship to HTTP A reliable message delivery protocol which operates in an environment based on internet standards must be defined as a set of request/response messages as defined by HTTP/1.1. 3.2 Messaging Service URI The reliable messaging service represented by a particular software agent must be represented by a URI as defined by RFC2396. 3.3 Units of Work As soon as a batch of messages is sent, the sender is in doubt as to its safe arrival. The sender must record these doubts in some kind of persistent storage, using transactional mechanisms sufficient to guarantee that failure of the sender, regardless of when, will not lead to loss or duplication of messages; the receiver has similar problems. The nature of these mechanisms is beyond the scope of this protocol. Still, the protocol must take these actions into account, especially as they relate to communication between sender and receiver to provide resolution of failures, to determine which, if any, messages require retransmission. Thus, the protocol must call for exchange of information on identified units of work or transactions at appropriate times, and must describe the kinds of information that must be stored if correct functioning of the protocol is to be achieved, even if it doesn't specify how those transactions are actually performed. 3.4 Message Ordering A sequence of messages sent from one application to another via a reliable delivery protocol must be received in exactly the same order as they were sent. To preserve ordering across a network, there must only be one path for the messages. If there are multiple paths, then ordering may be lost. 3.5 Session Support Capability negotiation has sufficient cost that unnecessary renegotiation should be avoided. However, the vagaries of HTTP request processing, with the possibility of different TCP/IP connections being routed to different origin servers, makes reuse Ibbotson & King Expires May 13, 2002 [Page 6] Internet-Draft Requirements for reliable message delivery November 2001 difficult. It would be a beneficial option to support the association of a session identifier with a set of interactions such that negotiated capabilities would apply to all of them, even across multiple TCP/IP connections. 3.6 Data Conversion Data conversion should always be the responsibility of the software agent that receives the message. In the case of the pull operation, this will be the client. In the case of the push operation, this will be the server. The data conversion mechanism will take advantage of any encoding or content type information provided as part of the received message. 3.7 Security A reliable message delivery protocol should take advantage of underlying security mechanisms such as HTTPS to achieve point-to- point authentication and privacy for the messages. Such a protocol should not make provision for end-to-end authentication and privacy where a message flows over a multi-hop infrastructure. Support for these security features should be separate to the design of the reliable delivery protocol. Ibbotson & King Expires May 13, 2002 [Page 7] Internet-Draft Requirements for reliable message delivery November 2001 4. Relationship to Other Protocols SCTP [RFC2960 [1]] is a transmission protocol intended to provide reliability similar to TCP while overcoming some of TCP's limitations. Some applications, for example, don't need the ordering guarantees provided by TCP, while others want greater reliability through the use of multi-homed server groups. SCTP does not, however, address problems of recovery that span total server failures, and does not guarantee freedom from the possibility of the duplication of application messages. BEEP [RFC3080 [2]] provides connection-oriented, asynchronous request/response interactions, but lacks the generality necessary for arbitrary application messaging. Reliable messaging also requires consideration of connection failure and use of persistent storage, which are ignored by BEEP, as it only deals with what goes on within a single BEEP session, which takes place during a single TCP/IP connection. Ibbotson & King Expires May 13, 2002 [Page 8] Internet-Draft Requirements for reliable message delivery November 2001 References [1] Stewart, R., "Stream Control Transmission Protocol", RFC 2960, October 2000. [2] Rose, M., "The Blocks Extensible Exchange Protocol Core", RFC 3080, March 2001. Authors' Addresses John B Ibbotson IBM UK Ltd Hursley Park Winchester SO21 2JN UK Phone: +44 (0)1962 815188 Fax: +44 (0)1962 816898 EMail: john_ibbotson@uk.ibm.com Richard P King IBM Research 30 Saw Mill River Road Hawthorne NY 10532 Phone: +1 914 784 7245 EMail: rpk@us.ibm.com Ibbotson & King Expires May 13, 2002 [Page 9] Internet-Draft Requirements for reliable message delivery November 2001 Full Copyright Statement Copyright (C) The Internet Society (2001). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. Ibbotson & King Expires May 13, 2002 [Page 10]