HTTP/1.1 200 OK Date: Mon, 08 Apr 2002 23:50:24 GMT Server: Apache/1.3.20 (Unix) Last-Modified: Mon, 07 Jul 1997 11:07:00 GMT ETag: "2e991c-635c-33c0cdd4" Accept-Ranges: bytes Content-Length: 25436 Connection: close Content-Type: text/plain Internet-Draft K. Evans Expires in 6 months J. Klein Tandem Computers J. Lyon Microsoft July 4 1997 Transaction Internet Protocol - Requirements Status of this Memo This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." To learn the current status of any Internet-Draft, please check the "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast). Distribution of this document is unlimited. Please send comments to the authors at , , or . Abstract This document describes the purpose (usage scenarios) and requirements for the Transaction Internet Protocol [1]. It is intended to help qualify the necessary features and functions of the protocol. Introduction Transactions are a very useful programming paradigm, greatly simplifying the writing of distributed applications. When transactions are employed, no matter how many distributed application components participate in a particular unit-of-work, the number of possible outcomes is reduced to only two; that is, either all of the work completed successfully, or none of it did (this characteristic is known as atomicity). Applications programming is therefore much less complex since the programmer does not have to deal with a multitude of possible failure scenarios. Typically, transaction semantics are provided by some underlying system infrastructure (usually in the form of products such as Transaction Processing Monitors, and/or Databases). This infrastructure deals with failures, and performs the necessary recovery actions to guarantee the property of atomicity. The use of transactions enables the development of reliable distributed applications which would otherwise be difficult, if not impossible. A key technology required to support distributed transactions is the two-phase commit protocol (2-pc). 2-pc protocols have been used in commercial Transaction Processing (TP) systems for many years, and are well understood (e.g. the LU6.2 2-pc (syncpoint) protocol was first implemented more than 12 years ago). Today a number of different 2-pc protocols are supported by a variety of TP monitor and database products. 2-pc is used between the components participating in a distributed unit-of-work (transaction) to ensure agreement by all parties regarding the outcome of that work (regardless of any failure). Today both standard and proprietary 2-pc protocols exist. These protocols typically employ a "one-pipe" model. That is, the transaction and application protocols are tightly-integrated, executing over the same communications channel. An application may use only the particular communications mechanism associated with the transaction protocol. The older standard protocols (OSI TP, LU6.2) are complex, with a large footprint and extensive configuration and administration requirements. For these reasons they are not very widely deployed. A new protocol, CORBA OTS, still adheres to the one-pipe model, and is tightly coupled with a particular application environment. The net of all this is restricted application flexibility and interoperability if transactions are to be used. Applications may wish to use a number of communications protocols for which there are no transactional variants (e.g. HTTP), and be deployed in very heterogeneous application environments. In summary, transactions greatly simplify the programming of distributed applications, and the 2-pc protocol is a key transactional technology. Current 2-pc protocols only offer transaction semantics to a limited set of applications, operating within a special-purpose (complex, homogeneous) infrastructure, using a particular set of intercommunication protocols. The restrictions thus imposed by current 2-pc protocols limits the widespread use of the transaction paradigm, thereby inhibiting the development of new distributed business applications. (See [2] for more information re transactions, atomicity, and two-phase commit protocols in general.) The Transaction Internet Protocol (TIP) TIP is a 2-pc protocol which is intended to provide ubiquitous distributed transaction support, in a heterogeneous (networked) environment. TIP removes the restrictions of current 2-pc protocols and enables the development of new distributed business applications. This goal is achieved primarily by satisfying two key requirements: 1. Keep the protocol simple (yet functionally sufficient). If the protocol is complex it will not be widely deployed or quickly adopted. Simplicity also means suitability to a wide range of application environments. 2. Enable the protocol to be used with any applications communications protocol (e.g. HTTP). This ensures heterogeneous environments can participate in distributed work. TIP does not reinvent the 2-pc protocol itself, the well-known presumed-abort 2-pc protocol is used as a basis. Rather the novelty and utility of TIP is in its separation from the application communications protocol (the two-pipe model). Scope Note that TIP does not describe how business transactions or electronic commerce are to be conducted on the internet, it specifies only the 2-pc transaction protocol (which is an aid in the development of such applications). TIP does not specify Application Programming Interfaces. Anticipated Usage of TIP As described above, transactions are a very useful tool in simplifying the programming of distributed applications. TIP is therefore targeted at any application that involves distributed work. Such applications may comprise components executing within a single system, across a corporate intranet, across the internet, or any other distributed system configuration. The application may be of "enterprise" class (requiring high-levels of performance and availability), or be less demanding. TIP is intended to be generally applicable, meeting the requirements of any application type which would benefit from the provision of transaction semantics. TIP Compliant Systems There are two classes of TIP compliant Transaction Manager system: 1. Client-only systems. Those which provide an application interface to demarcate TIP transactions, but which do not offer access to local recoverable resources. Such a lightweight implementation is useful for systems which host client applications only (e.g. desktop machines). Such client systems may be unreliable, and are not appropriate as transaction coordinators (their unavailability might cause resources on other transaction participant systems to remain locked and unavailable). These so-called "volatile client" systems therefore delegate the responsibility to coordinate the transaction (and recover from failures), to other "full" (server) TIP system implementations. For these lightweight systems, only the TIP IDENTIFY, BEGIN, COMMIT, and ABORT commands are needed; no transaction log is required. 2. Server systems. Those which offer the above support, plus TIP transaction coordination and recovery services. These systems may also provide access to recoverable resources (e.g. relational databases). Server systems support all TIP commands, and provide a recoverable transaction log. A TIP compliant Transaction Manager (TM), will also supply application programming interfaces to demarcate transactions (e.g. the X/Open TX interface [3]), plus commands to generate TIP URLs, to PUSH/PULL TIP transactions, and to set the current TIP transaction context. TIP support can be added to TMs with existing APIs and 2-pc protocols, and transactions may comprise both proprietary and TIP transaction branches (it is assumed existing TM implementations will provide "TIP gateway" facilities which will coordinate between TIP and other transaction protocols). Relationship to the X/Open DTP Model The X/Open Distributed Transaction Processing (DTP) Model [4] defines four components: 1) Application Program (AP), 2) Transaction Manager (TM), 3) Resource Manager (RM), and 4) Communications Resource Manager (CRM). In this model, TIP defines a TM to TM interoperability protocol, which is independent of application communications (there is no such equivalent protocol specified by X/Open, where all transaction and application communication occurs between CRMs (the one-pipe model)). Programmatic interfaces between the AP and TM/RM are unaffected by, and may be used with TIP. The TM to RM interaction is defined via the X/Open XA interface specification [5]. TIP is compatible with XA, and a TIP transaction may comprise applications accessing multiple RMs where the XA interface is being used to coordinate the RM transaction branches. Example TIP Usage Scenario It is expected that a typical internet usage of TIP will involve applications using the agency model. In this model, the client node itself is not directly involved in the TIP protocol at all, and does not need the services of a local TIP TM. Instead, an agency (server) application handles the dialogue with the client, and is responsible for the coordination of the TIP transaction. The agency works with other service providers to deliver the service to the client. e.g. as a Travel Agency acts as an intermediate between airlines/hotels/etc and the customer. A big benefit of this model is that the agency is trusted by the service providers, and there are fewer such agencies (compared to user clients), so issues of security and performance are reduced. Consider a Travel Agency example. A client running a web browser on a network PC accesses the Travel Agency web page. Via pages served up by the agency (which may in turn may be constructed from pages provided by the airline and hotel servers), the client creates an itinerary involving flights and hotel choices. Finally, the client clicks the "make reservation" button. At this point the following sequence of events occurs (user-written application code is invoked by the various web servers, via any of the standard or proprietary techniques available (e.g. CGI)): 1. The travel agency begins a local transaction, and gets a TIP URL for this transaction (both of these functions are performed using the API of the local TM). The TIP URL contains the listening endpoint IP address of the local TM and the transaction identifier of the current transaction. 2. The travel agency application sends a request to the airline server (via some protocol (e.g. HTTP) specified via some URL), requesting the "book_flight" service, passing the flights selected by the client, and the TIP URL (obtained in 1. above). 3. The request is received by the airline server which invokes the book_flight application. This application retrieves the TIP URL from the input data, and passes this on a "tip_pull" API request to its local TM. The tip_pull function causes the following to occur: a. the local TM creates a transaction branch (under which the work will be performed), b. if a TIP connection does not already exist to the superior (travel agency) TM (as identified via the IP address passed in the TIP URL), one is created and an IDENTIFY exchange occurs (if multiplexing is to be used on the connection, this is followed by a MULTIPLEX exchange), c. a PULL command is sent to the superior TM, d. in response to the PULL, the superior TM associates the subordinate (airline) TM with the transaction (by associating the connection with the transaction), e. the subordinate TM returns control to the book_flight application, which is now executing in the context of the newly created local transaction branch. 4. The book_flight application does its work (which may involve access to a recoverable resource manager (e.g. an RDBMS), in which case the local TM will associate the RM with the transaction branch (via the XA interface or whatever)). 5. The book_flight application returns to the travel agency application indicating success. 6. Steps 2-5 are then repeated with the hotel server "book_room" application. At the conclusion of this, the superior TM has registered two subordinate TMs as participants in the transaction, there are TIP connections between the agency TM and the airline and hotel TMs, and there are inflight transactions at the airline and hotel servers. [Note that steps 2-5 and 6 could be performed in parallel.] 7. The travel agency application issues a "commit transaction" request (using the API of the local TM). The local TM sends a PREPARE command on the TIP connections to the airline and hotel TMs (as these are registered as subordinate transaction participants). 8. The TMs at the airline and hotel servers perform the necessary steps to prepare their local recoverable resources (e.g. by issuing xa_prepare requests). If successful, the subordinate TMs change their TIP transaction state to Prepared, and log recovery information (e.g. local and superior transaction branch identifiers, and the IP address of the superior TM). The subordinate TMs then send PREPARED commands to the superior TM. 9. If both subordinates respond PREPARED, the superior TM logs that the transaction is Committed, with recovery information (e.g. local and subordinate transaction identifiers, and subordinate TM IP addresses). The superior TM then sends COMMIT commands on the two subordinate TIP connections. 10. The TMs at the airline and hotel servers perform the necessary steps to commit their local recoverable resources (e.g. by issuing xa_commit requests). The subordinate TMs forget the transaction. The subordinate TMs then send COMITTED commands to the superior TM. 11. The superior TM forgets the transaction. The TIP connections between the superior and subordinate TMs return to Idle state (not associated with any transaction). The superior TM returns success to the travel agency application "commit transaction" request. 12. The travel agency application returns "reservation made" to the client. TIP Transaction Recovery Until the transaction reaches the Prepared state, any failure results in the transaction being aborted. If an error occurs once the transaction has reached the Prepared state, then transaction recovery must be performed. Recovery behaviour is different for superior and subordinate; the details depend upon the outcome of the transaction (committed or aborted), and the precise point at which failure occurs. In the travel agency application for example, if the connection to the hotel server fails before the COMMIT command has been received by the hotel TM, then (once the connection is restored): 1. The superior (travel agency) TM sends a RECONNECT command (passing the subordinate transaction identifier (recovered from the transaction log if necessary)). 2. The subordinate (hotel) TM responds RECONNECTED (since it never received the COMMIT command, and still has the transaction in Prepared state (if the failure had occurred after the subordinate had responded COMMITTED, then the subordinate would have forgotten the transaction, and responded NOTRECONNECTED to the RECONNECT command)). 3. The superior TM sends a COMMIT command. The subordinate TM commits the transaction and responds COMMITTED. The transaction is now resolved. 4. If the subordinate TM restores the connection to the superior TM before receiving a RECONNECT command, then it may send a QUERY command. In this case, the superior TM will respond QUERIEDEXISTS, and the subordinate TM should wait for the superior to send a RECONNECT command. If the transaction had been aborted, then the superior may respond QUERIEDNOTFOUND, in which case the subordinate should abort the transaction (note that the superior is not obliged to send a RECONNECT command for an aborted transaction (i.e. it could just forget the transaction after sending ABORT and before receiving an ABORTED response)). Security The means by which applications communicate and perform distributed work are outside the scope of the TIP protocol. The mechanisms used for authentication and authorisation of clients to access programs and information on a particular system are part of the application communications protocol and the application execution infrastructure. Use of the TIP protocol does not affect these considerations. Security relates to the TIP protocol itself inasmuch that systems require to protect themselves from the receipt of unauthorised TIP commands, or the impersonation of a trusted partner TIP TM. Probably the worst consequence of this is the possibility of undetected data inconsistency resulting from violations of the TIP commitment protocol (e.g. a COMMIT command is injected on a TIP connection in place of an ABORT command). TIP implementations concerned about this kind of attack can use the Secure Socket Layer protocol [6] to restrict access to only trusted partners (i.e. to control from which remote endpoints TIP transactions will be accepted, and to verify that an end-point is genuine), and to encrypt TIP commands. TIP TM implementations will also likely provide local means to time-out and abort transactions which have not completed within some time period (thereby preventing unavailability of resources due to malicious intent). TIP Requirements Most of these requirements stem from the primary objective of making transactions a ubiquitous system service, available to all application classes (much as TCP may be assumed to be available everywhere). In general this requires imposing as few restrictions regarding the use of TIP as possible (applications should not be required to execute in some "special" environment in order to use transactions), and keeping the protocol simple and efficient. This enables the widespread implementation of TIP (it's cheap to do), on a wide range of systems (it's cheap to run). 1. Application Communications Protocol Independence The TIP protocol must be defined independently of the communications protocol used for transferring application data, to allow TIP usage in conjunction with any application protocol. It must be possible for applications using arbitrary communications protocols to begin, end, and propagate TIP transactions. This implies that the TIP protocol employ a 2-pipe model of operation. This model requires the separation of application communications and transaction coordination, into two discrete communication channels (pipes). This separation enables the use of the transaction coordination protocol (TIP), with any application communications protocol (e.g. HTTP, ODBC, plain TCP/UDP, etc). 2. Support for Transaction Semantics The TIP protocol must provide the functionality of the de-facto standard presumed-abort 2-pc protocol, to guarantee transactional atomicity even in the event of failure. It should provide a means to construct the transaction tree, as well as provide commitment and recovery functions. 3. Application Transaction Propagation and Interoperability In order to facilitate protocol independence, application interoperability, and provide a means for TIP transaction context propagation, a standard representation of the TIP transaction context information is required (in the form of a URL). This information must include the listening endpoint address of the partner TIP TM, and transaction identifier information. 4. Ease of Implementation The TIP protocol must be simple to implement. It should support only those features necessary to provide a useful, performant 2-pc protocol service. The protocol should not add complexity in the form of extraneous optimisations. 5. Suitability for All Application Classes The TIP protocol should be complete and robust enough not only for electronic commerce on the web, but also for intranet applications and for traditional TP applications spanning heterogenous transaction manager environments. The protocol should be performant and scaleable enough to meet the needs of low to very high throughput applications. - the TIP protocol should support the concept of client-only transaction participants (useful for ultra-lightweight implementations on low-end platforms) - since some clients may be unreliable, TIP must provide support for delegation of transaction coordination (to a more reliable (trusted) node) - the TIP protocol must scale between 1 and n (> 1) concurrent transactions per TCP connection - TIP commands should be able to be concatenated (pipelined) - TIP should be compatible with the X/Open XA interface 6. Security The TIP protocol must be compatible with existing security mechanisms, potentially including encryption, firewalls, and authorization mechanisms (e.g. SSL should be used to authenticate the sender of a TIP command, and for encryption of TIP commands). Nothing in the protocol definition should prevent TIP working within any security environment. 7. TIP Protocol Transport Independence It would be beneficial to some applications (though not mandatory) to define the TIP protocol independently of the transport for the TIP protocol itself. This would allow the TIP protocol to flow over multiple different transport protocols. The benefit is when using different transport protocols for the application data, the same transport can be used for the TIP 2PC protocol. 8. Recovery Recovery semantics need to be defined sufficiently to avoid ambiguous results in the event of any type of communications transport failure. References [1] Internet Draft "The Transaction Internet Protocol" J. Lyon et al. [2] Transaction Processing: Concepts and Techniques Morgan Kaufmann Publishers (ISBN 1-55860-190-2) J. Gray, A. Reuter. [3] X/Open CAE Specification, April 1995, Distributed Transaction Processing: The TX Specification (ISBN 1-85912-094-6) [4] X/Open Guide, November 1993, Distributed Transaction Processing: Reference Model Version 2 (ISBN 1-85912-019-9) [5] X/Open CAE Specification, December 1991, Distributed Transaction Processing: The XA Specification (ISBN 1-872630-24-3) [6] Internet Draft "The SSL Protocol Version 3.0" A. Freier, P. Karlton, P. Kocher. Authors' Addresses Jim Lyon Keith Evans Microsoft Corporation Tandem Computers Inc, LOC 252-30 One Microsoft Way 5425 Stevens Creek Blvd Redmond, WA 98052-6399, USA Santa Clara, CA 95051-7200, USA Phone: +1 (206) 936 0867 Phone: +1 (408) 285 5314 Fax: +1 (206) 936 7329 Fax: +1 (408) 285 5245 Email: JimLyon@Microsoft.Com Email: Evans_Keith@Tandem.Com Johannes Klein Tandem Computers Inc. 10555 Ridgeview Court Cupertino, CA 95014-0789, USA Phone: +1 (408) 285 0453 Fax: +1 (408) 285 9818 Email: Klein_Johannes@Tandem.Com