Open Pluggable Edge Services A. Rousskov Internet-Draft The Measurement Factory Expires: February 25, 2004 August 27, 2003 OPES Callout Protocol Core draft-ietf-opes-ocp-core-01 Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http:// www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on February 25, 2004. Copyright Notice Copyright (C) The Internet Society (2003). All Rights Reserved. Abstract This document specifies Open Pluggable Edge Services (OPES) Callout Protocol (OCP). OCP is an application-agnostic protocol that facilitates exchange of application messages between an OPES processor and a callout server, for the purpose of adaptation of application messages at the callout server. Rousskov Expires February 25, 2004 [Page 1] Internet-Draft OPES Callout Protocol Core August 2003 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1 Application proxies and OCP scope . . . . . . . . . . . . . 4 1.2 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Protocol Development Status . . . . . . . . . . . . . . . . 6 2. Overall Operation . . . . . . . . . . . . . . . . . . . . . 8 2.1 Initialization . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 Original Dataflow . . . . . . . . . . . . . . . . . . . . . 8 2.3 Adapted Dataflow . . . . . . . . . . . . . . . . . . . . . . 8 2.4 Termination . . . . . . . . . . . . . . . . . . . . . . . . 9 2.5 Exchange Patterns . . . . . . . . . . . . . . . . . . . . . 9 2.6 OCP Environment . . . . . . . . . . . . . . . . . . . . . . 10 3. Messages . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.1 Message Format . . . . . . . . . . . . . . . . . . . . . . . 11 3.2 Message Examples . . . . . . . . . . . . . . . . . . . . . . 13 3.3 Message Names . . . . . . . . . . . . . . . . . . . . . . . 13 4. Transactions . . . . . . . . . . . . . . . . . . . . . . . . 15 5. Negotiation . . . . . . . . . . . . . . . . . . . . . . . . 16 6. Capability and State Inquiry . . . . . . . . . . . . . . . . 17 7. Message Parameters . . . . . . . . . . . . . . . . . . . . . 18 7.1 Parameter Types . . . . . . . . . . . . . . . . . . . . . . 18 7.1.1 Uri . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 7.1.2 Uni . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 7.1.3 Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 7.1.4 Boolean . . . . . . . . . . . . . . . . . . . . . . . . . . 19 8. Parameter Definitions . . . . . . . . . . . . . . . . . . . 20 8.1 xid . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 8.2 rid . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 8.3 service . . . . . . . . . . . . . . . . . . . . . . . . . . 20 8.4 services . . . . . . . . . . . . . . . . . . . . . . . . . . 20 8.5 sg-id . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 8.6 am-id . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 8.7 size-request . . . . . . . . . . . . . . . . . . . . . . . . 21 8.8 offset . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 8.9 sizep . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 8.10 modp . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 8.11 result . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 8.12 error . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 8.13 feature . . . . . . . . . . . . . . . . . . . . . . . . . . 22 9. Message Definitions . . . . . . . . . . . . . . . . . . . . 23 9.1 Connection Start (CS) . . . . . . . . . . . . . . . . . . . 23 9.2 Connection End (CE) . . . . . . . . . . . . . . . . . . . . 24 9.3 Create Service Group (SGC) . . . . . . . . . . . . . . . . . 25 9.4 Destroy Service Group (SGD) . . . . . . . . . . . . . . . . 26 9.5 Transaction Start (TS) . . . . . . . . . . . . . . . . . . . 26 9.6 Transaction End (TE) . . . . . . . . . . . . . . . . . . . . 26 9.7 Application Message Start (AMS) . . . . . . . . . . . . . . 27 Rousskov Expires February 25, 2004 [Page 2] Internet-Draft OPES Callout Protocol Core August 2003 9.8 Application Message End (AME) . . . . . . . . . . . . . . . 28 9.9 Data Use Mine (DUM) . . . . . . . . . . . . . . . . . . . . 28 9.10 Data Use Yours (DUY) . . . . . . . . . . . . . . . . . . . . 29 9.11 Data Pause (data-pause) . . . . . . . . . . . . . . . . . . 30 9.12 Data Paused (data-paused) . . . . . . . . . . . . . . . . . 31 9.13 Data Need (data-need) . . . . . . . . . . . . . . . . . . . 31 9.14 Data ACK (DACK) . . . . . . . . . . . . . . . . . . . . . . 32 9.15 I Am Here (pong) . . . . . . . . . . . . . . . . . . . . . . 33 9.16 Are You There? (ping) . . . . . . . . . . . . . . . . . . . 33 9.17 Negotiation Offer (NO) . . . . . . . . . . . . . . . . . . . 34 9.18 Negotiation Response (NR) . . . . . . . . . . . . . . . . . 35 9.19 I Support (i-can) . . . . . . . . . . . . . . . . . . . . . 36 9.20 Can You Support (can-you) . . . . . . . . . . . . . . . . . 36 9.21 I Currently Use (i-do) . . . . . . . . . . . . . . . . . . . 36 9.22 Do You Currently Use (do-you) . . . . . . . . . . . . . . . 37 10. Application Protocol Requirements . . . . . . . . . . . . . 38 11. IAB Considerations . . . . . . . . . . . . . . . . . . . . . 39 12. Security Considerations . . . . . . . . . . . . . . . . . . 40 13. Compliance . . . . . . . . . . . . . . . . . . . . . . . . . 42 14. To-do . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 A. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 45 B. Change Log . . . . . . . . . . . . . . . . . . . . . . . . . 46 Normative References . . . . . . . . . . . . . . . . . . . . 52 Informative References . . . . . . . . . . . . . . . . . . . 53 Author's Address . . . . . . . . . . . . . . . . . . . . . . 53 Intellectual Property and Copyright Statements . . . . . . . 54 Rousskov Expires February 25, 2004 [Page 3] Internet-Draft OPES Callout Protocol Core August 2003 1. Introduction The Open Pluggable Edge Services (OPES) architecture [I-D.ietf-opes-architecture], enables cooperative application services (OPES services) between a data provider, a data consumer, and zero or more OPES processors. The application services under consideration analyze and possibly transform application-level messages exchanged between the data provider and the data consumer. The OPES processor can delegate the responsibility of service execution by communicating and collaborating with one or more remote callout servers. As described in [I-D.ietf-opes-protocol-reqs], an OPES processor communicates with and invokes services on a callout server by using a callout protocol. This document specifies the core of such a protocol. OCP Core specification documents general, application-independent protocol mechanisms. A separate series of documents describe application-specific aspects of OCP. For example, "OPES adaptation of HTTP" [XXX] describes, in part, how HTTP messages and HTTP meta-information can be communicated over OCP. 1.1 Application proxies and OCP scope (XXX: a better section title would be nice) As an application proxy, OPES processor proxies a single application protocol or converts from one application protocol to another. At the same time, OPES processor may be an OCP client, using OCP to facilitate adaptation of proxied messages at callout servers. It is therefore natural to assume that OPES processor takes application messages being proxied, passes them over OCP to callout servers, and then puts the adaptation results back on the wire. However, such an assumption implies that OCP is applied directly to application messages that OPES processor is proxing, which may not be the case. "OPES processor" "callout server" +-----------------+ +-----------------+ | pre-processing | "OCP scope" | | | +- - - - - - - - - - - - - - - - - - -+ | | iteration | <== ( application data ) ==> | adaptation | | +- - - - - - - - - - - - - - - - - - -+ | | post-processing | | | +-----------------+ +-----------------+ Figure 1 Rousskov Expires February 25, 2004 [Page 4] Internet-Draft OPES Callout Protocol Core August 2003 OPES processor may preprocess (or postprocess) proxied application messages before (or after) they are adapted at callout servers. For example, a processor may take an HTTP response being proxied and pass it as is, along with metadata about the corresponding HTTP connection. Another processor may take an HTTP response, extract its body, and pass that body, along with the content-encoding metadata. Moreover, to perform adaptation, OPES processor may execute several callout services, iterating over several callout servers. Such preprocessing, postprocessing, and iterations make it impossible to rely on any specific relationship between application messages being proxied and application messages being sent to a callout service. Similarly, specific adaptation actions at the callout server are outside of OCP Core scope. This specification does not define or require any specific relationship among application messages being proxied by the OPES processor and application messages being exchanged with callout servers via OCP. OPES processor usually provides some mapping among these application messages, but processor's specific actions are beyond OCP scope. In other words, this specification is not concerned with the OPES processor role as an application proxy, or as an iterator of callout services. The scope of OCP Core is communication between a single OPES processor and a single callout server. Furthermore, an OPES processor is at liberty to choose which proxied application messages or information about them to send over OCP. All proxied messages on all proxied connections (if connections are defined for a given application), everything on some connections, selected proxied messages, or nothing might be sent over OCP to callout servers. OPES processor and callout server state related to proxied protocols can be relayed over OCP as application message metadata. 1.2 Terminology The all-caps keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. OPES processor works with messages from application protocols and may relay information about those application messages to a callout server. OCP is also an application protocol. Thus, protocol elements like "message", "connection", or "transaction" exist in OCP and other application protocols. In this specification, all references to elements from application protocols other than OCP are used with an explicit "application" qualifier. References without the "application" qualifier, refer to OCP elements. Rousskov Expires February 25, 2004 [Page 5] Internet-Draft OPES Callout Protocol Core August 2003 (XXX: Some OCP elements are called "callout" elements in the OCP requirements document. We assume that OCP is equivalent to "callout" in this context. For example, OCP connection is the same as callout connection. Should we be more consistent?) OCP message: OCP message is a basic unit of communication between an OPES processor and a callout server. Message is a sequence of octets formatted according to syntax rules (Section 3.1). Message semantics is defined in Section 9. application message: An entity defined by OPES processor and callout server negotiation. Usually, the negotiated definition would match the definition from an application protocol (e.g., [RFC2616] definition of an HTTP message, including headers, body, and trailers). application message data: An opaque sequence of octets representing complete or partial application message. OCP Core does not distinguish application message structure (if any). Application message data may be empty. data: Same as application message data. original Referring to application message flowing from the OPES processor to a callout server. (XXX: we need a better term than "original") adapted Referring to application message flowing from an OPES callout server to the OPES processor. adaptation: Any kind of access by a callout server, including modification and copying. For example, translating or logging an SMTP message is adaptation of that application message. agent: Client or server for a given communication protocol. A proxy is both a client and a server and, hence, also an agent. For example, OPES processor and callout server are OCP agents. immediate: Performing the specified action before processing new incoming messages or sending any new messages unrelated to the specified action. 1.3 Protocol Development Status (XXX: this section is to be removed from the final protocol specs) This specification is not fully suitable for writing OCP Rousskov Expires February 25, 2004 [Page 6] Internet-Draft OPES Callout Protocol Core August 2003 implementations as some behavioral and formatting aspects are not yet documented. They will be. Note that application-specific details are documented separately, as described in the "Introduction" section. Section 14 contains a list of to-be-implemented items. Rousskov Expires February 25, 2004 [Page 7] Internet-Draft OPES Callout Protocol Core August 2003 2. Overall Operation OPES processor may use OPES callout protocol (OCP) to communicate with callout servers. Adaptation using callout services is sometimes called a "bump in the wire" architecture. (XXX: is this paragraph needed? out of place? 2.1 Initialization OPES processor establishes transport connections with callout servers for the purpose of exchanging application messages with the callout server(s) using OCP. After a transport-layer connection (usually TCP/ IP) is established, communicating OCP agents exchange Connection Start (CS) (Section 9.1) messages. Next, OCP features can be negotiated between the processor and the callout server (see Section 5). For example, OCP agents have to agree on transport encryption and application message definition. When negotiation is complete, OCP agents may start exchanging application messages. 2.2 Original Dataflow When OPES processor wants to adapt an application message, the OPES processor sends a Transaction Start (TS) (Section 9.5) message to initiate an OCP transaction dedicated to that application message. Some transaction properties may need to be renegotiated at this time (XXX: undocumented and might not be needed). The processor then sends a Application Message Start (AMS) (Section 9.7) message to prepare the callout server for application data that will follow. Once application message scope is established, application data can be sent to the callout server, using Data Use Mine (DUM) (Section 9.9) and related OCP message(s). All these messages correspond to original dataflow. 2.3 Adapted Dataflow The callout server receives data and metadata sent by the OPES processor (original data flow). The callout server analyses metadata and adapts data as it comes in. The server usually builds its version of metadata and responds to OPES processor with an 'app-message-start' message. Adapted application message data can be sent next, using Data Use Mine (DUM) (Section 9.9) OCP message(s). The application message is then announced to be "closed" using 'app-message-close' message. The transaction may be closed using 'xaction-end' message as well. All these messages correspond to adapted data flow. Rousskov Expires February 25, 2004 [Page 8] Internet-Draft OPES Callout Protocol Core August 2003 +---------------+ +-------+ | OPES | == (original data flow) ==> |callout| | processor | <== (adapted data flow) === |server | +---------------+ +-------+ Figure 2 Depending on the negotiated application message definition, it may be possible or even required for callout server to respond with more than one application message within the same transaction. In other words, the callout server may adapt a single original application message into multiple application messages. Each application message sent by the callout server is individually identified by "am-id" parameter and can be sent independently from other application messages within the same transaction (this allows for logical- and transport-level interleaving of OCP messages related to different application messages). The OPES processor receives the application message sent by the callout server. Other OPES processor actions specific to the application message received are out of this specification scope. 2.4 Termination Either OCP agent can terminate application message delivery, transaction, or connection by sending an appropriate OCP message. Usually, the callout server terminates application message delivery and the transaction. Abnormal terminations at arbitrary times are supported. Termination OCP message include a result description. 2.5 Exchange Patterns OCP agents may also exchange messages related to their configuration, state, transport connections, application connections, etc. A callout server may remove itself from the application message processing loop. A single OPES processor can communicate with many callout servers and vice versa. It is possible to think of an OPES processor as an ``OCP client'' and of a callout server as an ``OCP server''. The OPES architecture document [I-D.ietf-opes-architecture] describes configuration possibilities. The following informal rules illustrate relationships between transport connections, transactions, OCP messages, and application messages: o An OCP agent may communicate with multiple OCP agents. Communication with multiple OCP agents is outside of this specification scope. Rousskov Expires February 25, 2004 [Page 9] Internet-Draft OPES Callout Protocol Core August 2003 o An OPES processor may have multiple concurrent OCP connections to a callout server. Communication over multiple OCP connections is outside of this specification scope. o A connection may carry multiple concurrent transactions. A transaction is always associated with a single connection (i.e., a transaction cannot span multiple concurrent connections). o A connection may carry at most one message at a time, including control messages and transaction-related messages. A message is always associated with a single connection (i.e., a message cannot span multiple concurrent connections). o A transaction is a sequence of messages related to application of a given set of callout services to a single application message. A sequence of transaction messages from an OPES processor to a callout server is called original flow. A sequence of transaction messages from a callout server to an OPES processor is called adapted flow. The two flows may overlap in time. o A transaction is always associated with a single (original) application message. Adapted flow may transfer information about multiple (adapted) application messages. o An application message (adapted or original) is transferred using a sequence of OCP messages. 2.6 OCP Environment OCP communication is assumed to usually take place over TCP/IP connections on the Internet (though no default TCP port is assigned to OCP). This does not preclude OCP from being implemented on top of any other transport protocol, on any other network. OCP only presumes a reliable connection-oriented transport; any protocol that provides such guarantees can be used; the mapping of OCP message structures onto the transport data units of the protocol in question is outside the scope of this specification. OCP is application agnostic but it is not suitable for all applications. This specification documents known application scope limitations in Section 10. OCP messages can carry application-specific information as payload or application-specific extension parameters. Rousskov Expires February 25, 2004 [Page 10] Internet-Draft OPES Callout Protocol Core August 2003 3. Messages As defined in Section 1.2, an OCP message is a basic unit of communication between an OPES processor and a callout server. A message is a sequence of octets formatted according to syntax rules (Section 3.1). Message semantics is defined in Section 9. Messages are transmitted on top of OCP transport. OCP messages deal with transport and transaction management as well as application data exchange between a single OPES processor and a single callout server. Some messages can only be emitted by an OPES processor; some only by a callout server; some can be emitted by both OPES processor and callout server. Some messages require responses (one could call such messages "requests"); some can only be used in response to other messages ("responses"); some may be sent without solicitation and/or may not require a response. 3.1 Message Format An OCP message consists of a message name followed by optional parameters and payload. The exact message syntax is defined by the following Augmented Backus-Naur Form (ABNF) [RFC2234]: message = name [anonym-parameters] [named-parameters] [payload] ";" CRLF anonym-parameters = 1*(SP anonym-parameter) ; spaced parameters named-parameters = 1*(CRLF named-parameter) ; CRLF-separated params payload = CRLF data anonym-parameter = value named-parameter = name ":" SP value value = atom / structure / list atom = bare-value / quoted-value structure = "{" [ value *(SP value) ] "}" ; spaced values list = "(" [ value *("," value) ] ")" ; comma-separated values name = ALPHA *safe-OCTET bare-value = 1*safe-OCTET quoted-value = DQUOTE data DQUOTE data = size ":" OCTET ; == size safe-OCTET = ALPHA / DIGIT / "-" / "_" size = dec-number ; 0-2147483647 dec-number = 1*DIGIT ; no leading zeros or signs Rousskov Expires February 25, 2004 [Page 11] Internet-Draft OPES Callout Protocol Core August 2003 Figure 3 Several normative rules accompany the above ABNF: o There is no "implied linear space" (LWS) rule. LWS rules are common to MIME-based grammars, but are not used here. The whitespace syntax is restricted to what is explicitly allowed by the above ABNF. o All protocol elements are case sensitive unless specified otherwise. In particular, message names and parameter names are case sensitive. o Sizes are interpreted as decimal values and cannot have leading zeros. o Sizes do not exceed 2147483647. o The size attribute in a quoted-value encoding specifies the exact number of OCTETs following the column (':') separator. If size OCTETs are not followed by a quote ('"') character, the encoding is syntactically invalid. o Empty quoted-values are encoded as a 4-OCTET sequence "0:". o Any parameter value MAY be encoded as a quoted-value. A quoted-value MUST be interpreted after the encoding is removed. For example, number 1234 can be encoded as four OCTETs 1234 or as eight OCTETs "4:1234", yielding exactly the same meaning. (XXX: this makes digital signatures more difficult, see todo) o By default, all values MUST be interpreted as having UTF-8 encoding. Note that ASCII is a UTF-8 subset, and that the syntax prohibits non-ASCII characters outside of the "data" element. (XXX: check that this is enough to satisfy i18n and whatever internationalization requirements IETF has) Messages violating formatting rules are, by definition, invalid (see Section XXX for rules on processing invalid messages). Informal note to implementors: OCP messages have three major parts: anonymous parameters, named parameters, and payload. One or two character lookups are sufficient to determine the next part. If the first character is a semicolumn (";"), all optional parts have been parsed. If the first lookup character is a space, anonymous parameters follow. If the first lookup character is CRLF then the second lookup is needed: if the second character is a digit, payload follows; otherwise, named parameters follow. Other syntax and Rousskov Expires February 25, 2004 [Page 12] Internet-Draft OPES Callout Protocol Core August 2003 semantics rules must still be obeyed, of course. For example, once a named parameter or payload has been discovered, no anonymous parameters should be accepted. Similarly, a single character lookup is sufficient to distinguish a bare value from a quoted one since only quoted values start with a quote character. 3.2 Message Examples OCP syntax provides for compact representation of short control messages and required parameters while allowing for parameter extensions. Below are examples of short control messages. Required CRLF sequences at the end of messages are not shown explicitly. TS 1; ping 123 2; data-pause 22 1; i-can "28:http://iana.org/opes/ocp/TLS"; Figure 4 Optional parameters and extensions are possible using named parameters approach as illustrated by the following example. The 'data-need' message in the example has three anonymous parameters and two named parameters (the last one being an extension). Required CRLF sequence at the end of each line is not shown explicitly. data-need 1 3 12345 size-request: 16384 x-need-info: "26:twenty six octet extension"; Figure 5 Finally, any message may have a payload part. For example, the Data Use Mine (DUM) (Section 9.9) message below carries 8865 bytes of raw data. Required CRLF sequence at the end of each line is not shown explicitly. DUM 1 3 0 8865 modp: 75 sizep: 65537 8865:<... 8865 bytes of data ...>; Figure 6 3.3 Message Names Most OCP messages defined in this specification have short names, Rousskov Expires February 25, 2004 [Page 13] Internet-Draft OPES Callout Protocol Core August 2003 formed by abbreviating or compressing a longer but human-friendlier message title. Short names without a central registration system (like this specification or IANA registry) are likely to cause conflicts. Informal protocol extensions should avoid short names. To emphasize what is already defined by message syntax, implementations must not assume that all message names are very short. Rousskov Expires February 25, 2004 [Page 14] Internet-Draft OPES Callout Protocol Core August 2003 4. Transactions OCP transaction is a logical sequence of OCP messages processing a single original application message. The result of the processing may be zero or more application messages, adapted from the original. A typical transaction consists of two message flows: a flow from the OPES processor to the callout server (sending original application message) and a flow from the callout server to the OPES processor (sending adapted application messages). The number of application messages produced by the callout server and whether the callout server actually modifies original application message may depend on the requested callout service and other factors. The OPES processor or the callout server can terminate the transaction by sending a corresponding message to the other side. A OCP transaction starts with a explicit 'xaction-start' message sent by the OPES processor. A transaction ends with the first 'xaction-end' message, explicit or implied, which can be sent by either side. Zero or more OCP messages associated with the transaction can be exchanged in between. The figure below illustrates possible message sequence (prefix "P" stands for OCP Client, OPES processor; prefix "S" stands for OCP callout server). P: TS 10; P: AMS 10 1; ... processor sending application data to the callout server S: AMS 10 2; ... callout server sending application data to the processor ... processor sending application data to the callout server P: AME 10 1 result; S: AME 10 2 result; P: TE 10 result; Figure 7 Rousskov Expires February 25, 2004 [Page 15] Internet-Draft OPES Callout Protocol Core August 2003 5. Negotiation The negotiation mechanism allows OCP client and server to agree on mutually acceptable set of features, including optional and application-specific behavior as well as OCP extensions. For example, transport encryption, data format, and support for a new message can be negotiated. Negotiation implies intent for a behavioral change. A related mechanism allowing an agent to query capabilities of its counterpart without changing counterpart's behavior is described in Section 6. Most negotiations require at least one round trip time delay. In rare cases when other side's response is not required immediately, negotiation delay can be eliminated. Two core negotiation primitives are supported: negotiation offer and negotiation response. The Negotiation Offer (NO) message (Section 9.17) allows an agent to specify a set of features from which the responder has to select exactly one feature it prefers. The selection is sent using a Negotiation Response (NR) message (Section 9.18). If the response is positive both sides assume that the selected feature is in effect. If the response is negative, no behavioral changes are assumed. In either case, further offers may follow. Negotiation Offer (NO) messages may be sent by either agent. Feature specifications MAY restrict initiator role to one of the agents. For example, negotiation of transport security feature [XXX] is initiated exclusively by OPES processors to avoid situations where both agents wait for each other to make an offer. Since either agent may make an offer, two "concurrent" offers may be made at the same time, from the two communicating agents. Unmanaged concurrent offers may lead to a negotiation deadlock. By giving OPES processor a priority, offer handling rules (Section 9.17) ensure that only one offer per transport connection is honored at a time, and the other concurrent offers are ignored by both agents. Violation of negotiation rules leads to OCP connection termination. This design reduces the number of negotiation scenarios resulting in a deadlock when one of the agents is not compliant. (XXX: add examples) Rousskov Expires February 25, 2004 [Page 16] Internet-Draft OPES Callout Protocol Core August 2003 6. Capability and State Inquiry This section describes OCP interface for querying the capability or state of an agent. A related mechanism allowing agents to negotiate features is described in Section 5. OCP supports two inquiry primitives: capability inquiry and state inquiry. Capability inquiry (see Section 9.20) is concerned about supported, but not necessarily active, features. A response to such a query (see Section 9.19) may contain ranges of supported feature parameters. State inquiry (Section 9.22) focuses on the current state of enabled and active features. A response to a state inquiry (Section 9.21) contains feature parameters specific to agent's current state at the time the inquiry is received. For example, a capability inquiry may reveal that an agent supports two transport security mechanisms while a state inquiry may show a specific security profile being enabled now. The primary purpose of these inquiries is debugging and troubleshooting rather than automated fine-tuning of cooperating agent behavior and configurations. The latter is directly supported by OCP negotiation mechanism. (XXX: do we need this OPTIONS-like feature at all?) (XXX: add examples) Rousskov Expires February 25, 2004 [Page 17] Internet-Draft OPES Callout Protocol Core August 2003 7. Message Parameters This section defines parameters that are used for message definitions (Section 9). For ease of reference, each parameter is given a unique name, even if it is used exclusively as an anonymous parameter. 7.1 Parameter Types This sections defines parameter types. In OCP context, a type is a named set of semantic rules attached to a known value syntax construct (e.g., atom, list, or structure). This specification documents expected types for each formal parameter. Before using a parameter value, an implementation MUST check whether it matches the expected type. A mismatch means that the message is invalid. Specifications based on OCP Core MAY define their own types and MUST declare types of every new formal parameter they introduce. 7.1.1 Uri Uri (universal resource identifier) is an atom formatted according to URI rules in [RFC2396]. Often, a uri parameter is used as a unique (within a given scope) identifier. Many uri parameters are URLs. Unless noted otherwise, URL identifiers do not imply existence of a serviceable resource at the location they specify. For example, an HTTP request for "http:// ietf.org/opes/ocp/raw/tcp" URL (XXX: identifying an OCP transport profile) may result in a 404 (Not Found) response. 7.1.2 Uni Uni (universal numeric identifier) is an atom formatted as dec-number and with a value in the [0, 2147483647] inclusive range. Often, a uni parameter is used as a unique (within a given scope) identifier. 7.1.3 Size Size is an atom formatted as dec-number and with a value in the [0, 2147483647] inclusive range. OCP cannot handle application messages that exceed 2147483647 OCTETs in size or require larger sizes as a part of OCP marshaling process. However, since the definition of an application message is up to OCP agents, it is possible to work around this limitation at a processing level above OCP. Rousskov Expires February 25, 2004 [Page 18] Internet-Draft OPES Callout Protocol Core August 2003 7.1.4 Boolean Boolean type is an atom formatted as dec-number and with a value in the [0, 1] inclusive range. A value of zero ("0") is interpreted as "false". A value of one ("1") is interpreted as "true". Rousskov Expires February 25, 2004 [Page 19] Internet-Draft OPES Callout Protocol Core August 2003 8. Parameter Definitions 8.1 xid "Xid", an OCP transaction identifier, has "uni" type. "Xid" uniquely identifies an OCP transaction originated by a given OPES processor. 8.2 rid "Rid", an OCP request identifier, has "uni" type. "Rid" uniquely identifies an OCP request message on a connection. Request identifiers are used to match certain requests and responses. 8.3 service "Service" is an "{id}" structure, where the id member is an OPES service identifier of type "Uri". Services may have service-dependent parameters. A document defining the service identifier for use with OCP MUST also define service-dependent parameters as additional "service" structure members, if any. For example, a "service" value may look like this: {"28:http://ietf.org/opes/ocp/tls" "8:blowfish"} 8.4 services "Services" is a list of "service" values. Unless noted otherwise, the order of the values is the requested or actual service application order. This parameter MAY appear in any message from the callout server that has an "am-id" parameter. If this parameter appears in a message from the callout server that carries or refers to application data, its value indicates the services actually applied to that data. If this parameter appears in a message from the callout server that neither carries nor refers to application data, its value indicates the services that MAY be applied to that application message in the future. (XXX: say where it cannot appear?)(XXX: make it symmetric with processor) 8.5 sg-id "Sg-id", a service group identifier, is of type "uni". "Sg-id" uniquely identifies a group of services on an OCP connection. 8.6 am-id Rousskov Expires February 25, 2004 [Page 20] Internet-Draft OPES Callout Protocol Core August 2003 "Am-id", an application message identifier, is of type "uni". "Am-id" uniquely identifies an application message within an OCP transaction. 8.7 size-request "Size-request" is of type "Size". The sender uses "size-request" to specify the number of data or metadata OCTETs it desires to receive. 8.8 offset "Offset" is of type "Size". "Offset" describes payload start position relative to the application message data or metadata. The offset of the first application byte has a value of zero. 8.9 sizep Remaining application data size prediction in octets. The value excludes data in the current OCP message, if any. The prediction applies to a single application message. This parameter can be used with any OCP message that has am-id parameter. 8.10 modp Future data modification prediction in percents. A modp value of 0 (zero) means the sender predicts that there will be no data modifications. A value of 100 means the sender is predicts that there will be data modifications. The value excludes data in the current OCP message, if any. The prediction applies to a single application message. This parameter can be used with any OCP message that has am-id parameter. 8.11 result OCP processing result. Result is a structure with two documented members: a required Uni status code and an optional string containing informative textual information, not intended for automated processing. For example, { 200 "2:OK" } This specification defines two result codes: 200 (success) and 400 (failure). This specification does not document any result-specific actions for 200 (success) result recipients. A recipient of a 400 (failure) result of a AME, TE, or CE message MUST destroy any state or data associated with the corresponding data flow, transaction, or connection. For example, adapted version of the application message data must be purged from the proxy cache if the OPES processor receives an Application Message End (AME) message with result code of Rousskov Expires February 25, 2004 [Page 21] Internet-Draft OPES Callout Protocol Core August 2003 400 (failure). Extending result semantics is possible by adding new structure members or explicitly negotiating additional result codes (e.g., as a part of a negotiated profile). A recipient of an unknown result code MUST treat it as if code 400 (failure) was received. The result parameter is optional. By definition, an absent result parameter is semantically equivalent to a {200} result. (XXX: how to make this statement more normative?). (XXX: Since we only support success/failure indication, and since there is probably no reason to supply textual information with successful messages, we should probably just replace "result" with an optional named parameter "Error: text"! That parameter would have the same semantics as code 400 (failure) documented above). 8.12 error A flag indicating abnormal conditions at the sender that cannot be expressed via result parameter. It is RECOMMENDED that the recipient deletes all state associated with the corresponding OCP message. 8.13 feature A OCP feature identifier with optional feature parameters (sometimes called attributes). Used to declare support and negotiate use of OCP optional or extension features. This specification defines three features: TLS transport encryption (Section XXX), Raw Application Binding (Section XXX), and Processor Data Copying (Section XXX). Rousskov Expires February 25, 2004 [Page 22] Internet-Draft OPES Callout Protocol Core August 2003 9. Message Definitions This section describes specific OCP messages. Each message is given a unique name and usually has a set of anonymous and/or named parameters. The order of anonymous parameters is specified in the message definitions below. No particular order for named parameters is implied by this specification. No more than one named-parameter with a given name can appear in the message; messages with multiple equally-named parameters are semantically invalid. A recipient MUST be able to parse any syntactically valid message (see Section 3.1), subject to recipient resources limitations. If resources are exhausted or if a syntactically malformed message is received, the recipient MUST terminate processing of the corresponding connection using a Connection End (CE) message (Section 9.2) with an error flag. (XXX: the error information should identify the problem and distinguish resource limitation from syntax errors?) Unknown or unexpected message names, parameters, and payloads may be valid extensions. For example, an "extra" anonymous parameter may be used for a given message, in addition to what is documented in the message definition below. A recipient MUST ignore any unknown or unexpected name, parameter, or payload. Recipients MAY report (e.g., log) unknown or unexpected elements, of course. Except for messages that introduce new identifiers, all sent identifiers MUST be known (i.e., introduced and not ended by previous messages). Except for messages that introduce new identifiers, the recipient MUST ignore any message with an unknown identifier. For example, recipient must ignore a Data Use Mine (DUM) (Section 9.9) message if the xid parameter refers to an unknown transaction. Message definitions below clearly state rare exceptions to the above rules. (XXX can we define "ignore"?) (XXX move these rules elsewhere?) (XXX Message parameters in [square brackets] are OPTIONAL. Other parameters are REQUIRED.) 9.1 Connection Start (CS) name: CS anonymous parameters: none named parameters: none Rousskov Expires February 25, 2004 [Page 23] Internet-Draft OPES Callout Protocol Core August 2003 payload: no senders: OPES processor only A Connection Start (CS) message indicates the start of an OCP communication from the OPES processor. An OPES processor MUST send this message immediately after establishing a transport connection to the callout server. If the first message a callout server receives is not Connection Start (CS), the callout server MUST terminate the connection with a Connection End (CE) message (Section 9.2). Upon receiving of this message, the callout server MUST either start maintaining connection state or refuse further processing by responding with a Connection End (CE) message (Section 9.2). A callout server MUST maintain the state until it detects the end of the connection or until it terminates the connection itself. A callout server MUST NOT send this message (XXX: why?). If the first message received by an OPES processor is a Connection Start (CS) message, the processor MUST terminate the connection with a Connection End (CE) message (Section 9.2). An OPES processor MUST NOT resend this message. If a callout server receives this message and it is not the first message on a connection, then, the callout server MUST terminate the connection with a Connection End (CE) message (Section 9.2). With TCP/IP as transport, raw TCP connections (local and remote peer addresses) identify an OCP connection. Other transports may provide OCP connection identifiers to distinguish connections that share the same transport. For example, a single BEEP [RFC3080] channel may be designated as a single OCP connection. 9.2 Connection End (CE) name: CE anonymous parameters: [result] named parameters: [error] payload: no senders: both OPES processor and callout server Indicates an end of a transport connection. The agent initiating closing or termination of a connection MUST send this message immediately prior to closing or termination. The recipient MUST free Rousskov Expires February 25, 2004 [Page 24] Internet-Draft OPES Callout Protocol Core August 2003 associated state, including transport state. The destruction of the state ensures that messages outside of the old connection are ignored. Connection termination without a Connection End (CE) message indicates that the connection was prematurely closed without the closing-side agent prior knowledge or intent. When an agent detects a prematurely closed connection, the agent MUST behave as if an Connection End (CE) message indicating a fatal error was received. A Connection End (CE) message implies the end of all transactions, negotiations, and service groups opened or active on the connection being ended. 9.3 Create Service Group (SGC) name: SGC anonymous parameters: sg-id services named parameters: none payload: no senders: both OPES processor and callout server Create Service Group (SGC) message instructs the recipient to associate a list of services with a given service group identifier ("sg-id"). The group can then be referred by the sender using the identifier. The recipient MUST maintain the association until a corresponding Destroy Service Group (SGD) message is received or implied. Service groups have a connection scope. Transaction management messages do not affect existing service groups. Maintaining service group associations requires resources (e.g., storage to keep the group identifier and a list of service IDs). Thus, there is a finite number of associations an implementation can maintain. Callout servers MUST be able to maintain at least one association for each OCP connection they accept. If a recipient of the Create Service Group (SGC) message does not create the requested association, it MUST immediately terminate the connection with a Connection End (CE) message. (XXX: document that wrong sq-id lead to semantically invalid messages) Rousskov Expires February 25, 2004 [Page 25] Internet-Draft OPES Callout Protocol Core August 2003 9.4 Destroy Service Group (SGD) name: SGD anonymous parameters: sg-id named parameters: none payload: no senders: both OPES processor and callout server Destroy Service Group (SGC) message instructs the recipient to forget about the service group associated with the specified "sg-id" identifier. If "sg-id" refers to an existing group, the recipient MUST destroy the association. Otherwise, the recipient MUST treat the message as invalid. 9.5 Transaction Start (TS) name: TS anonymous parameters: xid sg-id named parameters: none payload: no senders: OPES processor only Indicates the start of an OCP transaction. A callout server MUST NOT send this message. Upon receiving of this message, the callout server MUST either start maintaining transaction state or refuse further processing by responding with a 'xaction-end' message. A callout server MUST maintain the state until it receives a message indicating the end of the transaction or until it terminates the transaction itself. The "sg-id" parameter refers to a service group created with a Create Service Group (SGC) message. If no group is associated with "sg-id", the callout server MUST treat the message as invalid. Otherwise, the callout server MUST use the list of services associated with "sg-id". This message introduces transaction identifier (xid). 9.6 Transaction End (TE) Rousskov Expires February 25, 2004 [Page 26] Internet-Draft OPES Callout Protocol Core August 2003 name: TE anonymous parameters: xid, result named parameters: [error] payload: no senders: both OPES processor and callout server Indicates the end of the OCP transaction. The recipient MUST free associated state. The destruction of the state ensures that future messages referring to the same transaction, if any, will be ignored. This message terminates the life of the transaction identifier (xid). An OCP agent MUST send a Transaction End (TE) message immediately after it makes a decision to send no more messages related to the corresponding transaction. Violating this requirement may cause, for example, unnecessary delays and even timeouts for OPES processors that rely on this end-of-file condition to proceed. 9.7 Application Message Start (AMS) name: AMS anonymous parameters: xid, am-id named parameters: none payload: no senders: both OPES processor and callout server Indicates the start of processing of an application message. The recipient MUST either start processing the application message (and maintain its state) or refuse further processing with an 'app-message-end' message. The recipient MUST maintain the state until it receives a message indicating the end of application message processing or until it terminates the processing itself. When 'app-message-start' message is sent to the callout server, the callout server usually sends an app-message-start message back, announcing the creation of an adapted version of the original application message. Such response may be delayed. For example, the callout server may wait for more information to come from the OPES processor. Rousskov Expires February 25, 2004 [Page 27] Internet-Draft OPES Callout Protocol Core August 2003 When 'app-message-start' message is sent to the OPES processor, an OPTIONAL "services" parameter describes callout services that the server MAY apply to the application message. Usually, the "services" value matches what was asked by the OPES processor. This message introduces application message identifier (am-id). 9.8 Application Message End (AME) name: AME anonymous parameters: xid, am-id, result named parameters: [error] payload: no senders: both OPES processor and callout server Informs the recipient that there will be no more data for the corresponding application message and indicates the end of application message processing. The recipient MUST free associated application message state. The destruction of the state ensures that future messages referring to the same application message, if any, will be ignored. An AME (Application Message End) message ends any data preservation commitments associated with the corresponding application message. This message terminates the life of the application message identifier (am-id). An OCP agent MUST send an Application Message End (AME) message immediately after it makes a decision to send no more data for the corresponding application message. Violating this requirement may cause, for example, unnecessary delays and even timeouts for callout servers that rely on this end-of-file condition to proceed. 9.9 Data Use Mine (DUM) name: DUM anonymous parameters: xid, am-id, offset named parameters: [As-is: am-id offset], [Kept: offset], [Wont-Use: used-size], [sizep], [modp], [ack] Rousskov Expires February 25, 2004 [Page 28] Internet-Draft OPES Callout Protocol Core August 2003 payload: yes senders: both OPES processor and callout server This is the only OCP message that may carry application data. There MUST NOT be any gaps in data supplied by Data Use Mine (DUM) and Data Use Yours (DUY) messages (i.e., the offset of the next data message must be equal to the offset plus the payload size of the previous data message) (XXX: we do not need offset then; should we keep it as a validation mechanism?) (XXX: document what to do when this MUST is violated). Zero payload size is permitted and is useful for communicating predictions without sending data. When an OPES processor sends a "Kept" parameter, the processor MUST keep a copy of the attached data (the preservation commitment starts). The offset parameter specifies the offset of the first OCTET in the payload (XXX: relative to what? DUM messages may transmit metadata that is not kept). The first "Kept" offset sent is zero. The next "Kept" offset sent is the previous "Kept" offset sent plus the size of the previous "Kept" payload, modulo 2^32. Recipients of invalid "Kept" parameters, MUST either terminate the corresponding transaction or MUST not use any "Kept" parameter during the transaction, including those received before the invalid one. This requirement helps preserve data integrity when "Kept" optimization is used by the processor. If a Wont-Use named parameter is present, is semantics is determined exactly as for Data Use Yours (DUY) messages. An "as-is" parameter, sent only by the callout server, MUST specify a data fragment in the original data flow. The callout server informs the processor that the attached data is identical to an original data fragment the processor sent earlier. Identical means that all adapted OCTETs have the same numeric value as the corresponding original OCTETs. The "am-id" field MUST correspond to the original application message identifier for the same transaction. If the parameter does not specify any original data fragment, the parameter is invalid. Invalid "as-is" parameters MUST be ignored. The recipient of an "ack" parameter MUST respond with a 'data-ack' message. Note that OCP Core does not require these acknowledgments to successfully exchange data; they are supported for debugging and similar important applications outside of the Core scope. 9.10 Data Use Yours (DUY) Rousskov Expires February 25, 2004 [Page 29] Internet-Draft OPES Callout Protocol Core August 2003 name: DUY anonymous parameters: xid, am-id, kept-offset, kept-size named parameters: [Wont-Use: used-size] payload: no senders: callout server only Tells the OPES processor to use "kept-size" bytes of preserved data starting at "kept-offset" offset, as if that data came from the callout server in a Data Use Mine (DUM) message with payload size equal to the "kept-size" parameter. The identified data fragment MUST be under the preservation commitment. If the OPES processor receives a Data Use Yours (DUY) message for data not under preservation commitment, the message is invalid. If a Data Use Yours (DUY) message is invalid, the OPES processor MUST abort am-id message processing (XXX: document how processing should be aborted). If no Wont-Use named parameter is present, the Data Use Yours (DUY) message terminates preservation commitment (if any) for data below (kept-offset + kept-size) offset. If a valid Wont-Use named parameter is present, the DUY message terminates preservation commitment (if any) for data below (0 + used-size). OPES processor MUST ignore a Wont-Use parameter with a used-size value below previously received used-size values. Note that (0 + used-size) may be less than (kept-offset + kept-size), indicating that some of the kept data referred to in the DUY message needs to be preserved. Similarly, (0 + used-size) may exceed (kept-offset + kept-size), indicating that callout server is not going to use preserved data that has not been referred to yet. For example, "Wont-Use: 2147483647" message parameter indicates that the server is not going to send any more DUY messages. 9.11 Data Pause (data-pause) name: data-pause anonymous parameters: xid am-id named parameters: none Rousskov Expires February 25, 2004 [Page 30] Internet-Draft OPES Callout Protocol Core August 2003 payload: no senders: callout server only Sent by a callout server, the data-pause message informs the OPES processor that it must stop sending data to the callout server until the callout server explicitly asks for more data using a 'data-need' message. Upon receiving a 'data-pause' message, the OPES processor SHOULD stop sending application message data to the callout server. If the OPES processor stops sending, it SHOULD send a corresponding 'data-paused' message to the callout server. Until the OPES processor receives the message, it may continue sending data to the callout server, of course. Thus, when the callout server sends this message, it MUST NOT mark the application message as "paused". (XXX: should we use MUST or MAY instead of SHOULDs above?) An OPES processor MUST NOT send this message. A callout server MUST ignore this message. 9.12 Data Paused (data-paused) name: data-paused anonymous parameters: xid, am-id named parameters: none payload: no senders: OPES processor only Sent by an OPES processor, the 'data-paused' message informs the callout server that there will be no more data for the specified application message until the callout server explicitly asks for data using a 'data-need' message. After sending a 'data-paused' message, the OPES processor MUST stop sending application message data to the callout server. At that time, there may be still unprocessed data in the callout server queue, of course. When the callout server receives the message, it MAY mark the application message as "paused". If the callout server receives data for a paused message (a violation of the above MUST), the callout server MAY abort application message processing. A callout server MUST NOT send this message. An OPES processor MUST ignore this message. 9.13 Data Need (data-need) Rousskov Expires February 25, 2004 [Page 31] Internet-Draft OPES Callout Protocol Core August 2003 name: data-need anonymous parameters: xid am-id offset named parameters: [Size-request: size] payload: no senders: callout server only Informs the OPES processor that the callout server needs more application message data. The "offset" parameter indicates the amount of data already received. If a "Size-request" parameter is present, its value is the suggested data size, and it MAY be ignored by the OPES processor. An absent "Size-request" parameter implies "any size". The callout server MUST clear the "paused" state of the application message processing just before sending this message. The OPES processor MUST ignore a data-need message if the OPES processor already sent request data. An OPES processor MUST NOT send data-need messages (XXX: should we give an OPES processor the same abilities to pause/resume message processing that a callout server has?) 9.14 Data ACK (DACK) name: DACK anonymous parameters: xid, am-id, offset, size named parameters: [wont-forward] payload: no senders: callout server only Informs the OPES processor that the corresponding data chunk has been received by the callout server. An optional "wont-forward" flag terminates preservation commitment for the corresponding data, if any. The flag is defined for callout server 'data-ack' messages only. Responding with Data ACK (DACK) messages to Data Use Mine (DUM) (Section 9.9) messages with an "ack" flag is REQUIRED. Responding Rousskov Expires February 25, 2004 [Page 32] Internet-Draft OPES Callout Protocol Core August 2003 with DACK messages to DUM messages without an "ack" flag is OPTIONAL. Implementations SHOULD be able to support debugging mode where every DUM message is acked. (XXX: should we require responses for Data Use Yours messages as well?) A 'data-ack' response SHOULD be sent as soon as possible. If the callout server does not know immediately whether it will forward the data, it MUST respond without a "wont-forward" flag. If, at any time, the callout server decides that it will not forward the data, it SHOULD send a 'data-ack' message with a "wont-forward" flag. Thus, multiple 'data-ack' messages and unsolicited 'data-ack' messages are allowed. Sending of a DACK message means that a complete DUM message has been received, but does not imply that the data has been processed in any other way. The data acknowledgment mechanism has several purposes: to allow OPES processor to gauge the speed at which the callout server is receiving data (for optimization purposes); to send back "wont-forward" notifications; and to assist in debugging OCP communications. 9.15 I Am Here (pong) name: pong anonymous parameters: [xid [am-id]] named parameters: none payload: no senders: both OPES processor and callout server Parameterless form informs the recipient that the sender is still maintaining the OCP connection. If "xid" or "am-id" identifier(s) are used, the message informs the recipient that the sender is still processing the corresponding transaction or an application message. An 'i-am-here' message MAY be sent without solicitation. An 'i-am-here' message MUST be sent in response to an 'are-you-there' request. The response MUST have the same set of "xid" and "am-id" parameters if those identifiers are still valid. The response MUST NOT use invalid identifiers. 9.16 Are You There? (ping) Rousskov Expires February 25, 2004 [Page 33] Internet-Draft OPES Callout Protocol Core August 2003 name: ping anonymous parameters: [xid [am-id]] named parameters: none payload: no senders: both OPES processor and callout server Solicits an immediate 'i-am-here' response. If the response does not use the same set of "xid" and "am-id" parameters, the recipient MAY assume that missing identifier(s) correspond to OCP transaction or application message that was not maintained at the time the response was generated. The recipient MUST handle an 'are-you-there' request even if transaction or application message identifiers are invalid from the recipient point of view. Normally, messages with invalid identifiers are ignored. 9.17 Negotiation Offer (NO) name: NO anonymous parameters: (list of features) named parameters: [SG: sg-id] payload: no senders: both OPES processor and callout server A Negotiation Offer (NO) message solicits a selection of a single "best" feature out of a supplied list, using a Negotiation Response (NR) message. The sender is expected to list preferred features first when possible. The recipient MAY ignore sender preferences. If the list of features is empty, the negotiation is bound to fail but remains valid. Both OPES processor and callout server are allowed to send Negotiation Offer (NO) messages. The rules in this section ensure that only one offer is honored if two offers are submitted concurrently. An agent MUST NOT send a Negotiation Offer (NO) message if it still expects a response to its previous offer on the same connection. If an OPES processor receives a Negotiation Offer (NO) message while Rousskov Expires February 25, 2004 [Page 34] Internet-Draft OPES Callout Protocol Core August 2003 its own offer is pending, the processor MUST disregard the server offer. Otherwise, it MUST respond immediately. If a callout server receives a Negotiation Offer (NO) message when its own offer is pending, the server MUST disregard its own offer. In either case, it MUST respond immediately. If an agent receives a message sequence that violates any of the above rules in this section, the agent MUST terminate the connection with an error. An optional "SG" parameter assigns the result of negotiation to the specified service group. If SG is present, the negotiated features are enabled only for transactions that use the specified service group ID. An invalid service group identifier makes the entire Negotiation Offer message invalid. 9.18 Negotiation Response (NR) name: NR anonymous parameters: [feature] named parameters: [SG: sg-id], [Rejects], [Unknowns] payload: no senders: both OPES processor and callout server A Negotiation Response (NR) message conveys recipient reaction to a Negotiation Offer (NO) request. An accepted offer is indicated by the presence of a "feature" parameter, containing the selected feature. If the selected feature does not match any of the offered features, the offering agent MUST consider negotiation failed and MAY terminate the connection. A rejected offer is indicated by omitting the "feature" parameter. If negotiation offer contains an SG parameter, the responder MUST include that parameter in the Negotiation Response (NR) message. The recipient of a Negotiation Response (NR) message without the expected SG parameter MUST treat negotiation response as invalid. If negotiation offer lack an SG parameter, the responder MUST NOT include that parameter in the Negotiation Response (NR) message. The recipient of a Negotiation Response (NR) message with an unexpected SG parameter MUST treat negotiation response as invalid. Rousskov Expires February 25, 2004 [Page 35] Internet-Draft OPES Callout Protocol Core August 2003 When accepting or rejecting an offer, the sender of the NR message MAY supply additional details via Rejects and Unknowns parameters. The Rejects parameter can be used to list features that were known to the NO recipient but could not be supported given negotiated state that existed when NO message was received. The Unknowns parameter can be used to list features that were unknown to the NO recipient. 9.19 I Support (i-can) name: i-can anonymous parameters: [feature] named parameters: none payload: no senders: both OPES processor and callout server An I Support (i-can) message is sent in response to a Can You Support (can-you) question. If the sender supports feature identifier, the sender MUST respond with a "feature" parameter, set to match actually supported feature and its attributes, if any. Otherwise, the sender MUST respond without a "feature" parameter. Note that supported features attributes of the sender may differ from those in the Can You Support (can-you) question, indicating a partial match or a mismatch. 9.20 Can You Support (can-you) name: can-you anonymous parameters: feature named parameters: none payload: no senders: both OPES processor and callout server A Can You Support (can-you) message solicits a declaration of support for the supplied feature, using an I Support (i-can) message. The recipient MUST respond immediately. 9.21 I Currently Use (i-do) Rousskov Expires February 25, 2004 [Page 36] Internet-Draft OPES Callout Protocol Core August 2003 name: i-do anonymous parameters: feature named parameters: none payload: no senders: both OPES processor and callout server 9.22 Do You Currently Use (do-you) name: do-you anonymous parameters: feature named parameters: none payload: no senders: both OPES processor and callout server Rousskov Expires February 25, 2004 [Page 37] Internet-Draft OPES Callout Protocol Core August 2003 10. Application Protocol Requirements Not all application protocols can be adapted with OCP. Compiling a complete list of known limitations is impossible since "application protocol" is not a well defined term. However, listing known limitations can help it determining OCP applicability. This section is not a normative part of the OCP specification. Application protocol messages must have byte boundaries. OCP can only handle application messages with the number of bits divisible by 8. XXX Rousskov Expires February 25, 2004 [Page 38] Internet-Draft OPES Callout Protocol Core August 2003 11. IAB Considerations OPES treatment of IETF Internet Architecture Board (IAB) considerations [RFC3238] are documented in [XXX]. Rousskov Expires February 25, 2004 [Page 39] Internet-Draft OPES Callout Protocol Core August 2003 12. Security Considerations This section examines security considerations for OCP. OPES threats are documented in [XXX-Threat-Doc]. OCP relays application messages that may contain sensitive information. Appropriate transport encryption can be negotiated to prevent information leakage or modification (see section XXX on transport security profile negotiation), but OCP agents may support unencrypted transport by default. Such default OCP agent configurations will expose application messages to third party recording and modification, even if OPES agents themselves are secure. OCP implementation bugs may lead to security vulnerabilities in OCP agents, even if OCP traffic itself remains secure. For example, a buffer overflow in a callout server caused by a malicious OPES processor may grant that processor access to information from other (100% secure) OCP connections, including connections with other OPES processors. Careless OCP implementations may rely on various OCP identifiers to be unique across all OCP agents. A malicious agent can inject an OCP message that matches identifiers used by other agents, in an attempt to get access to sensitive data. OCP implementations must always check an identifier for being "local" to the corresponding connection before using that identifier. OCP is a stateful protocol. Several OCP commands increase the amount of state that the recipient has to maintain. For example, a Create Service Group (SGC) message instructs the recipient to maintain an association between a service group identifier and a list of services. Implementations that cannot handle resource exhaustion correctly increase security risks. The following are known OCP-related resources that may be exhausted during a compliant OCP message exchange: OCP message structures: OCP message syntax does not limit the nesting depth of OCP message structures and does not place an upper limit on the length (number of OCTETs) of most syntax elements. concurrent connections: OCP does not place an upper limit on the number of concurrent connections that a callout server may be instructed to create via Connection Start (CS) messages. Rousskov Expires February 25, 2004 [Page 40] Internet-Draft OPES Callout Protocol Core August 2003 service groups: OCP does not place an upper limit on the number of service group associations that a callout server may be instructed to create via Create Service Group (SGC) messages. concurrent transactions: OCP does not place an upper limit on the number of concurrent transactions that a callout server may be instructed to maintain via Transaction Start (TS) messages. concurrent flows: OCP Core does not place an upper limit on the number of concurrent adapted data flows that an OPES processor may be instructed to maintain via Application Message Start (AMS) messages. Denial of service attacks using OCP may slow a callout server down, affecting performance of many independent OPES processors and, hence, user-perceived performance. (XXX: this has nothing to do with OCP and should be deleted from these specs, right?) Rousskov Expires February 25, 2004 [Page 41] Internet-Draft OPES Callout Protocol Core August 2003 13. Compliance This specification defines compliance for the following subjects: OCP client implementations (OPES processors), OCP server implementations (callout servers), OCP application profiles, and OCP extensions. A subject is compliant if it satisfies all applicable "MUST" and "SHOULD" level requirements. By definition, to satisfy a "MUST" level requirement means to act as prescribed by the requirement; to satisfy a "SHOULD" level requirement means to either act as prescribed by the requirement or have a reason to act differently. A requirement is applicable to the subject if it instructs (addresses) the subject. Informally, OCP compliance means that there are no known "MUST" violations, and all "SHOULD" violations are conscious. In other words, a "SHOULD" means "MUST satisfy or MUST have a reason to violate". It is expected that compliance claims are accompanied by a list of unsupported SHOULDs (if any), in an appropriate format, explaining why preferred behavior was not chosen. Only normative parts of this specification affect compliance. Normative parts are: parts explicitly marked using the word "normative", definitions, and phrases containing unquoted capitalized keywords from [RFC2119]. Consequently, examples and illustrations are not normative. Rousskov Expires February 25, 2004 [Page 42] Internet-Draft OPES Callout Protocol Core August 2003 14. To-do L4+ binding: Mention that transport might be L4 or above timeouts: document what messages cause what timers to be [re]set. parameter scope: Document that parameter names have message scope. A known parameter name in an unknown message does not identify a known parameter. name named-paramters: Document names for all named parameters. header signatures: Current syntax allows for quoting of values that do not need to be quoted. Named parameter order is also not fixed. These make digital signatures of headers impossible without interpretation. Is this a problem? modified: replace with as-is approach meta-data format: How/when do OPES processor and callout server agree on meta-data format and contents? Note that meta-data should usually describe actual data encoding. Data-encoding may, however, be also negotiated. How? When? explain preservation: Add a section explaining data preservation mechanism. asis: Can a callout server refer to parts of [copied] data messages from the OPES processor? If yes, do we need to worry about fragmentation if yes? If no, will this restriction kill the optimization for mid-size application messages (the common case?) that are likely to be passed to the callout server in just one or two chunks? dack: Make data acknowledgment mechanism symmetric. Both agents should be able to request DACK messages. Should we make the acknowledgment mechanism to work with any OCP message? Merge with ping/pong mechanism? flags: Document flag parameter syntax ("0/1" or present/absent). str vs token: Document somewhere that strings such as "OK" in {200 "2:OK"} result must be encoded as strings, not tokens. partial: Should we support partial application message exchange (exchange only a part of the application message)? Who decides what parts to exchange? Should the callout server be able to ask which part it wants? How will it describe the part if it has not Rousskov Expires February 25, 2004 [Page 43] Internet-Draft OPES Callout Protocol Core August 2003 seen the entire message? loss: Should OPES processor be able to signal loss of data to the callout server. The current wording assumes that offset is incremented using sizes of actually received data fragments; if the processor detects loss it cannot pass that information and can only hope that the callout server will notice (by interpreting the data) or will not care (the server may be application- and/or loss-agnostic; e.g., a logging or billing server) break: allow a callout server to get out of the processing loop without losing the data. Add a i-want-out server message. xact boundary: Document that transactions cannot cross connections, but see "fast track" todo item. fast track: Document messages that may be sent on alternative connections. Require other-connections messages to be duplicated on the primary connection. modp: Min and max values (0 and 100) should be "commitments" rather than "probabilities". transactions-end: Decide whether we need a 'transactions-end' message to terminate multiple transactions efficiently. Is terminating a connection good enough? error: Do we need this flag or should we use result codes to relay the same meaning? abort negotiation: Should we let the other side affect the abort decision on OPES level? Perhaps the callout server is doing some logging or accounting and MUST see every byte received by the OPES processor, even if the application message is aborted by the processor. Should we add some kind of 'xaction-need-all' message? Or should we assume that the dispatcher always knows callout server needs and vice versa? proxying Can OCP be proxied above transport layer? Perhaps to implement parts of a given service, transparently to the OPES processor? normative IDs: To be normative, OPES Internet-Drafts must be replaced with corresponding RFCs when the latter are published. Rousskov Expires February 25, 2004 [Page 44] Internet-Draft OPES Callout Protocol Core August 2003 Appendix A. Acknowledgements The author gratefully acknowledges the contributions of: Abbie Barbir (Nortel Networks), Oskar Batuner (Independent Consultant), Karel Mittig (France Telecom R&D), Markus Hofmann (Bell Labs), Hilarie Orman (The Purple Streak), Reinaldo Penno (Nortel Networks), Martin Stecher (Webwasher) as well as an anonymous OPES working group participant. Special thanks to Marshall Rose for his xml2rfc tool. Rousskov Expires February 25, 2004 [Page 45] Internet-Draft OPES Callout Protocol Core August 2003 Appendix B. Change Log Internal WG revision control ID: $Id: ocp-core.xml,v 1.13 2003/08/28 03:58:28 rousskov Exp $ head-sid15 * Removed leftovers of data-have message name. Use Data Use Mine instead (Karel Mittig). * Anonymized named parameters and removed currently unused "rid" parameter in ping and pong messages (Karel Mittig). * Renamed DUM.please-ack to "DUM.ack" (Karel Mittig). More work is needed to polish and simplify acknowledgment mechanism. head-sid14 * Documented known resource-exhaustion security risks. * Polished compliance definition. Avoid two levels of compliance. head-sid13 * Added SG parameter to Negotiation Offer (NO) and Negotiation Response (NR) messages to limit the result of negotiations to the specified service group. Still need to document SG-related logic in the Negotiation section. * Removed "services" parameter from Transaction Start (TS) message because we have to rely on service groups exclusively, because only service groups can have negotiated application profiles associated with them. * Replaced data-id parameter with "Kept: kept-offset" and "Wont-Use: used-size" parameter. We probably need octet-based granularity, and old data-id only offered fragment-based granularity. * Made AME and TE messages required. * Documented result parameter syntax and two result codes: 200 (success) and 400 (failure). * Added optional "result" parameter to CE. Rousskov Expires February 25, 2004 [Page 46] Internet-Draft OPES Callout Protocol Core August 2003 head-sid12 * Fixed BNF to remove extra SP and "," in front of structure and list values. * Fixed the type of "id" field in a "service" structure. * Documented "sg-id" parameter. * Renamed "copied" to "data-id" so that it can be used by both agents. An OPES processor uses named "Copied: data-id" parameter and a callout server uses anonymous "data-id" parameter (instead of previously documented "copy-am-offset"). * Removed "rid" parameter from Negotiation Offer (NO) message as unused. * Removed "size" parameter from messages with payload since payload syntax includes an explicit size value. * Renamed Data Have (DH) message to Data Use Mine (DUM) message to preserve the symmetry with Data Use Yours (DUY) message and to prepare for possible addition of Data Check Mine (DCM) message. * Finished phasing out the "modified" message parameter. * Added an "As-is" named-parameter to mark adapted pieces of data identical to the original. * Replaced a huge "message nesting" figure with a set of short specific rules illustrating the same concept. Added a new "Exchange Patterns" subsection to accommodate the rules and related matters. The figure was not clear enough. Hopefully, the rules are. head-sid10 * Removed the concept of OCP connection as a group of messages sharing the same group of callout services. Now there is no difference between OCP connection and transport connection. * Added a concept of a Service Group, which is a list of services with an identifier, for now. A given Service Group is referenced by the creating/destroying side only, to prevent destruction synchronization. * Removed Connection Services (CSvc) message. Rousskov Expires February 25, 2004 [Page 47] Internet-Draft OPES Callout Protocol Core August 2003 * Removed connection priority until proven generally useful. Can be implemented as an extension. head-sid9 * Added Negotiation and Capability Inquiry sections. * Deleted data-end message because AME (Application Message End) already does the same thing and because there is no data-start message. * Deleted meta-* messages. Data-* messages are now used for both metadata and data since OCP does not know the difference, but must provide the same exchange mechanism for both. * Use a single message name (short or long, depending on the message) instead of using full and abbreviated versions and trying to enforce abbreviations on the wire. Be more consistent in creating short message names. * Resurrected OCP scope figure based on popular demand. * Applied Martin Stecher comments dated 2003/05/30. head-sid8 * Added structure and list values to ABNF syntax. * Messages with multiple equally-named parameters are semantically invalid. * Added types for message parameters. * Started replacing complicated, error-prone, and probably mostly useless "modified" parameter with a clear and simple "as-is" parameter. * Converted parameter descriptions from list items to subsections. * OCP syntax requires one or two character lookups to determine the next message part. Fixed a comment for implementors saying that one lookup is always sufficient. head-sid7 * Mentioned TCP/IP/Internet as assumed transport/network, with any other reliable connection-oriented transport/network usable Rousskov Expires February 25, 2004 [Page 48] Internet-Draft OPES Callout Protocol Core August 2003 as well. We do not document how OCP messages are mapped to TCP but it should be obvious. See Overall Operation section. * Applied Martin Stecher's corrections to OCP message syntax and definitions of messages. * Restricted full message name use to documentation, debuggers, and such. The differences in abbreviated and full name usage still need more consideration and polishing. * IAB Considerations section now refers to the future opes-iab draft. head-sid6 * Added OCP message syntax. Reformatted message descriptions to match new syntax concepts. * Started adding meta-have message to exchange metadata details. Removed negotiation messages for now (posted new messages to the list for a discussion). * Added Security Considerations section (based on Abbie Barbir's original text). head-sid4 * Changed document labels to reflect future "WG draft" status. * Added Acknowledgments section. * Added "Core" to the title since we expect application specific drafts to follow and because this document, even when complete, cannot specify a "working" protocol without application-specific parts. This change is still debatable. * Added reference to required future application-specific specs in the Introduction. * Moved all rant about irrelevance of application protocols proxied by an OPES processor to the "Application proxies and OCP scope" section. Removed "processor input" and "processor output" terms. No reason to define a new term when its only purpose is to document irrelevance? * Moved "OCP message" definition to the terminology section. * Clarified "application message" definition based on recent WG Rousskov Expires February 25, 2004 [Page 49] Internet-Draft OPES Callout Protocol Core August 2003 discussions and suggestions. There seems to be consensus that "application message" is whatever OPES processor and callout server define or agree on, but OCP needs some minimal structure (content + metadata) * Synced data and metadata definitions with the new "application message" definition. * Simplified "Overall Operation" section since it no longer need to talk about irrelevance of application protocols proxied by an OPES processor. * Illustrated nesting/relationship of key OCP concepts (application message, OCP message, transaction, connection, transport connection, etc.). The figure needs more work. * Listed all from-processor and from-server OCP messages in one place, with references to message definitions. * Added "services" message parameter, assuming that more than one service may be requested/executed with one transaction. * Gave callout server ability to report what services were actually applied (see "services" parameter definition). head-sid3 * clarified application message definition and OCP boundaries by introducing three kinds of "applications": processor input, processor output, and OCP application * made "Overall Operation" a top-level section since it got long and has its own subsections now; lots of editorial changes in this sections, new figures * added illustrations of OCP messages, transactions, and connections head-sid2 * introduced a notion of meta-data to both simplify OCP and make OCP agnostic to application meta-data; previous approach essentially assumed existence of a few common properties like protocol name or application message source/destination while not allowing any other properties to be exchanged between OCP agents); specific meta-data format/contents is not important to OCP but OCP will help agents to negotiate that format/contents Rousskov Expires February 25, 2004 [Page 50] Internet-Draft OPES Callout Protocol Core August 2003 * removed wording implying that OCP adapts application messages; OCP only used to exchange data and meta-data (which facilitates adaptation) * changed most of the definitions; added definitions for meta-data, original/adapted flows, and others * split 'data-pause' message into 'data-pause' request by the callout server and 'data-paused' notification by the OPES processor; fixed "paused" state management * added motivation for data acking mechanism * replaced "am-proto", "am-kind", "am-source", and "am-destination" parameters with "meta-data" * replaced SERVER and CLIENT placeholders with "callout server" and "OPES processor" * added editing marks Rousskov Expires February 25, 2004 [Page 51] Internet-Draft OPES Callout Protocol Core August 2003 Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 2234, November 1997. [RFC2396] Berners-Lee, T., Fielding, R. and L. Masinter, "Uniform Resource Identifiers (URI): Generic Syntax", RFC 2396, August 1998. [I-D.ietf-opes-architecture] Barbir, A., "An Architecture for Open Pluggable Edge Services (OPES)", draft-ietf-opes-architecture-04 (work in progress), December 2002. Rousskov Expires February 25, 2004 [Page 52] Internet-Draft OPES Callout Protocol Core August 2003 Informative References [I-D.ietf-opes-protocol-reqs] Beck, A., "Requirements for OPES Callout Protocols", draft-ietf-opes-protocol-reqs-03 (work in progress), December 2002. [I-D.ietf-opes-scenarios] Barbir, A., "OPES Use Cases and Deployment Scenarios", draft-ietf-opes-scenarios-01 (work in progress), August 2002. [I-D.ietf-fax-esmtp-conneg] Toyoda, K. and D. Crocker, "SMTP Service Extension for Fax Content Negotiation", draft-ietf-fax-esmtp-conneg-06 (work in progress), February 2003. [RFC2616] Fielding, R., Gettys, J., Mogul, J., Nielsen, H., Masinter, L., Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. [RFC3080] Rose, M., "The Blocks Extensible Exchange Protocol Core", RFC 3080, March 2001. [RFC3238] Floyd, S. and L. Daigle, "IAB Architectural and Policy Considerations for Open Pluggable Edge Services", RFC 3238, January 2002. Author's Address Alex Rousskov The Measurement Factory EMail: rousskov@measurement-factory.com URI: http://www.measurement-factory.com/ Rousskov Expires February 25, 2004 [Page 53] Internet-Draft OPES Callout Protocol Core August 2003 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on the IETF's procedures with respect to rights in standards-track and standards-related documentation can be found in BCP-11. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementors or users of this specification can be obtained from the IETF Secretariat. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this standard. Please address the information to the IETF Executive Director. Full Copyright Statement Copyright (C) The Internet Society (2003). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assignees. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION Rousskov Expires February 25, 2004 [Page 54] Internet-Draft OPES Callout Protocol Core August 2003 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. Rousskov Expires February 25, 2004 [Page 55]