HTTP/1.1 200 OK Date: Tue, 09 Apr 2002 09:45:18 GMT Server: Apache/1.3.20 (Unix) Last-Modified: Thu, 27 Mar 1997 18:47:00 GMT ETag: "324022-be79-333ac0a4" Accept-Ranges: bytes Content-Length: 48761 Connection: close Content-Type: text/plain Internet Engineering Task Force Vincent P. Laviano INTERNET-DRAFT J. Mark. Pullen draft-laviano-srtp-01.txt George Mason U. March 24, 1997 Expires: August, 1997 Selectively Reliable Transmission Protocol Status of this Memo This document is an Internet Draft. Internet Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet Drafts. Internet Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet Drafts as reference material or to cite them other than as "work in progress." To learn the current status of any Internet Draft, please check the "1id-abstracts.txt" listing contained in the Internet Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast). Distribution of this document is unlimited. Abstract This memo describes a protocol for selectively reliable transmission of Distributed Interactive Simulation (DIS) data in a wide-area multicast environment. The Selectively Reliable Transmission Protocol (SRTP) operates in three distinct modes: best-effort multicast, reliable multicast using negative acknowledgements with a NAK suppression mechanism to avoid congestion at the sender, and lightweight reliable transaction-oriented unicast. SRTP is designed to run in user space and form a sublayer between applications and the kernel-level Internet protocol stack. Table of Contents 1. Introduction 2 2. DIS Communication Requirements 2 3. Protocol Description 4 3.1. Definition of Terms 4 3.2. Message Format 4 4. Protocol Behavior 7 4.1. Mode 0 Operation 7 4.2. Mode 1 Operation 8 4.3. Mode 2 Operation 10 5. Implementation Issues 12 5.1. Host Requirements 12 5.2. API Requirements 12 5.3. An Example API 13 Future Work 14 References 15 Security Considerations 15 Authors' Addresses 16 1. Introduction Distributed Interactive Simulation (DIS) is a real-time multicast application that requires reliable transmission of a subset of its data [DIS94]. Large-group multicast applications such as DIS cannot take advantage of fully reliable transport for all data and still achieve real-time transmission. Since no suitable transport protocol exists today [Pull96], DIS applications must use the best-effort User Datagram Protocol (UDP) in order to achieve real-time transmission. However, in some cases, the use of an unreliable transport protocol forces these applications to perform tasks such as the detection and recovery of lost messages, in order to ensure the reliable transmission of that subset of their data that requires it. These tasks would be performed by the transport protocol if transport protocols meeting the needs of DIS existed. This increases the complexity of DIS applications, making them more costly to design, implement and maintain. This document describes a Selectively Reliable Transmission Protocol (SRTP) that operates in three distinct modes: best-effort multicast, reliable multicast using negative acknowledgements with a NAK suppression mechanism to avoid congestion at the sender, and reliable transaction-oriented unicast We believe that such a standardized protocol would remove an undue burden from DIS application developers and increase interoperability among DIS applications. This document is based upon our prior work with the DIS Standards Community and our experience implementing an SRTP prototype in a UNIX environment [PuLa95] [PuLa96]. Such as protocol can also serve as a significant component of the Runtime Infrastructure (RTI) required by the new DoD High-Level Architecture (HLA). 2. DIS Communication Requirements One of the largest challenges being faced by the DIS community is scaling DIS simulations over a wide-area with thousands to hundreds of thousands of simulated entities. Using the default DIS communication paradigm in which entity state information is exchanged among all participating simulators, the amount of data that must be transmitted over the network and processed by each simulator grows on the order n^2 as the number of simulated entities increases. This is because each of the n participating entities must exchange state information with the other n-1 entities. This amount of traffic makes it difficult for large-scale DIS applications to maintain low enough latencies to support real-time interactions among participants in the distributed virtual environment. In addition to the requirement for low latencies, DIS applications require the reliable exchange of entity state information and entity interaction information among the participating simulators in order to maintain the consistency of the virtual environment across all of the simulators. The original solution to these problems was to broadcast the complete state of each simulated entity to all others at rates from 2 to 15 entity-state PDUs per second. This does not scale to large configurations. A goal of the DIS community that is met by some current DIS applications is to take advantage of the multicast addressing and routing capabilities of the underlying network in order to reduce wide-area network traffic and to reduce the amount of data that must be processed at each receiver. Multicast addressing can be used to filter the data at each receiver by associating multicast groups with spatial, temporal, or functionally related classes of entities [MZPBB95]. Also, it has been noted that frequent transmission of full entity state information as a "heartbeat" for loss detection and for the benefit of late joiners is wasteful because much of this information, such as the country of origin of a vehicle, changes infrequently or not at all. Several algorithms and techniques have been proposed to reduce network traffic between sites by extending the heartbeat interval or eliminating the heartbeat altogether while still dealing with loss detection and late joiners and by compressing entity state information, using a reference/offset scheme [CSTVH95] [HSC95] [VHCS95]. The particular approach proposed for SRTP was inspired by D. Cohen [Cohen94a] [Cohen94b]. In such a scheme, reference PDUs containing infrequently changing entity state information are sent periodically, while smaller offset PDUs containing variations from the reference data are sent more frequently. This idea of sending only state data that has changed is one of the design principles of the DoD High Level Architecture (HLA) for modeling and simulation. The classic tradeoff between the fundamentally conflicting requirements for reliabilty and low latency becomes particularly problematic in the DIS multicast environment. By "reliable" we mean that intact delivery is confirmed automatically with a defined margin of error for confirmation. The Transmission Control Protocol (TCP) is usually used for reliable unicast communication over a wide-area network, but its use of positive acknowledgements to provide reliability makes it impractical in a large-scale multicast environment due to "ACK implosion". Also, to avoid the latency of TCP connection establishment in the middle of a real-time interaction between two entities, connections would have to be established and maintained between all pairs of entities that could possible interact during the course of the simulation, another arrangement that does not scale. The unsuitability of TCP in a large-scale multicast environment has prompted most DIS application developers to use the User Datagram Protocol (UDP). UDP creates minimal overhead, meeting the latency requirements for DIS, but it provides no reliability. In many real-time applications, the loss or corruption of some small portion of an uncompressed data stream is tolerated, because each new PDU supercedes all others, eliminating the value of retransmitting lost or corrupted data. This is the case with uncompressed DIS entity state information, but not with the reference data when a reference/offset scheme is used to compress DIS entity state information. Also, specific interactions in DIS still require reliable communication. Using UDP, each application is forced to make its own provisions for reliable transmission of interaction and reference entity state data. This creates an unnecessary burden for application developers, and it violates the widely-accepted layering principles used to ease the design, implementation and maintenance of communication software. Therefore, we contend that a protocol for selectively reliable transmission should be developed for use in the DIS multicast environment. Such a protocol will also find direct application within the Run-Time Infrastructure (RTI) that supports DoD's new High Level Architecture, because the RTI is responsible to provide efficient real-time networking for distributed simulation in a manner similar to that required for DIS, but more general. In the area of reliable multicast protocols, we doubt that a generic solution is practical. We believe the protocol needed to support distributed simulation is likely to be different from other reliable multicast protocols that have been put forward for multimedia or file delivery. We characterize the DIS and HLA requirements as a selectively reliable transmission protocol with lowest possible latency in a large multicast environment under the circumstances that: 1. Some data is multicast reliably, 2. Most data is multicast best-effort, and 3. Occasional, short, reliable point-to-point transmissions are required among arbitrary members of a large multicast group. 3. Protocol Description SRTP operates in three distinct transmission modes in order to deliver varying levels of reliability that correspond to the needs of individual DIS protocol data units (PDUs): mode 0 for multicast data that does not require reliable transmission, mode 1 for data that must be received reliably by all members of a multicast group, and mode 2 for data that must be received reliably by a single dynamically-determined member of a multicast group. Mode 0 operates as a pure best-effort service for offset data, but with reference to associated mode 1 transmissions for loss detection. Mode 1 operates with negative acknowledgements only, triggered principally by mode 0 arrivals following loss of a mode 1 message. Mode 2 uses a positive acknowledgement for each message to provide reliability and low latency by avoiding the requirement to open a TCP connection. SRTP supports reference/offset schemes for transmitting entity state information by providing an entity ID field that can be used to associate related reference and offset PDUs and by providing semantics that expect a dependent relationship between mode 1 and mode 0 PDUs with the same entity ID. See sections 3.2.4, 4.1.1 and 4.2 for further details. SRTP expects that only a small fraction of PDUs generated by DIS applications will require reliable multicast, and an even smaller fraction will require reliable unicast. This is due to the fact that the vast majority of DIS traffic is composed of Entity State PDUs, as described in section 2 above. 3.1 Definition of Terms Locality Window Given an integral window size, S, in the range 0 to 32767, the locality window about a particular sequence number SN is defined as all sequence numbers SN' in the sixteen bit sequence number space that satisfy: abs((SN - SN') modulo 65536) <= S An implementation must allow itself to be configured with any window size in the range 0 to 32767. 3.2 Message Format An SRTP message consists of a protocol header followed by a variable amount of data. The protocol header, as shown in Figure 1, has a length of 12 bytes, and is part of every message. The remainder of the message is user data. The total length of the SRTP message, including the header, may not exceed 1464 bytes. This value was selected to exceed 1280 bytes, the maximum size of a DIS Entity State PDU. It should also be noted that a 1464-byte SRTP message (1492 bytes including UDP and IP headers) exceeds neither the IEEE 802.3/802.2 MTU of 1492 bytes nor the Ethernet MTU of 1500 bytes, and will therefore not be fragmented by IP in these local-area environments [Stev94]. 0 7 8 11 12 15 0 ------------------------------------------------------------- --- | protocol | message | transmission | | | version | type | mode | | 16 ------------------------------------------------------------- | | entity ID | | | | | 32 ------------------------------------------------------------- | | category code | | | | 48 ------------------------------------------------------------- message | sequence number (SN) | header | | 64 ------------------------------------------------------------- | | | | | | | 80 | destination group address | | | | | | | | 96 ------------------------------------------------------------- --- | payload user data (data messages only) | ... (variable length, not to exceed 1452 bytes) ... | | ------------------------------------------------------------- Figure 1. SRTP Message Format Protocol Version The first byte of the message is the protocol version number. In the current version 0x01, if a received message does not have a version field with a value of 0x01 it is to be silently discarded. Message Type Bits 8, 9, 10 and 11 represent the message type. The following table lists the possible field values and describes their meaning. bit 8 bit 9 bit 10 bit 11 description 0 0 0 0 data message 0 0 0 1 negative acknowledgement (NAK) 0 0 1 0 positive acknowledgement (ACK) all other bit patterns reserved for future use Transmission Mode Bits 12, 13, 14 and 15 represent the transmission mode for the message. The following table lists the possible field values and describes their meaning. bit 12 bit 13 bit 14 bit 15 description 0 0 0 0 mode 0: best-effort multicast 0 0 0 1 mode 1: reliable multicast 0 0 1 0 mode 2: reliable unicast all other bit patterns reserved for future use Type 1 (NAK) messages are required to be mode 1, and type 2 (ACK) messages are required to be mode 2. The three transmission modes will be discussed in detail in section 4 of this document. Entity ID Bytes 3 and 4 of the header are the entity ID of the message. An entity ID is an integer in the range 0 - 65535 that is used to associate related messages with each other. The intent of the entity ID field is to support reference/offset entity state compression, but the field can be used by an application to support any data stream in which the mode 0 messages are dependent on mode 1 messages transmitted earlier. To circumvent the problem of assigning globally unique entity IDs, the value of the entity ID field will be logically appended to the IP address of the sending host (available from the IP header) when a unique entity ID is required. This mechanism allows each host to generate 65536 different data streams. Category Code Bytes 5 and 6 of the header are the category code of the message. A category code is an integer in the range 0 - 65535 that is used to label messages of different application-defined classes for relevance filtering purposes. A message that is labeled with category code 0 cannot be filtered and is received by all members of the multicast group to which it is sent. The use of category 0 when not strictly necessary is strongly discouraged. This mechanism allows the set of messages to be partitioned into up to 65535 different classes. Messages labeled with the same category code may use different transmission modes. Sequence Number (SN) Bytes 7 and 8 of the header are the sequence number, or SN, of the message. The SN is an integer in the range 0 - 65535 that is used to detect missing messages and to associate specific control messages with specific data messages. The precise use of the SN field varies based on the type and transmission mode of the message. Specific details are given throughout section 4 of this document. Destination Group Address The last four bytes of the header are the IP address to which the message should be delivered. This is an IP multicast address for mode 0 and mode 1 messages and a unicast IP address for mode 2 messages. This field is redundant with the destination IP address in the IP header, but serves to identify to which multicast group a received multicast message was sent in the case where the message is received from a communication endpoint (e.g., a UNIX socket) that has multiple multicast group memberships. The authors are not aware of any other way to obtain this information, and welcome suggestions from readers with experience in this area. Payload User Data Type 1 (NAK) and type 2 (ACK) messages consist solely of a 12-byte message header and do not contain any user data. Type 0 (data) messages contain up to 1452 bytes of payload user data following the message header. An implementation of SRTP must not examine, use, or change this data. It is permissible for a data message to contain 0 bytes of user data. 4. Protocol Behavior This section describes the behavior of each of the three transmission modes, both from the perspective of responding to application requests and from the perspective of responding to received SRTP messages. The various requests that can be made of an SRTP implementation by an application should be considered part of the protocol, but no particular API is mandated. A example API is described in section 5 of this document. 4.1 Mode 0 Operation Mode 0 is for multicast messages that do not require reliable transmission, because they are part of a real-time stream of data that is periodically updated with high frequency. It is assumed to be pointless for a receiver to request retransmission of such messages when they are lost or corrupted, because any such message is very likely to have been superceded by a more recent update by the time retransmission is complete. It is often more efficient for the receiver to simply wait for the next update to arrive. Mode 0 messages are intended to be used as frequent state updates (in offset form) to some (less frequently updated) reference state. Such offset messages will be expected to be part of a data stream that also contains reference messages (mode 1 messages with the same entity ID). Due to the constraints imposed in section 3.2.3, mode 0 messages are always data messages and never control messages. DIS Entity State PDUs are an example of data that should be transmitted using mode 0. Sending Mode 0 Messages When an application requests transmission of mode 0 data, an entity ID, a category code and a destination multicast group must be provided to SRTP along with the data to be sent. If the category code is nonzero, at least one mode 1 message with the same entity ID must already have been sent to the destination multicast address by this instance of SRTP. After the parameters are validated, the following steps must be performed by SRTP: 1. An SRTP message must be generated with the following characteristics: the version is set to 0x01, the message type is set to 0x0, the transmission mode is set to 0x0, the entity ID and category code fields are set to the application-provided values, and the destination group address is set to the application-provided IP multicast address. If the category code is 0, the SN is set to 0x00. Otherwise, the SN is set equal to the SN of the most recently sent mode 1 message with the same entity ID. Finally, the user data is included after the message header. If the message cannot be generated as described above, the user data is discarded and the error must be reported to the application. 2. If step 1 was completed without error, the newly generated message must be sent to the IP multicast address contained in its destination address field, by means of a UDP datagram. If the UDP interface on the sending system reports an error to SRTP when the attempt to send the SRTP message is made, an implementation may attempt to resend the message any finite number of times. The implementation must report to the application whether or not the message was ultimately accepted by UDP. Receiving Mode 0 Messages When a mode 0 message is received by SRTP it should be processed as follows after verifying the version, message type, and destination group address fields: 1. If the category code is 0, the SN must be verified to be 0x00. If this is not the case, the message must be silently discarded. Once the sequence number has been verified to be 0x00, the user data must be delivered to all applications that have registered for the multicast group contained in the message's destination group address field. The delivery mechanism is left up to individual implementations. 2. If the category code is nonzero, the SN must be verified to be equal to the SN of the most recently received mode 1 message with the same entity ID sent to the same multicast address as that contained in the message's destination group address field. If this test is successful, the user data must be delivered to all applications that have registered for both the multicast group and category code of the message. If the test is unsuccessful because no such mode 1 message exists, SRTP must consider sending a NAK for the mode 1 message in question. This process is described in detail below. If (1) an appropriate mode 1 message exists, (2) the sequence number of the mode 0 message is within the locality window about the SN of the mode 1 message, and (3) the SN of the mode 0 message is less than that of the mode 1 message with respect to the locality window (which is guaranteed to be smaller than the entire sequence number space), the message must be silently discarded. Otherwise, SRTP must consider sending a NAK for a mode 1 message with the same entity ID and sequence number as the mode 0 message to the multicast address in the destination group address field of the mode 0 message. 4.2. Mode 1 Operation Mode 1 is for multicast data that requires reliable transmission. A mode 1 message can be either a data message or a NAK. Mode 1 data messages are expected to be part of a data stream that is likely to contain mode 0 messages as well (see section 4.1), but it is possible for a data stream to be comprised solely of mode 1 messages. Sending Mode 1 Data Messages When an application requests the transmission of mode 1 data, an entity ID, a category code, and a destination multicast group must be provided to SRTP along with the data to be sent. After validating the parameters, the following steps must be performed by SRTP: 1. An SRTP message must be generated with the following characteristics: the version is set to 0x01, the message type is set to 0x0, the transmission mode is set to 0x1, the entity ID and category code fields are set to the application-provided value, and the destination group address is set to the application-provided IP multicast address. The SN is set equal to the SN of the most recently sent mode 1 message with the same entity ID, incremented by 1 modulo 65536. If no such mode 1 message exists, a uniformly distributed random number in the interval 0 to 65535 is generated and assigned as the SN for the new message. Finally, the user data is included after the message header. If the message cannot be generated as described above, the user data is discarded and the error must be reported to the application. 2. The newly generated message must then be buffered, replacing any formerly buffered mode 1 message of the same entity ID and destination multicast address. If the message cannot be buffered, the user data is discarded and the error must be reported to the application. 3. If steps 1 and 2 were completed without error, the newly generated message must be sent to the IP multicast address contained in its destination group address field, by means of a UDP datagram. If the UDP interface on the sending system reports an error to SRTP when the attempt to send the SRTP message is made, an implementation may attempt to resend the message any finite number of times. The implementation must report to the application whether or not the message was ultimately accepted by UDP. Receiving Mode 1 Data Messages When a mode 1 data message is received by SRTP it should be processed as follows after validating the version and destination group address fields: 1. If no mode 1 message with the same entity ID has yet been received, the SN of the received message must be buffered, and the payload user data must then be delivered to all applications that have registered for the category and multicast group of the received message. 2. If a mode 1 message with the same entity ID has already been received, then the SN of the most recently received such message will have been buffered. If the SN of the received message is within the locality window about the buffered SN and the SN of the received message is less than the buffered SN with respect to the locality window, the message must be silently discarded. Otherwise, the SN of the received message must be buffered, replacing the previously buffered SN for this entity ID, and the payload user data must then be delivered to all applications that have registered for the category and multicast group of the received message. 3. Additionally, when the new SN is buffered, NAK suppression information should be reset for the appropriate entity ID. See section 4.2.4. below. Sending a Negative Acknowledgement Negative Acknowledgements (NAKs) are triggered by the receipt of a mode 0 message of a nonzero category bearing a sequence number that indicates that a mode 1 message with the same entity ID has been lost. NAKs are used by SRTP for reliable multicast messages in order to avoid an "ACK implosion" at the original sender. However, with a large multicast group spread out over a congested wide-area network, there is the potential for enough group members to fail to receive the message and generate NAKs to cause considerable congestion at the original sender despite the use of NAKs instead of ACKs. For this reason, SRTP uses a NAK suppression mechanism to reduce the number of NAKs generated in response to any single lost message [VHCS95]. The NAK suppression mechanism is as follows: 1. When SRTP wants to send a NAK for a particular mode 1 message, it must first check to see if any member of the message's destination multicast group has sent a NAK for that message within some user- configurable time interval that we will refer to as the "NAK threshold." If not, a NAK is generated with version set to 0x01, message type set to 0x1, transmission mode set to 0x1. Entity ID, category code, SN, and destination address are set to that of the mode 1 message for which the NAK is being sent. As noted before, a NAK consists solely of a header and thus the user data field remains empty. If a NAK has been sent by any member of the destination multicast group for the mode 1 message in question within the NAK threshold, no NAK is generated. 2. If a NAK is generated, it is sent at a randomly generated time in the near future. This is to avoid the situation in which several receivers all detect that a message has been missed at roughly the same time, determine that no member of their multicast group has sent a NAK for the message, and all send NAKs simultaneously -- thus creating the situation we hoped to avoid. The interval of time for which to wait before sending a NAK is an uniformly distributed random number in the range 0 to 2^N, where N is the number of the times a NAK has been sent for this particular mode 1 message by any member of the destination multicast group. Receiving a Negative Acknowledgement When a NAK is received by SRTP, it must processed as follows after the version, transmission mode, category code, and destination group address have been validated: 1. If this instance of SRTP has ever sent a mode 1 message with the entity ID indicated in the NAK to the multicast address indicated in the NAK, the most recent such message (which is buffered) should be immediately retransmitted to the multicast address indicated in the received NAK. 2. Whether or not step 1 results in the retransmission of a message, the event of receiving the NAK and the (local machine) time at which the NAK was received should be buffered. Each instance of SRTP should know how many NAKs have been received for each entity ID-multicast address pair since the most recent mode 1 message of the same pair was received and the time at which the most recent of these NAKs was received. This is the information that is reset in step 3 in section 4.2.2. above. 4.3. Mode 2 Operation Mode 2 is for infrequent reliable transaction-oriented communication between two dynamically determined members of a multicast group. TCP could be used for such communication, but there would be unnecessary overhead in establishing a stream-oriented connection for a single exchange of data, whereas there is already an ongoing stream of best-effort data between the hosts that require mode 2 transmission. A DIS collision PDU is an example of data that would benefit from using transmission mode 2. Sending Mode 2 Data Messages When an application requests the transmission of mode 2 data, an entity ID, a category code, and a destination unicast IP address must be provided to SRTP along with the data to be sent. After the parameters are validated, the following steps must be performed by SRTP: 1. An SRTP message must be generated with the following characteristics: the version is set to 0x01, the message type is set to 0x0, the transmission mode is set to 0x2, the entity ID and category code fields are set to the application-provided values, and the destination address is set to the application-provided IP address. The SN is set equal to the SN of the most recently sent mode 2 message with the same entity ID, incremented by 1 modulo 65536. If no such mode 2 message exists, a uniformly distributed random number in the interval 0 - 65535 is generated and assigned as the SN for the new message. Finally, the user data is included after the message header. If the message cannot be generated as described above, the user data is discarded and the error must be reported to the application. 2. The newly generated message must then be buffered. This new message does not replace any formerly buffered mode 2 messages. An implementation must provide a mode 2 message buffer that can hold one or more mode 2 messages. Mode 2 messages are expected to be infrequent, but it is still strongly recommended that an implementation provide a buffer of user- configurable size that can hold more than a single mode 2 message. If the message cannot be buffered, the user data is discarded and the error must be reported to the application. 3. If steps 1 and 2 were completed without error, the newly generated message must be sent to the IP address contained in its destination address field, by means of a UDP datagram. If the UDP interface on the sending system reports an error to SRTP when the attempt to send the SRTP message is made, an implementation may attempt to resend the message any finite number of times. The implementation must report to the application whether or not the message was ultimately accepted by UDP. 6. If some user-configurable period of time (which should be greater than the worst-case round-trip time for the multicast group) which we will refer to as the "ACK threshold" elapses without receipt of an ACK for the mode 2 message, it must be retransmitted. It is permissible for an implementation to define a maximum number of retransmissions that will be attempted before the mode 2 message is removed from the buffer, provided that this number is greater than or equal to one. It is strongly recommended that this number be a parameter of the application request for mode 2 transmission. Receiving Mode 2 Data Messages When a mode 2 data message is received by SRTP it should be processed as follows after the destination group address is validated: 1. For mode 2 messages, the sequence number field is actually used as a transaction id to associate the required ACK with a specific mode 2 message. For this reason, the payload user data is unconditionally delivered to all applications that have registered for the category and multicast group of the received message, regardless of the value of the SN field. 2. Additionally, an ACK must be sent to the host from which the mode 2 data message originated. See section 4.3.3. below for details. Sending a Positive Acknowledgement An ACK is triggered by the receipt of a mode 2 data message. Sending an ACK is straightforward: a new SRTP message is generated with version set to 0x01, message type set to 0x2, and transmission mode set to 0x2. Entity ID, category and SN must reflect those of the mode 2 data message being acknowledged. The destination address field must be set to the source IP address from which the data message was received. Since mode 2 data is unicast, no suppression mechanism is required for ACKs. The ACK should be transmitted with no delay. Receiving a Positive Acknowledgement When an ACK is received by SRTP, it must processed as follows after the version, transmission mode, and destination address have been validated: 1. If a mode 2 message with the same entity ID, SN, and destination address as the received ACK resides in the receiving host's mode 2 message buffer, it must be removed from the buffer. Otherwise, the ACK must be silently discarded. 5. Implementation Issues This section discusses implementation issues such as host requirements and API requirements, and gives an example of an API that satisfies these requirements. 5.1. Host Requirements SRTP has been designed to run in user space and form a sublayer between applications and the kernel-level UDP and IP protocols. Therefore, any host upon which SRTP is run must provide an Internet protocol stack with UDP, IP, and IGMP enabled. In addition, if SRTP is to be used beyond the confines of a local area network, some form of multicast routing must be available. A single instance of SRTP is intended to run on each SRTP-capable host. 5.2. API Requirements In addition to providing facilities to send and receive mode 0, 1 and 2 data, an implementation of SRTP must also provide a number of other facilities to applications: creation and deletion of an association with SRTP, registration and unregistration of interest in data based on categories and multicast groups, and the ability to set various user configurable parameters. Most of these facilities are described in more detail below. However, we will not discuss the setting of user configurable parameters in any more detail here. Creation and Deletion of Associations An implementation of SRTP must allow applications to create and delete associations with SRTP. An association represents state about the application that is needed by SRTP in order to allow the application to send and receive data, and register and unregister interest in categories and multicast groups. Registration and Unregistration of Interest An implementation of SRTP must allow applications to register interest in any subset of the nonzero categories. The result of such a registration is that the application will receive all data (mode 0, 1 and 2) that is labeled with the registered category code. An implementation must also allow applications to unsubscribe from any currently registered nonzero categories. The result of unsubscription is that the application will receive no data of the category that was received after the unsubscription took place (until the application again registers interest in the category). All applications are automatically registered for category 0 at all times. An application may not unsubscribe from category 0. Similarly, an implementation must allow applications to register and unregister interest in data sent to any multicast group. An application may send SRTP data without being registered for any multicast group. An implementation must support at least 20 unique multicast group subscriptions at any one time. It is strongly recommended that at least an order of magnitude more be supported (this can be achieved in UNIX through the use of multiple sockets and the "select" system call). 5.3. An Example API SRTP_INIT Purpose: Create an association between an application and SRTP. Calling Sequence: status = srtp_init(); Parameters: None. SRTP_TERM Purpose: Delete an association between an application and SRTP. Calling Sequence: status = srtp_term(); Parameters: None. SRTP_REG_CAT Purpose: Register interest in data labeled with a certain category code. After calling this function, a client will be able to receive all subsequent data of the registered category that is sent to any multicast groups to which the client is registered. Calling Sequence: status = srtp_reg_cat(category_code); Parameters: category_code: category code for which to register interest. SRTP_UNREG_CAT Purpose: Unregister interest in data labeled with a certain category code. After calling this function, a client will no longer receive new data of the registered category. Calling Sequence: status = srtp_unreg_cat(category_code); Parameters: category_code: category code for which to unregister interest. SRTP_REG_MCG Purpose: Register interest in data sent to a certain multicast group. After calling this function, a client will be able to receive all subsequent data that is sent to this multicast group, provided that it is labeled with a category that the client has registered an interest in. All clients are implicitly registered for category 0 at all times. Calling Sequence: status = srtp_reg_mcg(multicast_address); Parameters: multicast_address: multicast group for which to register interest. SRTP_UNREG_MCG Purpose: Unregister interest in data sent to a certain multicast group. After calling this function, a client will no longer receive new data that is sent to this multicast group. Calling Sequence: status = srtp_unreg_mcg(multicast address); Parameters: multicast_address: multicast group for which to unregister interest. SRTP_SEND Purpose: Provide data to SRTP for transmission. Calling Sequence: status = srtp_send(message, length, options); Parameters: message: user data to be sent. length: length of user data in bytes. options: a structure containing options such as transmission mode, entity ID, category code, destination group address, and multicast TTL. SRTP_RECV Purpose: Receive a message from SRTP into an application-provided buffer. This function is nonblocking. Calling Sequence: status = srtp_recv(buffer); Parameters: buffer: buffer for received message. Future Work The impact of reliable multicast traffic on other Internet traffic in times of congestion is a particular concern for the IETF at this time [MaRo96]. While it is unlikely that most DIS applications, which are primarily military simulations, will be run across the global Internet, it is still valuable to examine the behavior of SRTP in times of congestion. The authors believe that the NAK-suppression used during mode 1 (reliable multicast) transmission will help reduce congestion on the wide-area network, but specific experiments need to be conducted and measurements taken in the future in order to confirm this hypothesis. We are also considering the addition of a logging mechanism, as described in [HCS95], in which each site would elect one instance of SRTP to act as a logging server. The logging server would be responsible for buffering mode 1 messages. Whenever such a message was determined to have been lost, SRTP would identify its logging server and a NAK would be directed to this server instead of the orginal sender. If possible, the logger would retransmit the buffered message. Only if the message was not available at the logger would a NAK be sent over the wide-area network to the orignal source of the message. This approach avoids unnecessary interaction over the wide-area network, reducing overall latency and congestion. We believe that these additions would contribute towards the scalability of SRTP in the wide-area environment. References [CSTVH95] Calvin, J. O., J. Seeger, G. D. Troxel and D. J. Van Hook, "STOW Realtime Information Transfer and Networking System Architecture", 12th DIS Workshop on Standards for the Interoperability of Distributed Simulations, March 1995. [Coh94a] Cohen, Danny, "NG-DIS-PDU: The Next Generation of DIS-PDU (IEEE-P1278)", 10th DIS Workshop on Standards for the Interoperability of Distributed Simulations, March 1994. [Coh94b] Cohen, Danny, "DIS -- Back to Basics", 11th DIS Workshop on Standards for the Interoperability of Distributed Simulations, September 1994. [DIS94] Dis Steering Committee, The DIS Vision: A Map to the Future of Distributed Simulation, May 1994. [HCS95] Holbrook, H. W., S. K. Singhal and D. R. Cheriton, "Log-Based Receiver-Reliable Multicast for Distributed Interactive Simulation", Proceedings of SIGCOMM '95, August 1995. [MZPBB95] Macedonia, Michael R., Zyda, Michael J., Pratt, David R., Brutzman, Donald P., Barham, Paul T. "Exploiting Reality with Multicast Groups: A Network Architecture for Large-Scale Virtual Environments", IEEE Computer Graphics and Applications, September 1995 pp. 38-45. [MaRo96] Mankin, A. and A. Romanow, IETF Criteria for Evaluating Reliable Multicast Transport and Application Protocols, Work in Progress (Internet Draft), November 1996. [PMB96] Pullen, J. M., M. Myjak, and C. Bouwens, Limitations of Internet Protocol Suite for Distributed Simulation in the Large Multicast Environment, Work in Progress (Internet Draft), September 1996. [PuLa95] Pullen, J. M. and V. P. Laviano, "A Selectively Reliable Transport Protocol for Distributed Interactive Simulation", 13th DIS Workshop on Standards for the Interoperability of Distributed Simulations, September 1995. [PuLa96] Pullen, J. M. and V. P. Laviano, "Prototyping the Selectively Reliable Transport Protocol", 14th DIS Workshop on Standards for Distributed Interactive Simulation, March 1996. [Stev94] Stevens, W. R., TCP/IP Illustrated, Volume 1, Addison-Wesley, Reading, Mass., 1994. [VHCS95] Van Hook, D. J., J. O. Calvin, and J. E. Smith, "Data Consistency Mechanisms to Support Distributed Simulation", 13th DIS Workshop on Standards for Distributed Interactive Simulation, September 1995. Security Considerations DIS applications are primarily military simulations and are therefore likely to be run across private networks instead of the global Internet. The use of encryption and authentication mechanisms (e.g., IPSEC) should provide sufficient security for this environment. Hosts are considered secure, and each instance of SRTP trusts its own clients that reside on the same host. Authors' Addresses Vincent P. Laviano George Mason University C3I Center Mail Stop 4B5 Fairfax, VA 22030 Phone: (703) 993-1705 E-mail: vlaviano@cne.gmu.edu J. Mark Pullen George Mason University C3I Center/Computer Science Mail Stop 4A5 Fairfax, VA 22030 Phone: (703) 993-1538 E-mail: mpullen@gmu.edu