INTERNET-DRAFT R. Huang Intended Status: Standard Track Huawei October 19, 2015 RTP Payload Format for Interleaved Packets draft-huang-payload-rtp-interleave-01 Abstract This memo introduces a common RTP encapsulation for interleaved media. This method can be applied to any RTP payload formats for any applications when latency is not an issue. Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Copyright and License Notice Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect Expires April 21, 2016 [Page 1] INTERNET DRAFT October 19, 2015 to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Interleaving . . . . . . . . . . . . . . . . . . . . . . . . . 3 4. Interleaving RTP Payload Format . . . . . . . . . . . . . . . . 5 4.1 RTP Header . . . . . . . . . . . . . . . . . . . . . . . . . 5 4.2 Payload Structure . . . . . . . . . . . . . . . . . . . . . 6 4.3 Encapsulation frames . . . . . . . . . . . . . . . . . . . . 7 4.4 Example Packet . . . . . . . . . . . . . . . . . . . . . . . 8 5 IANA Considerations. . . . . . . . . . . . . . . . . . . . . . 10 5.1 Registration of audio/genitl . . . . . . . . . . . . . . . . 10 5.2 Registration of video/genitl . . . . . . . . . . . . . . . . 11 5.3 Usage of Interleaving . . . . . . . . . . . . . . . . . . . 12 6 Security Considerations . . . . . . . . . . . . . . . . . . . . 12 7 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 12 8 References . . . . . . . . . . . . . . . . . . . . . . . . . . 13 8.1 Normative References . . . . . . . . . . . . . . . . . . . 13 8.2 Informative References . . . . . . . . . . . . . . . . . . 13 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 13 Expires April 21, 2016 [Page 2] INTERNET DRAFT October 19, 2015 1 Introduction Interleaving is an effective method to disperse packet loss bursts into a series of isolated small losses which in general are easier to recover from and produce lower total distortion. It provides the advantages of requiring no increase in bit rate and can be combined with other types of error-resilience techniques like FEC, but at the cost of increasing latency. Interleaving is quite useful for applications which are not such sensitive of latency in network environments afflicted by fading an interference which may lead to burst losses, e.g., DSL and wireless network. This memo introduces a common RTP encapsulation for interleaved media. This method can be applied to any RTP payload formats for any applications when latency is not an issue. 2 Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. This document uses the following terms: Interleaving length: the packet number of an interleaving separation between packets or data originally adjacent. Interleaving depth: the interleaving separation count of an interleaver output buffer. Usually an interleaver output buffer size is equal to interleaving length*interleaving depth. 3. Interleaving An interleaver is simply used at the sender or at a middle box to interleave the RTP packets before transmission through the network. There are a lot of interleaving algorithms. A simple one could be that packets are firstly read into the interleaver in rows, with each row corresponding to a sequence block of n packets; and packets are transmitted by columns as soon as m rows of packets fill up. The interleaver permutes the packets so that the location of burst losses are converted into isolated ones. The effective ness of the interleaver depends on the interleaving length and interleaving depth, however, this is at the cost of latency. At the receiver, an interleaved packet cannot be used until all the packets it depends on are received. Figure 1 shows the interleaving scheme with n=4 and m=3, where m indicates the interleaving depth and n indicates the interleaving Expires April 21, 2016 [Page 3] INTERNET DRAFT October 19, 2015 length. For this interleaver, the i-th packet in the original order has to be transmitted in the (((i-1) mod n)*m + ((i-1) div n) + 1)-th place. The highest decoding delay corresponding to this interleave is (n-1)*(m-1). OUT n || || || || --0----1----2----3---> || || || || IN m --4----5----6----7---> || || || || --8----9----10---11--> || || || || \/ \/ \/ \/ Figure 1 Packet interleaving with interleaving length n=4 and depth m=3 This kind of interleaving, called packet leaving in this document, is useful for audio applications that are non-interactive because it would turn a burst of consecutive lost packets into a series of isolated packet loss events. However, for video frames that are usually large enough to be fragmented into several packets, losing several non-consecutive packets from a large frame may not lead to perceptually less degradation. Thus, some implementations may use another interleaving to help loss recovery: The payload of each RTP packet is divided into n parts and the interleaver combines some part of one packet with some part of other packets to form a new RTP packet. As Figure 2 shows, each of the original RTP packets P1, P2, P3 and P4 are divided into 4 packets. The interleaver combines the first parts of P1, P2, P3 and P4 together to form a new packet P1'. P2', P3' and P4' are formed in the same way. In this case, if P1' is lost during the transmission, P1, P2, P3 and P4 can be easily recovered by FEC mechanism. It is specified as data interleaving in this document. In data interleaving, the number of separations of one packet has a fix value. To simplify the mechanism, it is required to be equal to the interleaving depth in the document. OUT n P1' P2' P3' P4' || || || || P1 --P1(0)----P1(1)----P1(2)----P1(3)---> || || || || IN m P2 --P2(0)----P2(1)----P2(2)----P2(3)---> || || || || P3 --P3(0)----P3(1)----P3(2)----P3(3)---> || || || || P4 --P4(0)----P4(1)----P4(2)----P4(3)--> Expires April 21, 2016 [Page 4] INTERNET DRAFT October 19, 2015 \/ \/ \/ \/ Figure 2 An illustration of data interleaving However, data interleaving method introduces extra complexity when dividing and recovering packets. To reduce delay in some degree, some ways can be considered. For example, the interleaver has the ability to categorize the RTP packets into important ones and unimportant ones based on some application dependent rules, and then only applies this interleaving to the important packets, e.g., packets containing I frames, IDR frames or some other information frames, while leave unimportant ones intact. This method can achieve a compromise between reducing complexity caused delay and reducing burst losses. The detail discussion of how to reduce the interleaving delay is out of scope of this document. Proposing a common interleaving RTP encapsulation allows any RTP payload format to use the interleaving scheme freely when needed. 4. Interleaving RTP Payload Format This section introduces a new PTP payload format dedicated for interleaving. That means interleaved RTP packets could use this new RTP payload format when transmitting. It is allowed that interleaving RTP payload format is transmitted together with the uninterleaved payload format so that the de-interleaver can identify the interleaved packets to recover. In such a case, the specifications defined in [draft-ietf-avtcore-multi-media-rtp-session] SHOULD be considered. Interleaving can change the number of RTP packets, however the original RTP packets MUST be recovered at the de- interleaver. 4.1 RTP Header The format of RTP header is specified in [RFC3550] and showed in Figure 2 for convenience. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC |M| PT | sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | synchronization source (SSRC) identifier | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | contributing source (CSRC) identifiers | | .... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Expires April 21, 2016 [Page 5] INTERNET DRAFT October 19, 2015 Figure 2 RTP header of RFC3550 The RTP header information to be set according to this RTP payload format is set as follows: marker (M): 1 bit The interpretation of the marker is determined by the interleaved frame encapsulated in this payload. It MUST be set to the value that the market bit of the frame would have been if it were transported in its own RTP packets. If multiple frames are packed into one RTP packet, the marker bit in the RTP header MUST be set to the value in accordance with the last frame. payload type (PT): 7 bits The assignment of the RTP payload type for this format is outside the scope of this memo and will not be specified here. The assignment of a payload type has to be performed either through the profile used or in a dynamic way. However, only the packets of the same one RTP stream are allowed to interleaved in one interleaving stream which is identified by the payload type. sequence number: 16 bits Set and used in accordance with RFC 3550. timestamp: 32 bits Set to the value that the timestamp of the frame would have been if it were transported in its own RTP packets. If multiple frames are packed into one RTP packet, the timestamp in the RTP header MUST be set to the timestamp of the last frame which would have been if it were transported in its own RTP packets. 4.2 Payload Structure This section describes the interleaving payload structure. The interleaving payload structure is composed from a fixed interleaving header and one or multiple encapsulation frames, which is illustrated in the following figure. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | EPT | | +-+-+-+-+-+-+-+ | | | Expires April 21, 2016 [Page 6] INTERNET DRAFT October 19, 2015 | one or multiple encapsulation frames | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | :...OPTIONAL RTP padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 3: Format for the interleaving payload structure encapsulation payload type (EPT): 7 bits This field indicates the payload type of the encapsulation frames would have been if it were transported in their own RTP packets. Encapsulation frames with different payload type MUST NOT be packed into one RTP packet. The assignment of the encapsulation payload type has to be performed either through the profile used or in a dynamic way. 4.3 Encapsulation frames The structure of encapsulation frames is showed in the figure 4. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |T| SN Offset | timestamp offset (optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | size (optional) | | |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | | Original RTP payload | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | :...OPTIONAL RTP padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 4: Format for the encapsulation frame structure encapsulation frame type (T): 1 bit This field indicates the encapsulation frame type. T=0: Last encapsulation frame - This frame can be used alone or as the last encapsulation frame in an aggregation RTP packet which combines multiple encapsulation frames. T=1: Aggregated encapsulation frame - This frame MUST be only used in an aggregation RTP packet combining multiple encapsulation frame. And it MUST NOT be used as the last Expires April 21, 2016 [Page 7] INTERNET DRAFT October 19, 2015 encapsulation frame when used in the aggregation RTP packet. SN Offset: 8 bits This field records the signed offset between the original sequence number that the encapsulation frame would have been if it were transported in its own RTP packets and the sequence number field in this RTP header. The original sequence number implying the decoding order can be calculated as following: original sequence number = sequence number + SN offset A negative SN offset indicates that the encapsulation frame is delayed for transmission, and a non-negative SN offset means the encapsulation frame is transmitted ahead of it should be. When one RTP packet is divided into several parts to be interleaved, one part is encapsulated as one encapsulation frame and the different encapsulation frames belonging to the same RTP packet share the same SN offset. If multiple encapsulation frames combined in one RTP packet have the same SN offset, they SHOULD be handled according to their orders arranged in this packet. timestamp offset: 23 bits This field records the signed offset between the original timestamp that the encapsulation frame would have been if it were transported in its own RTP packets and the timestamp field in this RTP header. It is only applicable for aggregated encapsulation frame (T=1). The original timestamp can be calculated as following: original timestamp = timestamp + timestamp offset size: 16 bits This field is an unsigned size information of the following encapsulation frame, which does not include encapsulation frame type, SN offset and timestamp offset. It is only applicable for aggregated encapsulation frame (T=1). 4.4 Example Packet This section presents two example of an interleaving RTP packet. Figure 5 shows an example of an packet that only contain one encapsulation frame. In this example, the original timestamp is the timestamp in the RTP header. Expires April 21, 2016 [Page 8] INTERNET DRAFT October 19, 2015 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | EPT |0| SN Offset | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | | Original RTP payload | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | :...OPTIONAL RTP padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 5 An interleaving RTP packet including only one encapsulation frame Figure 6 shows an example of an packet that contains 3 encapsulation frame, labeled as 1, 2 and 3 in the figure. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | EPT |1| SN Offset 1 | TS offset 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TS offset 1| size 1 | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | : original RTP payload 1 : : : | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |1| SN Offset 2 | TS offset 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TS offset 2 | size 2 | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | : original RTP payload 2 : : : | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |0| SN Offset 3 | | | +-+-+-+-+-+-+-+-+-+ | : original RTP payload 3 : : : | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | :...OPTIONAL RTP padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 5 An interleaving RTP packet Expires April 21, 2016 [Page 9] INTERNET DRAFT October 19, 2015 including 3 encapsulation frames 5 IANA Considerations. This section specifies the new media subtypes registered with IANA and the associated parameters that MUST be used to indicate the features of the RTP stream. 5.1 Registration of audio/genitl The media subtype for audio interleaving payload is allocated from the IETF tree. The receiver MUST ignore any unrecognized parameter. Media Type name: audio Subtype name: genitl Required parameters: codec: This parameter indicates the original payload type of the interleaved payload. length: This parameter indicates the interleaving length of the interleaver. depth: This parameter indicates the interleaving depth of the interleaver. Optional parameters: type: This parameter indicates the interleaving type described in section 3. The permissible values are 0 and 1, where 0 indicates packet interleaving and 1 indicates data interleaving. If omitted, it has the default value of 0. Encoding considerations: This format is framed Interoperability considerations: none Published specification: this document. Applications that uses this media type: Additional information: none Person & email address to contact for further information: Expires April 21, 2016 [Page 10] INTERNET DRAFT October 19, 2015 rachel.huang@huawei.com Intended usage: COMMON Restrictions on usage: This media type depends on RTP framing, and hence is only defined for transfer via RTP. Transport within other framing protocols SHALL NOT be defined as this is a robustness mechanism for RTP. Author: Rachel Huang Change controller: IETF Payload Working Group delegated from the IESG 5.2 Registration of video/genitl The media subtype for video interleaving payload is allocated from the IETF tree. The receiver MUST ignore any unrecognized parameter. Media Type name: video Subtype name: genitl Required parameters: length: This parameter indicates the interleaving length of the interleaver. depth: This parameter indicates the interleaving depth of the interleaver. Optional parameters: type: This parameter indicates the interleaving type described in section 3. The permissible values are 0 and 1, where 0 indicates packet interleaving and 1 indicates data interleaving. If omitted, it has the default value of 0. Encoding considerations: This format is framed Interoperability considerations: none Published specification: this document. Applications that uses this media type: Additional information: none Expires April 21, 2016 [Page 11] INTERNET DRAFT October 19, 2015 Person & email address to contact for further information: rachel.huang@huawei.com Intended usage: COMMON Restrictions on usage: This media type depends on RTP framing, and hence is only defined for transfer via RTP. Transport within other framing protocols SHALL NOT be defined as this is a robustness mechanism for RTP. Author: Rachel Huang Change controller: IETF Payload Working Group delegated from the IESG 5.3 Usage of Interleaving the interleaving stream can be sent instead of the original stream or multiplexed with the original stream. In the latter case, interleaving packets are sent alternatively with part of the original packets, and the de-interleaver can identify them by different payload types. The SDP example is illustrated as following: m=audio 10000 RTP/AVP 100 a=rtpmap:96 G722/8000 a=rtpmap:100 genintl/8000 a=fmtp:100 96/4/3 This SDP indicates that an audio stream 100 is presented. The payload identifier 96 is the original payload payload type and 100 is the interleaved payload type. So this audio stream is an interleaved audio stream, who's original payload type is G.722 and the corresponding interleaving length and interleaving depth are 4 and 3. m=video 49600 RTP/AVP 100 101 a=rtpmap:100 H264/90000 a=rtpmap:101 genintl/90000 a=fmtp:101 100/4/3 This SDP indicates that interleaved RTP packets are sent together with some of the uninterleaved original packets. 6 Security Considerations TBD 7 Acknowledgments Expires April 21, 2016 [Page 12] INTERNET DRAFT October 19, 2015 TBD 8 References 8.1 Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3550] Schulzrinne, H., "RTP: A Transport Protocol for Real-Time Applications", RFC 3550, July 2003. [draft-ietf-avtcore-multi-media-rtp-session] Westerlund, M., Perkins, C., and J. Lennox, "Sending Multiple Types of Media in a single RTP Session", draft-ietf-avtcore-multi-media-rtp- session-07, March 2015. [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. 8.2 Informative References Authors' Addresses Rachel Huang Huawei 101 Software Avenue, Yuhua District Nanjing 210012 China Email: rachel.huang@huawei.com Expires April 21, 2016 [Page 13]