Network Working Group P. Westin Internet-Draft H. Lundin Intended status: Experimental M. Glover Expires: September 2, 2011 J. Uberti F. Galligan Google March 1, 2011 Proposal for the IETF on "RTP Payload Format for VP8 Video" draft-westin-payload-vp8-01 Abstract This memo describes an RTP Payload format for the VP8 video codec. The payload format has wide applicability, as it supports applications from low bit-rate peer-to-peer usage, to high bit-rate Video conferences. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on September 2, 2011. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of Westin, et al. Expires September 2, 2011 [Page 1] Internet-Draft RTP Payload Format for VP8 March 2011 the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Westin, et al. Expires September 2, 2011 [Page 2] Internet-Draft RTP Payload Format for VP8 March 2011 1. Introduction An encoded VP8 frame can be divided into two or more partitions, as described in "VP8 Data Format and Decoding Guide" [1]. The first partition (prediction or mode) contains prediction mode parameters and motion vectors for all macroblocks. The remaining partitions all contain the DCT/WHT coefficients for the residuals. The first partition is decodable without the remaining residual partitions. The subsequent partitions may be useful even if some part of the frame is lost. The format specification is described in Section 4. Section 5 describes a method to acknowledge receipt of reference frames using RTCP techniques is described. Both these examples serve as motivation for two of the fields included in the payload format: the "1st partition size" and "PictureID" fields. Westin, et al. Expires September 2, 2011 [Page 3] Internet-Draft RTP Payload Format for VP8 March 2011 2. Conventions, Definitions and Acronyms The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [2]. Westin, et al. Expires September 2, 2011 [Page 4] Internet-Draft RTP Payload Format for VP8 March 2011 3. Media Format Background VP8 is based on decomposition of frames into square subblocks of pixels, prediction of such subblocks using previously constructed blocks, and adjustment of such predictions (as well as synthesis of unpredicted blocks) using a discrete cosine transform (hereafter abbreviated as DCT). In one special case, however, VP8 uses a "Walsh-Hadamard" (hereafter abbreviated as WHT) transform instead of a DCT. An encoded VP8 frame is divided into two or more partitions, as described in [1]. The first partition (prediction or mode) contains prediction mode parameters and motion vectors for all macroblocks. The remaining partitions all contain the DCT/WHT coefficients for the residuals. Westin, et al. Expires September 2, 2011 [Page 5] Internet-Draft RTP Payload Format for VP8 March 2011 4. Payload Format The general RTP payload format for VP8 is depicted below. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC |M| PT | sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | synchronization source (SSRC) identifier | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | contributing source (CSRC) identifiers | | .... | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | VP8 payload descriptor (integer #bytes) | : : | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | : VP8 payload header (3 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | VP8 pyld hdr : | +-+-+-+-+-+-+-+-+ | : Bytes 4..N of VP8 payload : | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | : OPTIONAL RTP padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The VP8 payload descriptor and VP8 payload header will be described in the sequel. OPTIONAL RTP padding MUST NOT be included unless the P bit is set. Figure 1 Marker bit: The marker bit indicates the last packet part of an frame. This enables an decoder to finish decoding the picture, where it otherwise may need to wait for the next packet to explicitly know that. Timestamp: The RTP timestamp indicate the time when the frame was sampled at a clock rate of 90KHz. Sequence number: The sequence number are monotonically increasing and set as packets are sent. Westin, et al. Expires September 2, 2011 [Page 6] Internet-Draft RTP Payload Format for VP8 March 2011 The remaining RTP header fields are used as specified in RFC 3550 [4]. 4.1. VP8 Payload Descriptor The first bytes after the RTP header are the VP8 payload descriptor, with the following structure. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RSV |I|N|FI |B| PictureID (integer #bytes) | +-+-+-+-+-+-+-+-+ | : : | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | : (VP8 data or VP8 payload header; byte aligned)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 2 RSV: Bits reserved for future use. MUST be set to zero and MUST be ignored by the receiver. I: PictureID present. When set to one, a PictureID is provided after the first byte of the payload descriptor. When set to zero, the PictureID is omitted, and the one-byte payload descriptor is immediately followed by the VP8 payload. N: Non-reference frame. When set to one, the frame can be discarded without affecting any other future or past frames. FI: Fragmentation information field. This field contains information about the fragmentation of VP8 payloads carried in the RTP packet. The four different values are listed below. * 00 The RTP packet contains no fragmented VP8 partitions. The payload is one or several complete partitions. * 01 The RTP packet contains the first part of a fragmented partition. The fragment must be placed in its own RTP packet. * 10 The RTP packet contains a fragment that is neither the first nor the last part of a fragmented partition. The fragment must be placed in its own RTP packet. * 11 The RTP packet contains the last part of a fragmented partition. The fragment must be placed in its own RTP packet. Westin, et al. Expires September 2, 2011 [Page 7] Internet-Draft RTP Payload Format for VP8 March 2011 B: Beginning VP8 frame. When set to 1 this signals that a new VP8 frame starts in this RTP packet. PictureID: Multiple of 8 bits. This is a running index of the frames. The field is present only if the I bit is equal to one. The most significant bit of each byte is an extension flag. The 7 following bits carry (parts of) the PictureID. If the extension flag is one, the PictureID continues in the next byte. If the extension flag is zero, the 7 remaining bits are the last (and least significant) bits in the PictureID. The sender may choose any number of bytes, smaller or equal to 9 bytes as the maximum PictureID. The PictureID SHALL start on a random number, and MUST wrap after reaching the maximum ID. Leading zero bytes MUST be supressed. 4.2. VP8 Payload Header The first three bytes of an encoded VP8 frame are refered to as an "uncompressed data chunk" in [1], and co-serve as payload header in this RTP format. Note that the header is present only in packets which have the B bit equal to one in the payload descriptor. Subsequent packets for the same frame do not carry the payload header. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |P| VER |H| 1st partition size | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | : Bytes 4..N of VP8 payload : | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | : OPTIONAL RTP padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 3 P: Inverse key frame flag. When set to 0 the current frame is a key frame. When set to 1 the current frame is an interframe. Defined in [1] VER: A version number as defined in [1]. H: Show frame bit as defined in [1]. Westin, et al. Expires September 2, 2011 [Page 8] Internet-Draft RTP Payload Format for VP8 March 2011 1st partition size: 19 bits. A field containing the size of the first data partition in bytes, as defined in [1]. 4.3. Aggregated and Fragmented Payloads An encoded VP8 frame can be divided into two or more partitions, as described in Section 1. The fragmentation information described in Section 4.1 MUST be used to signal if any fragmentation is applied. Aggregation of encoded partitions is done without explicit signaling. Partitions MUST be aggregated in decoding order. Two fragments from different partitions MUST not be agregated into the same packet. An aggregation MUST have exactly one payload descriptor. Aggregated partitions MUST represent parts of one and the same video frame. Consequently, an aggregated packet will have one or no payload header, depending on whether the aggregate contains the first partition of a frame or not, respectively. Note that the length of the first partition can always be obtained from the first partition size parameter in the VP8 payload header. Fragments of encoded partitions MUST NOT be aggregated. 4.4. Examples of VP8 RTP Stream A few examples of how the VP8 RTP payload can be used are included below. Westin, et al. Expires September 2, 2011 [Page 9] Internet-Draft RTP Payload Format for VP8 March 2011 4.4.1. Key frame in a single RTP packet Marker bit = 1. I = 1. B = 1. PictureID = 17 = 0001001 binary. P = 0. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header M=1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 1 0 0 0 1:0 0 0 0 1 0 0 1|0: VER :1: 1st partition | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | size = L | | +-+-+-+-+-+-+-+-+ | | | : Bytes 4..L of first VP8 partition : | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | : Remaining VP8 partitions : | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | : OPTIONAL RTP padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Westin, et al. Expires September 2, 2011 [Page 10] Internet-Draft RTP Payload Format for VP8 March 2011 4.4.2. VP8 interframe in a single RTP packet; no PictureID Marker bit = 1. I = 0. B = 1. P = 1. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header M=1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 1|1: VER :1: 1st partition size = L | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | : Bytes 4..L of first VP8 partition : | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | : Remaining VP8 partitions : | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | : OPTIONAL RTP padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 4.4.3. VP8 partitions in separate RTP packets First RTP packet; marker bit = 0. I = 1. B = 1. PictureID = 17. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header M=0 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 1 0 0 0 1:0 0 0 0 1 0 0 1|1: VER :1: 1st partition | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | size = L | | +-+-+-+-+-+-+-+-+ | | | : Bytes 4..L of first VP8 partition : | | | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Westin, et al. Expires September 2, 2011 [Page 11] Internet-Draft RTP Payload Format for VP8 March 2011 Second RTP packet; marker bit = 1. B = 0. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header M=1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 1 0 0 0 0:0 0 0 0 1 0 0 1| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | : Remaining VP8 partitions : | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | : OPTIONAL RTP padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 4.4.4. VP8 frame fragmented across RTP packets First RTP packet; marker bit = 0. I = 1. FI = 00. B = 1. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header M=0 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 1 0 0 0 1:0 0 0 0 1 0 0 1|1: VER :1: 1st partition | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | size = L | | +-+-+-+-+-+-+-+-+ | | | : Bytes 4..L of first VP8 partition : | | | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Westin, et al. Expires September 2, 2011 [Page 12] Internet-Draft RTP Payload Format for VP8 March 2011 Second RTP packet; marker bit = 0. FI = 01. B = 0. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header M=0 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 1 0 0 1 0:0 0 0 0 1 0 0 1| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | : First fragment of second VP8 partition : | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Third RTP packet; marker bit = 0. FI = 10. B = 0. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header M=0 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 1 0 1 0 0:0 0 0 0 1 0 0 1| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | : Middle fragment of second VP8 partition : | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Last RTP packet; marker bit = 1. FI = 11. B = 0. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header M=1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 1 0 1 1 0:0 0 0 0 1 0 0 1| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | : Last fragment of second VP8 partition : | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | : OPTIONAL RTP padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Westin, et al. Expires September 2, 2011 [Page 13] Internet-Draft RTP Payload Format for VP8 March 2011 4.4.5. VP8 frame with long PictureID PictureID = 4711 = 01001001100111 binary (first 7 bits: 0100100, last 7 bits: 1100111). 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header M=1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 1 0 0 0 1:1 0 1 0 0 1 0 0 0 1 1 0 0 1 1 1|1: VER :1: 1st | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | partition size = L | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | : Bytes 4..N of first VP8 frame : | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Westin, et al. Expires September 2, 2011 [Page 14] Internet-Draft RTP Payload Format for VP8 March 2011 5. Using VP8 with RPSI and SLI Feedback The VP8 payload descriptor defined in Section 4.1 above contains an optional PictureID parameter. This parameter is included mainly to enable use of reference picture selection index (RPSI) and slice loss indication (SLI), both defined in RFC 4585 [3]. 5.1. RPSI The reference picture selection index is a payload-specific feedback message defined within the RTCP-based feedback format. The RPSI message is generated by a receiver and can be used in two ways. Either it can signal a preferred reference picture when a loss has been detected by the decoder -- preferably then a reference that the decoder knows is perfect -- or, it can be used as positive feedback information to acknowledge correct decoding of certain reference pictures. The positive feedback method is useful for VP8 used as unicast. The use of RPSI for VP8 is preferably combined with a special update pattern of the codec's two special reference frames -- the golden frame and the altref frame -- in which they are updated in an alternating leapfrog fashion. When a receiver has received and correctly decoded a golden or altref frame, and that frame had a PictureID in the payload descriptor, the receiver can acknowledge this simply by sending an RPSI message back to the sender. The message body (i.e., the "native RPSI bit string" in RFC 4585 [3]) is simply the PictureID of the received frame. 5.2. SLI The slice loss indication is another payload-specific feedback message defined within the RTCP-based feedback format. The SLI message is generated by the receiver when a loss or corruption is detected in a frame. The format of the SLI message is as follows [3]: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | First | Number | PictureID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 4 Here, First is the macroblock address (in scan order) of the first lost block and Number is the number of lost blocks. PictureID is the six least significant bits of the codec-specific picture identifier in which the loss or corruption has occurred. For VP8, this codec- specific identifier is naturally the PictureID of the current frame, Westin, et al. Expires September 2, 2011 [Page 15] Internet-Draft RTP Payload Format for VP8 March 2011 as read from the payload descriptor. If the payload descriptor of the current frame does not have a PictureID, the receiver MAY send the last received PictureID+1 in the SLI message. The receiver MAY set the First parameter to 0, and the Number parameter to the total number of macroblocks per frame, even though only parts of the frame is corrupted. When the sender receives an SLI message, it can make use of the knowledge from the latest received RPSI message. Knowing that the last golden or altref frame was successfully received, it can encode the next frame with reference to that established reference. 5.3. Example The use of RSPI and SLI is best illustrated in an example. In this example, the encoder may not update the altref frame until the last sent golden frame has been acknowledged with an RPSI message. If an update is not received within some time, a new golden frame update is sent instead. Once the new golden frame is established and acknowledge, the same rule applies when updating the altref frame. Event Sender Receiver Established reference +----+--------------------+--------------------------+------------+ 1000 Send golden frame PictureID = 0 Receive and decode golden frame 1001 Send RPSI(0) 1002 Receive RPSI(0) golden ... (sending regular frames) 1100 Send altref frame PictureID = 100 Altref corrupted or lost golden 1101 Send SLI(100) golden 1102 Receive SLI(100) 1103 Send frame with reference to golden Receive and decode frame (decoder state restored) golden ... (sending regular frames) 1200 Send altref frame Westin, et al. Expires September 2, 2011 [Page 16] Internet-Draft RTP Payload Format for VP8 March 2011 PictureID = 200 Receive and decode altref frame golden 1201 Send RPSI(200) 1202 Receive RPSI(200) altref ... (sending regular frames) 1300 Send golden frame PictureID = 300 Receive and decode golden frame altref 1301 Send RPSI(300) altref 1302 RPSI lost 1400 Send golden frame PictureID = 400 Receive and decode golden frame altref 1401 Send RPSI(400) 1402 Receive RPSI(400) golden +----+--------------------+--------------------------+------------+ Note that the scheme is robust to loss of the feedback messages. If the RPSI is lost, the sender will try to update the golden (or altref) again after a while, without releasing the established reference. Also, if an SLI is lost, the receiver can keep sending SLI messages at any interval, as long as the picture is corrupted. Westin, et al. Expires September 2, 2011 [Page 17] Internet-Draft RTP Payload Format for VP8 March 2011 6. Payload Format Parameters This section specifies the parameters that MAY be used to select optional features of the payload format and certain features of the bitstream. 6.1. Restrictions on usage This media type depends on RTP framing, and hence is only defined for transfer via RTP [RFC3550] [4].Transport within other framing protocols is not defined at this time. 6.2. MIME Registration The receiver MUST ignore any unspecified parameter. Media Type name: video Media subtype name: VP8 Required parameters: none Security considerations: - See Section 7 of RFC xxxx. 6.3. SDP Parameters The receiver MUST ignore any parameter unspecified in this memo. 6.3.1. Mapping of MIME Parameters to SDP The MIME media type video/VP8 string is mapped to fields in the Session Description Protocol (SDP) [6] as follows: o The media name in the "m=" line of SDP MUST be video. o The encoding name in the "a=rtpmap" line of SDP MUST be VP8 (the MIME subtype). o The clock rate in the "a=rtpmap" line MUST be 90000. o The OPTIONAL parameter "version", if included, MUST be in the a=fmtp SDP field. This parameter matches the VP8 bitstream version. Westin, et al. Expires September 2, 2011 [Page 18] Internet-Draft RTP Payload Format for VP8 March 2011 6.4. Example An example of media representation in SDP is as follows: m=video 49170 RTP/AVP 98 a=rtpmap:98 VP8/90000 a=fmtp:98 version=0 Westin, et al. Expires September 2, 2011 [Page 19] Internet-Draft RTP Payload Format for VP8 March 2011 7. Security Considerations RTP packets using the payload format defined in this specification are subject to the security considerations discussed in the RTP specification [4], and in any applicable RTP profile. The main security considerations for the RTP packet carrying the RTP payload format defined within this memo are confidentiality, integrity and source authenticity. Confidentiality is achieved by encryption of the RTP payload. Integrity of the RTP packets through suitable cryptographic integrity protection mechanism. Cryptographic system may also allow the authentication of the source of the payload. A suitable security mechanism for this RTP payload format should provide confidentiality, integrity protection and at least source authentication capable of determining if an RTP packet is from a member of the RTP session or not. Note that the appropriate mechanism to provide security to RTP and payloads following this memo may vary. It is dependent on the application, the transport, and the signalling protocol employed. Therefore a single mechanism is not sufficient, although if suitable the usage of SRTP [5] is recommended. This RTP payload format and its media decoder do not exhibit any significant non-uniformity in the receiver-side computational complexity for packet processing, and thus are unlikely to pose a denial-of-service threat due to the receipt of pathological data. Nor does the RTP payload format contain any active content. Westin, et al. Expires September 2, 2011 [Page 20] Internet-Draft RTP Payload Format for VP8 March 2011 8. IANA Considerations The IANA is requested to register the following values: - MIME registration as described in Section 6.2. Westin, et al. Expires September 2, 2011 [Page 21] Internet-Draft RTP Payload Format for VP8 March 2011 9. References [1] Google, Inc., "VP8 Data Format and Decoding Guide", July 2010, . [2] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, STD 1, December 1997. [3] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, "Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, STD 1, July 2006. [4] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", RFC 3550, STD 64, July 2003. [5] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. Norrman, "The Secure Real-time Transport Protocol (SRTP)", RFC 3711, STD 1, March 2004. [6] Handley, M. and V. Jacobson, "SDP: Session Description Protocol", RFC 2327, STD 1, April 1998. Westin, et al. Expires September 2, 2011 [Page 22] Internet-Draft RTP Payload Format for VP8 March 2011 Authors' Addresses Patrik Westin Google, Inc. Kungsbron 2 Stockholm, 11122 Sweden Email: patrik.westin@gmail.com Henrik Lundin Google, Inc. Michael Glover Google, Inc. Justin Uberti Google, Inc. Frank Galligan Google, Inc. Westin, et al. Expires September 2, 2011 [Page 23]