AVT J. Lennox Internet-Draft Vidyo Updates: 3984 (if approved) November 11, 2007 Intended status: Standards Track Expires: May 14, 2008 Source-Specific Media Format Parameters for H.264 and H.264 SVC draft-lennox-avt-h264-source-fmtp-00 Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on May 14, 2008. Copyright Notice Copyright (C) The IETF Trust (2007). Abstract When used in the Session Description Protocol Offer/Answer model, several of the media format parameters for the H.264 video format, and for its Scalabile Video Codec (SVC) extension, describe characterics of the stream an endpoint is prepared to send, not of streams it is prepared to receive. If an endpoint wishes to send multiple streams, these parameters may be incompatible. This document defines how such media format parameters may be given on a Lennox Expires May 14, 2008 [Page 1] Internet-Draft H.264 Source-Specific Format Parameters November 2007 per-source basis, using SDP source-specific fmtp attributes. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 4. Mapping of MIME parameters to SDP source attributes . . . . . . 4 4.1. Parameter Sets . . . . . . . . . . . . . . . . . . . . . . 4 4.2. Packetization Mode 2 Parameters . . . . . . . . . . . . . . 5 4.3. Capability Parameters . . . . . . . . . . . . . . . . . . . 5 4.4. H264 SVC Parameters . . . . . . . . . . . . . . . . . . . . 5 4.5. Other Parameters . . . . . . . . . . . . . . . . . . . . . 6 5. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 6. Backward Compatibility . . . . . . . . . . . . . . . . . . . . 6 7. Security Considerations . . . . . . . . . . . . . . . . . . . . 7 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 7 9. Normative References . . . . . . . . . . . . . . . . . . . . . 7 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 7 Intellectual Property and Copyright Statements . . . . . . . . . . 9 Lennox Expires May 14, 2008 [Page 2] Internet-Draft H.264 Source-Specific Format Parameters November 2007 1. Introduction Unlike many media packetization formats for the Real-Time Transport Protocol (RTP) [RFC3550], the the RTP packetization specifications for H.264 [RFC3984] and for H.264's Scalable Video Coding extension [I-D.ietf-avt-rtp-svc] define a number of MIME media type parameters that, when encoded in SDP [RFC4566], define characteristics of the media stream an endpoint is prepared to send, not of the streams it is prepared to receive. Most notably, streams' parameter sets (initial data required to correctly initialize a decoder) are encoded in SDP messages and sent out-of-band, to ensure that they are reliably received by a decoder before decoding begins. Because RTP is fundamentally a group communication protocol, however, an RTP session may contain many different media streams. In this case, different H.264 and H.264 SVC streams in an RTP session may need to be described by different and incompatible values for these MIME media type parameters. For example, an endpoint may be switching between video streams encoded by separate video encoders. In this case, it is not possible, using only the mechanisms of [RFC3984], to describe all the sources and to send their parameter sets out-of-band. An endpoint must instead fall back to sending parameter sets in-band, and retransmitting them with high enough frequency that there is a reasonably high likelihood of their being receceived successfully. To solve this difficulty, this document uses the framework introduced by [I-D.lennox-mmusic-sdp-source-attributes] to describe MIME media format parameters of individual H.264 sources in SDP. This enables all the benefits of out-of-band H.264 source description in the case when multiple H.264 sources will be sent in an RTP session. 2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119] and indicate requirement levels for compliant implementations. 3. Overview When used with the SDP [RFC4566] offer/answer model [RFC3264], several of the media format parameters of H.264 [RFC3984] and H.264 SVC [I-D.ietf-avt-rtp-svc] define characteristics of an RTP stream (media source) to be sent, not of a session receiver's capabilities. When multiple media sources are in use, it is sometimes useful to Lennox Expires May 14, 2008 [Page 3] Internet-Draft H.264 Source-Specific Format Parameters November 2007 describe source characteristics individually for each source. Per-source media format parameters are defined using the fmtp source parameter [I-D.lennox-mmusic-sdp-source-attributes]. This document updates the MIME type registration of video/H264 in [RFC3984] and of video/H264-SVC in [I-D.ietf-avt-rtp-svc] to specify the encoding of the MIME media format parameters into the fmtp source attribute. 4. Mapping of MIME parameters to SDP source attributes H.264 MIME media format parmaeters applicable to a specific source are are encoded into an "fmtp" source attribute for the H.264 payload type and the source being described. These parameters are expressed in the form of a semicolon-separated list of parameter=value pairs, the same syntax as the media-level fmtp value. For the purposes of discussion in this document, MIME media format parameters encoded into a source-specific fmtp attribute are called "source-specific parameters", while MIME media format parameters encoded into a media- level format attribute are called "media parameters". Source-specific parameters only describe characteristics of a specific source might send, while media parameters describe all sources of a stream. Source-specific parameters do not override media parameters, though they do in some cases further constrain them or provide additional information. 4.1. Parameter Sets The "sprop-parameter-sets" parameter encodes H.264 sequence parameter set (SPS) and picture parameter set (PPS) Network Adaptation Layer NAL) units. These parameter sets provide essential information necessary to decode an H.264 bitstream; encoding them in SDP ensures that they are delivered reliably. The "sprop-parameter-sets" parameter MAY be encoded as a source parameter. However, if the sprop-parameter-sets parameter is also present as a media parameter, the H.264 parameter sets described for each source MUST include all the H.264 parameter set described for the media. In an SDP answer, the "sprop-parameter-sets" for a source MUST follow the same constraints as for the media. I.e., the parameter sets for a source described in an answer MUST be a superset of the parameter sets for the media in the offer, and if an offer indicates "parameter-add=0" (false) for the media, the corresponding answer MUST NOT add additonal parameter sets for any source. Lennox Expires May 14, 2008 [Page 4] Internet-Draft H.264 Source-Specific Format Parameters November 2007 4.2. Packetization Mode 2 Parameters The "sprop-deint-buf-req", "sprop-interleaving-depth", "sprop-max- don-diff", and "sprop-init-buf-time" parameters describe characteristics of H.264 media streams using packetization mode 2 (interleaved mode). According to [RFC3984], the first two are mandatory for packetization mode 2 streams; the other two are optional. They specify the maximum size and time of the debuffering needed to deinterleave streams sent in interleaved mode. These parameters MAY be included as source parameters, overriding any corresponding values at the media level for the source. However, if they are, they MUST be less than or equal to the value specified for the parameter (if any) at the media level. All constraints for these values specified by [RFC3984] still apply. 4.3. Capability Parameters As defined in [RFC3984], the meaning of the capability parameters ("max-mbps", "max-fs", "max-cpb", "max-dpb", "max-br", "redundant- pic-cap", and "max-rcmd-nalu-size") at the media level depends on a media stream's "direction" attribute. When the "direction" attribute is "sendonly", then the parameters describe the limits of media sources that the sender is capable of producing. When the "direction" attribute is "sendrecv" or "recvonly", then the parameters describe the limitations of what the receiver accepts. When encoded as source parameters, these parameters always describe the limits of the source being described. In media streams whose "direction" is "sendonly", these parameters MUST be less than or equal to the values (if any) in the media parameters. In media streams whose "direction" is "sendrecv", source parameters for these values are unconstrained by stream's media parameters (which describe what the endpoint is willing to receive). However, in offer/answer mode, the values of these source parameters MUST be less than or equal to the values given in media parameters in the most recent (accepted) offer or answer for the stream. (Media streams whose "direction" is "recvonly" do not encode any sources.) 4.4. H264 SVC Parameters The H.264 SVC parameters "sprop-scalability-info" and "sprop-layer- ids" are defined in [I-D.ietf-avt-rtp-svc]. They describe the scalability structure of an H.264 Scalable Video Codec stream. These parameters MUST NOT be given both as source parameters and Lennox Expires May 14, 2008 [Page 5] Internet-Draft H.264 Source-Specific Format Parameters November 2007 media parameters. They MUST be specified at one level at most. 4.5. Other Parameters The parameters "profile-level-id", "packetization-mode", and "parameter-add" MUST NOT be used as a source parameters. 5. Examples This section gives examples of SDP descriptions of media sessions containing H.264 source parameters. For brevity, only the media sections of the descriptions are given. m=video 49170 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 packetization-mode=1 a=ssrc:12345 cname:cif-stream@example.com a=ssrc:12345 fmtp:96 sprop-parameter-sets=J0LgDZWWFglk,KM4Ecg== a=ssrc:67890 cname:vga-stream@example.com a=ssrc:67890 fmtp:96 sprop-parameter-sets=J0LgDZWWCgPZ,KM4Ecg== Figure 1: Example: declaration of two sources with different parameter sets The example in Figure 1 shows two H.264 streams with different, incompatbile parameter sets. (Specifically, the two streams encode different image sizes.) (TODO: add more examples.) 6. Backward Compatibility Unless a sender knows via some mechanism (not specified here or in [I-D.lennox-mmusic-sdp-source-attributes]) that a receiver definitely understands source parameters, it MUST send any parameter sets specified in sprop-parameter-sets source attributes in-band in the media stream as well. These parameter sets SHOULD be transmitted frequently enough that a receiver has a high probability of receiving them even in the presence of packet loss. For H.264 SVC streams, the scalability information Supplemental Encoder Information (SEI) message encoded in an sprop-scalability- info parameter set SHOULD be similarly transmitted in-band as well. Lennox Expires May 14, 2008 [Page 6] Internet-Draft H.264 Source-Specific Format Parameters November 2007 7. Security Considerations Source-specific encoding of media format parameters does not add any additional security considerations beyond those of [RFC3984] and [I-D.ietf-avt-rtp-svc]. 8. IANA Considerations This document updates the SDP encoding of the video/H264 MIME media type specified in [RFC3984], and of the video/H264-SVC MIME media type specified in [I-D.ietf-avt-rtp-svc]. 9. Normative References [I-D.ietf-avt-rtp-svc] Wenger, S., "RTP Payload Format for SVC Video", draft-ietf-avt-rtp-svc-02 (work in progress), July 2007. [I-D.lennox-mmusic-sdp-source-attributes] Lennox, J., "Source-Specific Media Attributes in the Session Description Protocol (SDP)", draft-lennox-mmusic-sdp-source-attributes-01 (work in progress), July 2007. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June 2002. [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003. [RFC3984] Wenger, S., Hannuksela, M., Stockhammer, T., Westerlund, M., and D. Singer, "RTP Payload Format for H.264 Video", RFC 3984, February 2005. [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. Lennox Expires May 14, 2008 [Page 7] Internet-Draft H.264 Source-Specific Format Parameters November 2007 Author's Address Jonathan Lennox Vidyo, Inc. 433 Hackensack Avenue Sixth Floor Hackensack, NJ 07601 US Email: jonathan@vidyo.com Lennox Expires May 14, 2008 [Page 8] Internet-Draft H.264 Source-Specific Format Parameters November 2007 Full Copyright Statement Copyright (C) The IETF Trust (2007). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Acknowledgment Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). Lennox Expires May 14, 2008 [Page 9]