Audio/Video Transport WG T. Kristensen Internet-Draft M. Walters Updates: 3984bis Cisco (if approved) October 25, 2010 Intended status: Standards Track Expires: April 28, 2011 Additional H.241 Parameter in the RTP Payload Format for H.264 Video draft-kristensen-avt-rtp-h241param-01 Abstract As systems increasingly are able to run video at 60 frames per second, there is a need to signal desired frame rate to optimise resolution choices and to avoid unnecessary resource or energy usage. This document defines a new optional parameter addressing recent extensions currently supported in H.323 systems: The signalling of the maximum frames per second that can be efficiently received or that can be sent. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on April 28, 2011. Copyright Notice Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect Kristensen & Walters Expires April 28, 2011 [Page 1] Internet-Draft New H.264 RTP parameters October 2010 to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction and Overview . . . . . . . . . . . . . . . . . . 3 1.1. New Parameter max-fps . . . . . . . . . . . . . . . . . . 3 1.2. Existing Schemes . . . . . . . . . . . . . . . . . . . . . 4 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Payload Format Parameters . . . . . . . . . . . . . . . . . . 4 3.1. Media Type Registration . . . . . . . . . . . . . . . . . 5 4. SDP Parameters . . . . . . . . . . . . . . . . . . . . . . . . 5 4.1. Mapping of the optional parameter to SDP . . . . . . . . . 5 4.2. Usage with SDP offer/answer . . . . . . . . . . . . . . . 5 4.3. Usage in Declarative Session Descriptions . . . . . . . . 6 4.4. Examples . . . . . . . . . . . . . . . . . . . . . . . . . 6 5. Co-existence . . . . . . . . . . . . . . . . . . . . . . . . . 6 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6 7. Security Considerations . . . . . . . . . . . . . . . . . . . 6 8. Open issues . . . . . . . . . . . . . . . . . . . . . . . . . 7 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 7 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 7 10.1. Normative References . . . . . . . . . . . . . . . . . . . 7 10.2. Informative references . . . . . . . . . . . . . . . . . . 8 Appendix A. Change History . . . . . . . . . . . . . . . . . . . 8 A.1. -00 to -01 . . . . . . . . . . . . . . . . . . . . . . . . 8 Appendix B. Figures . . . . . . . . . . . . . . . . . . . . . . . 8 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12 Kristensen & Walters Expires April 28, 2011 [Page 2] Internet-Draft New H.264 RTP parameters October 2010 1. Introduction and Overview The extended video procedures and control signals for H.300 series terminals in ITU-T H.241 [ITU.H241.2006], associated with the ITU-T H.264 [ITU.H264.2007] codec, continue to evolve. This document describes a new parameter specified in H.241 - Amendment 2 [ITU.H241Amd2.2009] used to indicate the maximum picture rate that can be efficiently received or the maximum picture rate that can be sent. This proposal defines a media type parameter for maximum picture rate signalling and allows use in SDP-based systems, as this is not covered in RFC3984bis [I-D.ietf-avt-rtp-rfc3984bis]. Editorial note: This draft may be used to specify further, new H.241 parameters if applicable and when new parameters are defined. 1.1. New Parameter max-fps The use of MaxMBPS amd MaxFS, either signalled implicitly via supported level or as the optional parameters max-mbps and max-fs [I-D.ietf-avt-rtp-rfc3984bis], underspecify the video stream and can lead to wasted bandwidth and processing resources, as well as unnecessary energy consumption and, where applicable, shorter battery duration. For example, a decoder signalling support for w1080p30 (243000 MBPS and 8100 MB) implicitly allows the sending of w720p60 (216000 MBPS and 3600 MB). Similarly, if a decoder signalled support for w720p30 (108000 MBPS and 3600 MB) the encoder could choose to send w480p60, w448p60 or even wCIF at 120 frames per second. Graphs visualizing these examples are included in Appendix B, where the existing parameters are used, with the addition of max-fps to constrain the allowed parameter range as preferred. If the encoder chooses to send at a higher frame rate than preferred by the receiver side, the decoder will normally discard the additional frames after decoding them. The transmission of the extra frames and the processing of frames to just discard them are wasteful and the bandwidth and processing could be used more effectively. The new parameter MaxFPS is introduced in H.241 - Amendment 2 to remove this issue, by constraining the allowed parameter range for systems supporting this extension. This draft proposes adding the optional parameter max-fps for use in SDP-based signalling. The specification of max-fps in this draft will also ease the gatewaying between H.323 and SIP systems by making the translation straight-forward for this parameter as well. Kristensen & Walters Expires April 28, 2011 [Page 3] Internet-Draft New H.264 RTP parameters October 2010 It is worth emphasizing that systems that does not support and understand the max-fps parameter are free to produce streams as before, within the bounds specified by the existing parameters. 1.2. Existing Schemes A general attribute "a=framerate" is specified in SDP [RFC4566] as a recommendation for the encoding of video. It is a media-level attribute and is defined only for video. The "a=framerate" attribute applies to all video payloads in the media section it belongs. This may work well in scenarios where the maximum frame rate is identical for all the video codecs present in the video media section. This is not always the case. In addition to the general "a=framerate" attribute, some video standards have some sort of frame rate signalling in their SDP description. For instance, in the H.263 and H.261 RTP payload format specifications [RFC4629][RFC4587] the maximum frame rate capability is signalled as an integer value Minimum Picture Interval (MPI). This MPI value assumes an upper bound of 30 frames per second. Another example is the VC-1 video codec [RFC4425], where an optional parameter framerate is specified as an "a=fmtp" parameter. As the examples above show, different media subtypes enables different means to signal the maximum frame rate capability. Some are explicit, others implicit. Some even have an implicit assumption of the maximum frame rate it is possible to express. Also, using only the SDP "a=framerate" attribute for all payloads associated with a media line is not feasible in general. Refer to Section 5 for recommended usage of "a=framerate" together with the max-fps parameter specified in this draft. 2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. In video formats based on NTSC, 30 frames per second and 60 frames per second are used repeatedly. However, the actual values are 30/ 1.001=29.97 and 60/1.001=59.94. Values derived from these fractional values MUST be used in signalling descriptions. 3. Payload Format Parameters The optional H264 media subtype parameter max-fps specified below Kristensen & Walters Expires April 28, 2011 [Page 4] Internet-Draft New H.264 RTP parameters October 2010 comes in addition to the list of optional H264 media subtype parameters defined in Section 8.1 of RFC3984bis [I-D.ietf-avt-rtp-rfc3984bis]. 3.1. Media Type Registration New OPTIONAL parameter: max-fps: This parameter SHOULD be used to indicate the maximum picture rate that can be efficiently received or the maximum picture rate that can be sent. The value of max-fps is an integer in units of hundredths of frames per second. Note that this is not necessarily the frame rate at which the maximum frame size can be sent. The encoder SHOULD use a frame rate equal to or less than this value. However, encoders MAY choose to send a higher frame rate. If the parameter is absent then the encoder is free to choose any frame rate. 4. SDP Parameters 4.1. Mapping of the optional parameter to SDP The optional parameter max-fps, when present, MUST be included in the "a=fmtp" line of SDP. The parameters in the "a=fmtp" line are expressed as a media type string, in the form of a semicolon separated list of parameter=value pairs. Editorial note: To be decided whether the stream property (sender capability) usage of max-fps is useful for SDP usage, i.e. with sendonly and for declarative usage. 4.2. Usage with SDP offer/answer When H.264 is offered over RTP using SDP in an offer/answer exchange [RFC3264] for negotiation of unicast streams, the following limitations and rules applies: The interpretation of the optional parameter max-fps depends on the direction attribute. When the direction attribute is sendonly, then it indicates the maximum picture rate that shall be sent. When the direction attribute is sendrecv or recvonly, the value of this parameter indicates the maximum picture frame rate that the receiver can efficiently handle. Any encoder that understands the parameter semantics shall constrain the frame rate to rates up to that specified. A receiver should have the ability to process video from a sender that does not understand this parameter. Kristensen & Walters Expires April 28, 2011 [Page 5] Internet-Draft New H.264 RTP parameters October 2010 The profile-level-id parameter MUST be present in the same receiver capability description that contains this parameter. 4.3. Usage in Declarative Session Descriptions When H.264 over RTP is offered with SDP in a declarative style, the max-fps parameter is used to declare the actual stream property and the value indicates the maximum picture rate that shall be sent. 4.4. Examples Two simple examples of SDP descriptions with the max-fps parameter. A NTSC-based system supporting maximum 30 frames per second could offer: m=video 49170 RTP/AVP 98 a=rtpmap:98 H264/90000 a=fmtp:98 profile-level-id=42A01E; max-fps=2997 A description for a system supporting a true 60 frames per second: m=video 49170 RTP/AVP 99 a=rtpmap:99 H264/90000 a=fmtp:99 profile-level-id=42A01E; max-fps=6000 5. Co-existence The "a=framerate" limitation allows a global maximum frame rate to be specified for the video media, this limit applies to all codecs in the video media section. In cases where both "a=framerate" and codec specific limits are specified, the maximum frame rate that can be used for a codec is the minimum of the values specified globally and specifically for a codec. 6. IANA Considerations This draft updates RFC3984bis by adding a new optional parameter to the H264 media subtype. The parameter is specified in Section Section 3 of this document. 7. Security Considerations No separate considerations introduced in this document. Refer to section 9 of RFC3984bis [I-D.ietf-avt-rtp-rfc3984bis]. Kristensen & Walters Expires April 28, 2011 [Page 6] Internet-Draft New H.264 RTP parameters October 2010 8. Open issues The open issues are marked as editorial notes in the draft. An overview of the issues: Placeholder: This draft may be used for new H.241 parameters if applicable. Depends on the timeline of this draft and proposals for H.241 extensions. See Section 1. Stream property: Specification for max-fps as stream property might be removed, if not feasible or actually required. Pending. See Section 4.1. 9. Acknowledgements The authors would like to acknowledge Mo Zanaty (mzanaty@cisco.com) and Roni Even (even.roni@huawei.com) for helpful input and comments. 10. References 10.1. Normative References [I-D.ietf-avt-rtp-rfc3984bis] Wang, Y., Even, R., Kristensen, T., and R. Jesup, "RTP Payload Format for H.264 Video", draft-ietf-avt-rtp-rfc3984bis-12 (work in progress), October 2010. [ITU.H241.2006] International Telecommunications Union, "Extended video procedures and control signals for H.300-series terminals", ITU-T Recommendation H.241, May 2006. [ITU.H241Amd2.2009] International Telecommunications Union, "Extended video procedures and control signals for H.300-series terminals", ITU-T Recommendation H.241 - Amendment 2, December 2009. [ITU.H264.2007] International Telecommunications Union, "Advanced video coding for generic audiovisual services", ITU- T Recommendation H.264, November 2007. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. Kristensen & Walters Expires April 28, 2011 [Page 7] Internet-Draft New H.264 RTP parameters October 2010 [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June 2002. 10.2. Informative references [RFC4425] Klemets, A., "RTP Payload Format for Video Codec 1 (VC-1)", RFC 4425, February 2006. [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. [RFC4587] Even, R., "RTP Payload Format for H.261 Video Streams", RFC 4587, August 2006. [RFC4629] Ott, H., Bormann, C., Sullivan, G., Wenger, S., and R. Even, "RTP Payload Format for ITU-T Rec. H.263 Video", RFC 4629, January 2007. Appendix A. Change History A.1. -00 to -01 1. Tightened text describing "a=maxprate" and H.26x frame rate signalling parameters, to get the real message clearer through. 2. Emphasized the optimization and energy consumption/battery duration as reasons for specifying max-pfs. 3. Added graphs as an aid visualizing the effect of max-fps, together with macroblock and frame size signalling 4. Clarified the 60/30 and 29.97/59.94 value usage in the document. 5. A better text regarding co-existence with "a=framerate" added. 6. Misc. editorial fixes and cleanup. Appendix B. Figures Graphical representation of the usage of the existing parameters for signalling supported number of macroblocks per second (MaxMBPS) and frame size (MaxFS), with addition of the proposed max-fps parameter in an example scenario. Figure 1 assumes that an encoder can encode at one of three resolutions wCIF, w720p and w1080p, and three different frame rates 30, 60 and 120 frames per second (fps), but not all combinations of these are possible. Kristensen & Walters Expires April 28, 2011 [Page 8] Internet-Draft New H.264 RTP parameters October 2010 | | 20 fps + | | | | 30 fps + * * * | | | | 60 fps + * * | | 120 fps + * | | +----------+----------+----------+----------+----------+-- 0 2000 4000 6000 8000 10000 MaxFS Figure 1: Encoder capabilities The decoder can then signal its capabilities. For example, if it can decode 108000 macroblocks per second, then the encoder can choose any combination of frame size and frame rate within the dotted area as shown in Figure 2. Kristensen & Walters Expires April 28, 2011 [Page 9] Internet-Draft New H.264 RTP parameters October 2010 |. . . . . . . . . . . . . . . . . . |. . . . . . . . . . . . . . . . . 20 fps +. . . . . . . . . . . . . . . . |. . . . . . . . . . . . . . . |. . . . . . . . . . . . . . |. . . . . . . . . . . . . |. . . . . . . . . . . . 30 fps +. * . . . . . . .*. . * |. . . . . . . . . . |. . . . . . . . . |. . . . . . . . |. . . . . . . 60 fps +. * . . . . * |. . . . . |. . . . 120 fps +. * . |. . |. +----------+----------+----------+----------+----------+-- 0 2000 4000 6000 8000 10000 MaxFS Figure 2: MaxMBPS indicated If the decoder signals a maximum frame size (either explicitly as max-fs or via the supported level), then this further constrains the choice of the decoder as shown in Figure 3. Kristensen & Walters Expires April 28, 2011 [Page 10] Internet-Draft New H.264 RTP parameters October 2010 |. . . . . . . . . |. . . . . . . . .| 20 fps +. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . .| 30 fps +. * . . . . . . .* * |. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . | |. . . . . . . | 60 fps +. * . . . . * |. . . . . | |. . . . | 120 fps +. * . | |. . | |. | +---------+----------+----------+----------+----------+-- 0 2000 4000 6000 8000 10000 MaxFS Figure 3: MaxMBPS and MaxFS indicated The parameter max-fps is added as a horizontal limit in Figure 4 restricting the allowed parameter space after the encoder's preferences. Kristensen & Walters Expires April 28, 2011 [Page 11] Internet-Draft New H.264 RTP parameters October 2010 |. . . . . . . . . |. . . . . . . . .| 20 fps +. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . .| 30 fps +. * . . . . . . .* * |. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . | |. . . . . . . | 60 fps +--*--------------*-------------------------------------- | | | | 120 fps + * | | | | | +---------+----------+----------+----------+----------+-- 0 2000 4000 6000 8000 10000 MaxFS Figure 4: Adding max-fps to further constrain Authors' Addresses Tom Kristensen Cisco Philip Pedersens vei 22 N-1366 Lysaker Norway Phone: +47 67125125 Email: tomkrist@cisco.com, tomkri@ifi.uio.no Malcolm Walters Cisco 14 Waterside Drive Langley, Slough UK Email: malcowal@cisco.com Kristensen & Walters Expires April 28, 2011 [Page 12]