Audio/Video Transport WG T. Kristensen Internet-Draft M. Walters Updates: 3984bis Cisco (if approved) September 14, 2010 Intended status: Standards Track Expires: March 18, 2011 Additional H.241 Parameter in the RTP Payload Format for H.264 Video draft-kristensen-avt-rtp-h241param-00 Abstract This document defines a new optional parameter addressing recent extensions currently supported in H.323 systems: The signalling of the maximum frames per second that can be efficiently received or that can be sent. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on March 18, 2011. Copyright Notice Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as Kristensen & Walters Expires March 18, 2011 [Page 1] Internet-Draft New H.264 RTP parameters September 2010 described in the Simplified BSD License. Table of Contents 1. Introduction and Overview . . . . . . . . . . . . . . . . . . . 3 1.1. Maximum Frame Rate Signalling . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3. Payload Format Parameters . . . . . . . . . . . . . . . . . . . 5 3.1. Media Type Registration . . . . . . . . . . . . . . . . . . 5 4. SDP Parameters . . . . . . . . . . . . . . . . . . . . . . . . 6 4.1. Mapping of the optional parameter to SDP . . . . . . . . . 6 4.2. Usage with SDP offer/answer . . . . . . . . . . . . . . . . 6 4.3. Usage in Declarative Session Descriptions . . . . . . . . . 6 4.4. Examples . . . . . . . . . . . . . . . . . . . . . . . . . 6 5. Co-existence . . . . . . . . . . . . . . . . . . . . . . . . . 7 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 7 7. Security Considerations . . . . . . . . . . . . . . . . . . . . 7 8. Open issues . . . . . . . . . . . . . . . . . . . . . . . . . . 7 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 8 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 8 10.1. Normative References . . . . . . . . . . . . . . . . . . . 8 10.2. Informative references . . . . . . . . . . . . . . . . . . 8 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 9 Kristensen & Walters Expires March 18, 2011 [Page 2] Internet-Draft New H.264 RTP parameters September 2010 1. Introduction and Overview The ITU-T H.264 [ITU.H264.2007] codec and the extended video procedures and control signals for H.300 series terminals in ITU-T H.241 [ITU.H241.2006], continue to evolve. The IETF H.264 RTP payload format and parameters need to be updated to include important new functionalities not covered in RFC3984bis [I-D.ietf-avt-rtp-rfc3984bis]. This document describes a new parameter specified in H.241 - Amendment 2 [ITU.H241Amd2.2009] used to indicate the maximum picture rate that can be efficiently received or the maximum picture rate that can be sent. This proposal defines media type parameters and allows use in SDP-based systems. Editorial note: This draft may be used to specify further, new H.241 parameters if applicable and when new parameters are defined. 1.1. Maximum Frame Rate Signalling As a motivation for adding "yet another parameter" to the H.264 media subtype registration, existing parameters for frame rate signalling are described, the shortcomings are analysed and the benefits of a new optional parameter for H.264 are highlighted. A general attribute is specified in SDP [RFC4566] as follows: a=framerate: This gives the maximum video frame rate in frames/sec. It is intended as a recommendation for the encoding of video data. Decimal representations of fractional values using the notation "." are allowed. It is a media-level attribute, defined only for video media, and it is not dependent on charset. The "a=framerate" attribute applies to all video payloads in the media section it belongs to. This may work well in scenarios where the maximum frame rate is identical for all the video codecs present in the video media section. This is not always the case. In addition to the general "a=framerate" attribute, some video standards have some sort of frame rate signalling in their SDP description. In the H.263 and H.261 RTP payload format specifications [RFC4629][RFC4587] the maximum frame rate capability for the H263-1998 and H261 media subtypes is signalled as an integer value Minimum Picture Interval (MPI). The desired frame rate capability is calculated based on an assumed upper frame rate limit of close to 30 frames per second. In the H263-1998 and H261 "a=fmtp" line, the optional parameters for the different resolutions have an associated MPI value. As an example, the optional parameter for CIF Kristensen & Walters Expires March 18, 2011 [Page 3] Internet-Draft New H.264 RTP parameters September 2010 is specified for H263-1998 [RFC4629] as follows: cif: Specifies the MPI (Minimum Picture Interval) for CIF resolution. Permissible values are integer values from 1 to 32, which correspond to a maximum frame rate of 30/(1.001 * the specified value) frames per second. For the H263-2000 media subtype [RFC4629] MPI values are not used, instead the maximum frame rate is signalled implicit in the optional parameter named level. Where level 50 and above implies a support for 60 frames per second, the lower levels 30 or 15 frames per second. This is, by the way, equivalent to the implicit frame rate signalling the existing optional parameters for H.264 [I-D.ietf-avt-rtp-rfc3984bis] offers today. Informational note: For the H263-1998 media subtype [RFC4629] it is possible to specify an arbitrary Custom Picture Clock Frequency (CPCF) and associated MPI values for the supported resolutions. However, the rationale for CPCF is to enable support for an actual clock rate deviating from the one specified in the "a=rtpmap" attribute, not to allow frame rates above 30 frames per second. For the VC-1 video codec [RFC4425] an optional parameter framerate is specified as an "a=fmtp" parameter. It is an integer greater than zero and specifying the number of frames per second multiplied with 1000 and rounded to nearest integer value. As the examples above show, the different media subtypes enables different means to signal the maximum frame rate capability. Some are explicit, others implicit. Some even have an implicit assumption of the maximum frame rate it is possible to express. Also, using the SDP "a=framerate" attribute for all payloads associated with a media line is not feasible in general. So, why are the existing optional parameters for H.264 not sufficient? Increasingly, systems are able to handle 60 frames per second and there is no feasible way to discover whether a system can handle such frame rates or not to ensure interoperability and to optimise resolution choices. The existing max-mbps and max-fs [I-D.ietf-avt-rtp-rfc3984bis] parameters underspecify the video stream and can lead to wasted bandwidth and processing resources. For example, a decoder signalling support for w1080p30 (243000 MBPS and 8100 MB) implicitly allows the sending of w720p60 (216000 MBPS and 3600 MB). Similarly, if a decoder signalled support for w720p30 (108000 MBPS and 3600 MB) the encoder could choose to send w480p60, w448p60 or even wCIF at 120 frames per second. Kristensen & Walters Expires March 18, 2011 [Page 4] Internet-Draft New H.264 RTP parameters September 2010 In many cases there is a maximum frame rate at which the decoder side can use data. This limit may stem from internal architecture or external factors, e.g. type of display attached. If the encoder chooses to send at a higher frame rate than this maximum limit, the decoder side will normally discard the additional frames after decoding them. The transmission of the extra frames and the processing of frames to just discard them is wasteful and the bandwidth and processing could be used more effectively. The new parameter MaxFPS is introduced in H.241 - Amendment 2 to remove this issue, by constraining the allowed parameter range. This draft proposes adding the optional parameter max-fps for use in SDP- based signalling. This adoption of the parameter will also ease the gatewaying between H.323 and SIP systems. 2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. In this document 30 frames per second and 60 frames per second are used repeatedly. However, the actual values when used in signalling description is typically 30/1.001=29.97 and 60/1.001=59.94. 3. Payload Format Parameters The optional H264 media subtype parameter max-fps specified below comes in addition to the list of optional H264 media subtype parameters defined in Section 8.1 of RFC3984bis [I-D.ietf-avt-rtp-rfc3984bis]. 3.1. Media Type Registration New OPTIONAL parameter: max-fps: This parameter MAY be used to indicate the maximum picture rate that can be efficiently received or the maximum picture rate that can be sent. The value of max-fps is an integer in units of hundredths of frames per second. Note that this is not necessarily the frame rate at which the maximum frame size can be sent. If the parameter is absent then the encoder is free to choose any frame rate. Kristensen & Walters Expires March 18, 2011 [Page 5] Internet-Draft New H.264 RTP parameters September 2010 4. SDP Parameters 4.1. Mapping of the optional parameter to SDP The optional parameter max-fps, when present, MUST be included in the "a=fmtp" line of SDP. The parameters in the "a=fmtp" line are expressed as a media type string, in the form of a semicolon separated list of parameter=value pairs. Editorial note: To be decided whether the stream property (sender capability) usage of max-fps is useful for SDP usage, i.e. with sendonly and for declarative usage. 4.2. Usage with SDP offer/answer When H.264 is offered over RTP using SDP in an offer/answer exchange [RFC3264] for negotiation of unicast streams, the following limitations and rules applies: The interpretation of the optional parameter max-fps depends on the direction attribute. When the direction attribute is sendonly, then it indicates the maximum picture rate that shall be sent. When the direction attribute is sendrecv or recvonly, the value of this parameter indicates the maximum picture frame rate that the receiver can efficiently handle. Any encoder that understands the parameter semantics shall constrain the frame rate to rates up to that specified. A receiver should have the ability to process video from a sender that does not understand this parameter. The profile-level-id parameter MUST be present in the same receiver capability description that contains this parameter. 4.3. Usage in Declarative Session Descriptions When H.264 over RTP is offered with SDP in a declarative style, the max-fps parameter is used to declare the actual stream property and the value indicates the maximum picture rate that shall be sent. 4.4. Examples Two simple examples of SDP descriptions with the max-fps parameter. A system supporting maximum 30 frames per second could offer: m=video 49170 RTP/AVP 98 a=rtpmap:98 H264/90000 a=fmtp:98 profile-level-id=42A01E; max-fps=2997 Kristensen & Walters Expires March 18, 2011 [Page 6] Internet-Draft New H.264 RTP parameters September 2010 A description for a system supporting 60 frames per second: m=video 49170 RTP/AVP 99 a=rtpmap:99 H264/90000 a=fmtp:99 profile-level-id=42A01E; max-fps=5994 5. Co-existence Editorial note: Whether the usage of the "a=framerate" parameter should be recommended in the case below or not is pending. Consider the text as basis for discussion. The SDP attribute "a=framerate" is used by some endpoints today, in cases where all the different video payloads in the media section has the same maximum frame rate limit. To support endpoints with this behaviour and possibly no support for max-fps, endpoints SHOULD add both max-fps for H.264 and "a=framerate" for the video media section if all payloads has the same maximum frame rate. 6. IANA Considerations This draft updates RFC3984bis by adding three new optional parameters to the H264 media subtype. The parameters are specified in Section Section 3 of this document. 7. Security Considerations No separate considerations introduced in this document. Refer to section 9 of RFC3984bis [I-D.ietf-avt-rtp-rfc3984bis]. 8. Open issues The open issues are marked as editorial notes in the draft. An overview of the issues: Placeholder: This draft may be used for new H.241 parameters if applicable. Depends on the timeline of this draft and proposals for H.241 extensions. See Section 1. Stream property: Specification for max-fps as stream property might be removed, if not feasible or actually required. Pending. See Section 4.1. Kristensen & Walters Expires March 18, 2011 [Page 7] Internet-Draft New H.264 RTP parameters September 2010 Co-existence: Pending whether recommending "a=framerate" usage in given scenarios belongs here. See Section 5. 9. Acknowledgements The authors would like to acknowledge Mo Zanaty (mzanaty@cisco.com) for helpful input and comments. 10. References 10.1. Normative References [I-D.ietf-avt-rtp-rfc3984bis] Wang, Y., Even, R., Kristensen, T., and R. Jesup, "RTP Payload Format for H.264 Video", draft-ietf-avt-rtp-rfc3984bis-11 (work in progress), June 2010. [ITU.H241.2006] International Telecommunications Union, "Extended video procedures and control signals for H.300-series terminals", ITU-T Recommendation H.241, May 2006. [ITU.H241Amd2.2009] International Telecommunications Union, "Extended video procedures and control signals for H.300-series terminals", ITU-T Recommendation H.241 - Amendment 2, December 2009. [ITU.H264.2007] International Telecommunications Union, "Advanced video coding for generic audiovisual services", ITU- T Recommendation H.264, November 2007. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June 2002. 10.2. Informative references [RFC4425] Klemets, A., "RTP Payload Format for Video Codec 1 (VC-1)", RFC 4425, February 2006. Kristensen & Walters Expires March 18, 2011 [Page 8] Internet-Draft New H.264 RTP parameters September 2010 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. [RFC4587] Even, R., "RTP Payload Format for H.261 Video Streams", RFC 4587, August 2006. [RFC4629] Ott, H., Bormann, C., Sullivan, G., Wenger, S., and R. Even, "RTP Payload Format for ITU-T Rec. H.263 Video", RFC 4629, January 2007. Authors' Addresses Tom Kristensen Cisco Philip Pedersens vei 22 N-1366 Lysaker Norway Email: tomkrist@cisco.com, tomkri@ifi.uio.no Malcolm Walters Cisco Langley Slough, England UK Email: malcowal@cisco.com Kristensen & Walters Expires March 18, 2011 [Page 9]