Network Working Group I. Johansson Internet-Draft Ericsson AB Intended status: Standards Track June 26, 2008 Expires: December 28, 2008 Negotiation of Generic Image Attributes in SDP draft-johansson-mmusic-image-attributes-01 Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on December 28, 2008. Abstract This draft proposes a new generic session setup attribute to make it possible to negotiate different image attributes such as image size. A possible use case is to make it possible for a e.g a low-end hand- held terminal to display video without the need to rescale the image, something that may consume large amounts of memory and processing power. The draft also helps to maintain an optimal bitrate for video as only the image size that is desired by the receiver is transmitted. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", Johansson Expires December 28, 2008 [Page 1] Internet-Draft Image Attributes in SDP June 2008 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Conventions, Definitions and Acronyms . . . . . . . . . . . . 3 3. Defintion of Attribute . . . . . . . . . . . . . . . . . . . . 3 3.1. SDPCapNeg . . . . . . . . . . . . . . . . . . . . . . . . 6 3.2. Asymmetric Sessions . . . . . . . . . . . . . . . . . . . 6 4. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4.1. Example 1 . . . . . . . . . . . . . . . . . . . . . . . . 6 4.2. Example 2 . . . . . . . . . . . . . . . . . . . . . . . . 7 4.3. Example 3 . . . . . . . . . . . . . . . . . . . . . . . . 7 4.4. Example 4 . . . . . . . . . . . . . . . . . . . . . . . . 7 4.5. Example 5 (SDPCapNeg) . . . . . . . . . . . . . . . . . . 8 4.6. Example 6 (Asymmetry) . . . . . . . . . . . . . . . . . . 8 5. Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 7. Security Considerations . . . . . . . . . . . . . . . . . . . 11 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 11 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11 9.1. Informative References . . . . . . . . . . . . . . . . . . 11 9.2. Normative References . . . . . . . . . . . . . . . . . . . 12 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 12 Intellectual Property and Copyright Statements . . . . . . . . . . 13 Johansson Expires December 28, 2008 [Page 2] Internet-Draft Image Attributes in SDP June 2008 1. Introduction This draft proposes a new attribute to make it possible to negotiate different image attributes such as image size. The term image size is defined here as it may differ from the physical screen size of e.g a hand-held terminal. For instance it may be beneficial to display a video image on a part of the physical screen and leave space on the screen for e.g menus and other info. There are a number of benefits with a possibility to negotiate the image size: o Less image distortion: Rescaling of images introduces additional distortion, something that can be avoided (at least on the receiver side) if the image size can be negotiated. o Reduced complexity: Image rescaling can be quite computation intensive. For low end devices this can be a problem. o Optimal quality for the given bitrate: The sender does not need to encode an entire CIF (352x288) image if only an image size of 288x256 is displayed on the receiver screen. This gives a saving in bitrate. o Memory requirement: The receiver device will know the size of the image and can then allocate memory accordingly. Several of the existing standards (see [RFC4587], [RFC4629] and [RFC3984]) has support for enabling the receiver to ask for different resolutions at different framerates. The purpose of this memo is to provide for a generic mechanism and is targeted mainly at the negotiation of the image size but to make it more general the attribute is named "imageattr". A problem statement and discussion that gives a motivation to this draft can be found in [S4-080144]. 2. Conventions, Definitions and Acronyms The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 3. Defintion of Attribute A new "imageattr" attribute is defined. The imageattr attribute contains a set of different image-resolution options that the offerer can provide. The receiver can then select the desired image Johansson Expires December 28, 2008 [Page 3] Internet-Draft Image Attributes in SDP June 2008 attribute (e.g image size) and has then the ability to avoid costly transformations (e.g rescaling) of the images. In this approach only the image resolution and optionally the framerate, sample aspect ratio and preference is covered but the framework makes it possible to extend with other image related attributes that makes sense. The syntax for the imageattr attribute is: Incomplete formulation: a=imageattr: 1*WSP * May be given in an offer to indicate that the answer MUST contain a complete formulation given below. Complete formulation: a=imageattr: 1*WSP list = set *(1*WSP set) ; defined in RFC 4566 is the payload type number sar is the a set of supported sample aspect ratios (optional) Each set for the complete formulation is defined by set=[x=range,y=range<,par=range><,fr=range><,q=value>] x is the horizontal image size range y is the vertical image size range fr is the framerate range for the given set (optional) par is a range of valid picture aspect ratios (optional) q is the preference for the given set in the range [0.0..1.0] (optional) range is (in matlab or octave syntax) given by range=value or range=[value:value] or range=[value:step:value] or range=[value,value...value] value is a positive integer or real value step is a positive integer or real value If step is left out in the syntax a stepsize of 1 is implied. Note the use of brackets [..] if more that one value specified. Selection of a desired image attribute entry follows the syntax: Johansson Expires December 28, 2008 [Page 4] Internet-Draft Image Attributes in SDP June 2008 a=imageattr: 1*WSP \ <[x=value,y=value]> or a=imageattr: 1*WSP \ <[x=value,y=value]> "\" denotes a line break that MUST NOT be present in the actual SDP The imageattr attribute is bound to a specific media by means of the payload type number. Several imageattr can be defined e.g for different video codec alternatives conditioned that the payload type number differs. The incomplete formulation (e.g "a=imageattr:97 *" ) in an offer indicates that the answer MUST contain a complete formulation or that the imageattr is removed from the answer. This is used for offer/ answer in a session in the likely case that the two participants have different display capabilities. The preference for each set is 0.5 by default, setting the optional q parameter to another value makes it possible to set different preferences for the sets. The sar parameter is a list of the possible SAR (Sample Aspect Ratio) conversions that is supported by the encoder, this applies to all the defined sets for the given imageattr line. This list is according to table E1 in [H.264]. A value of 255 indicates that the encoder supports any conversion. In this case the answer should specify the desired sar_w and sar_h values (if desired). The answer SDP indicates what sample aspect ratio (on the receiver display) that the sender shall compensate for. The par (width/height = x/y ratio) parameter indicates a range of picture aspect ratios. This is used to limit the number of x and y image size combinations, par is given as par=[par_min,par_max] Where par_min and par_max are the min and max allowed picture aspect ratios. The offerer MUST be able to support the image attributes that it offers. As an example, an offerer that cannot provide with compensation for sample aspect ratios MUST NOT include sar in the offer. The receiver MAY accept the imageattr but is not required to do so. The receiver MUST only include a valid entry taken from the offer in the SDP answer. Johansson Expires December 28, 2008 [Page 5] Internet-Draft Image Attributes in SDP June 2008 The answerer MUST only pick a valid entry from the multidimensional solution space spanned by the offer. If "fr" is given in the offer it MUST be present in the answer, the answer MAY limit the fr range but MUST NOT increase the range. 3.1. SDPCapNeg For use with the SDP capability negotiation framework (SDPCapNeg) an extra attribute "imcap" is defined with the syntax: a=imcap: 1*WSP is a number from 1 to 1^32-1 both included. list = set *(1*WSP set) ; defined in RFC 4566 The set and sar is defined as earlier in this text. The imcap attributes are referenced with the parameter "im" in the pcfg and acfg lines. Several imcap may be referenced in the potential configurations. The ability to support the imcap attribute MUST be specified with the "im-v0" identifier on the a=creq line or using the "+" option on the pcfg line and optionally also on the a=csup line. 3.2. Asymmetric Sessions As the display capabilities in the devices involved in a session may vary greatly it comes natural that this will likely lead to an asymmetry in terms of display size. As an example Alice (offerer) may prefer an image size of 320x240 while Bob prefers an image size of 800x600. This problem is made even worse if the two participants have different sample aspect ratios on the displays. To solve this issue it is necessary to use the sendonly and recvonly attributes as described in [RFC3264]. An example how this may look like is found in Section 4.6. 4. Examples A few examples to highlight the syntax 4.1. Example 1 a=imageattr:97 sar=[1:16] [x=800,y=640,q=0.6] [x=480,y=320] Johansson Expires December 28, 2008 [Page 6] Internet-Draft Image Attributes in SDP June 2008 Two image resolution alternatives are offered with 800x600 having the highest preference, furthermore the encoder supports conversion for the sar indices 1..16 given in table E1 in [H.264]. The answer may look like a=imageattr:97 sar=2 [x=800,y=640] Indicating that the receiver wish the encoder to compensate for a sample aspect ratio of 12:11 and desires an image size on its screen of 800x600. 4.2. Example 2 a=imageattr:97 sar=255 [fr=[5,15],x=800,y=640] \ [fr=[15,30],x=480,y=320] Two image resolution alternatives with different framerates are offered, furthermore the encoder supports conversion for any sample aspect ratio. The answer may look like a=imageattr:97 sar_w=13 sar_h=11 [x=480,y=320] Indicating a desired image size of 480x320 and requesting the encoder to compensate for a sample aspect ratio of 13:11. No specific framerate is requested. 4.3. Example 3 a=imageattr:97 \ [x=[480:16:800],y=[320:16:640],par=[1.2,1.3],q=0.6] \ [x=[176:8:208],y=[144:8:176],par=[1.2,1.3]] Two image resolution sets are offered with the first having a higher preference (q=0.6). The x-axis resolution can take the values 480 to 800 in 16 pixels steps and 176 to 208 in 8 pixels steps. The par parameter limits the set of possible x and y screen resolution combinations such that 800x640 (par=1.25) is a valid combination while 800x672 (par=1.19) or 800x608 (par=1.31) are invalid combinations. 4.4. Example 4 Given the offer a=imageattr:98 [x=[480:16:800],y=[320:16:640]] \ Johansson Expires December 28, 2008 [Page 7] Internet-Draft Image Attributes in SDP June 2008 [x=[176:8:208],y=[144:8:176]] An answer may look like a=imageattr:98 [x=192,y=160] Indicating that the answerer wish to render the video through a rectangular 192x160 block (image). 4.5. Example 5 (SDPCapNeg) An offer using SDPCapNeg may look like m=video .... a=creq:med-v0,im-v0 m=mcap:1 H_264 a=imcap:1 [x=[480:16:800],y=[320:16:640]] \ [x=[176:8:208],y=[144:8:176]] a=pcfg t=.. a=.. m=.. im=1 pt=1:97 The SDP answer may then look like m=video .... a=csup:med-v0,im-v0 a=imageattr:97 [x=192,y=160] a=acfg t=.. a=.. m=.. im=1 pt=1:97 Note the inclusion of the "clear text" imageattr in the answer that gives the desired image size. 4.6. Example 6 (Asymmetry) The offer/answer procedure between Alice (offer) and Bob with different display capabilities may look like below Offer: m=video 12340 RTP/AVP 97 a=rtpmap:97 H264/90000 a=imageattr:97 [x=[480:16:800],y=[320:16:640]] \ [x=[176:8:208],y=[144:8:176]] a=sendonly m=video 12342 RTP/AVP 98 a=rtpmap:98 H264/90000 a=imageattr:98 * a=recvonly The image attribute for the 2nd m-line is incomplete, which means that Bob must fill in the imageattr capabilities that it can support, Johansson Expires December 28, 2008 [Page 8] Internet-Draft Image Attributes in SDP June 2008 optionally Bob may remove this attribute for the 2nd m-line, thus indicating that it does not offer any special image attributes for the media in the direction from Bob to Alice other than what may be supported by the payload format. The answer may look like below Answer: m=video 12340 RTP/AVP 97 a=rtpmap:97 H264/90000 a=imageattr:97 [x=480,y=320] a=recvonly m=video 12342 RTP/AVP 98 a=rtpmap:98 H264/90000 a=imageattr:98 sar=[1:16] [x=[480:16:800],y=[320:16:640]] \ [x=[176:8:208],y=[144:8:176]] a=sendonly The answer indicates that Bob wish to render the received video stream with an image size of 480x320. The imageattr for the 2nd m-line is filled with the image attribute capabilities that Bob can support when sending media to Alice. Alice receives the answer and performs another offrer/answer, the 2nd offer answer is necessary to make it possible for Alice to select e.g a desired image size. Offer (2): m=video 12340 RTP/AVP 97 a=rtpmap:97 H264/90000 a=imageattr:97 [x=480,y=320] a=sendonly m=video 12342 RTP/AVP 98 a=rtpmap:98 H264/90000 a=imageattr:98 sar=2 [x=800,y=640] a=recvonly Answer (2): m=video 12340 RTP/AVP 97 a=rtpmap:97 H264/90000 a=imageattr:97 [x=480,y=320] a=recvonly m=video 12342 RTP/AVP 98 a=rtpmap:98 H264/90000 a=imageattr:98 sar=2 [x=800,y=640] a=sendonly The second offer indicates that Alice prefers to render the video on the screen with a image size of 800x600, furthermore it indicates Johansson Expires December 28, 2008 [Page 9] Internet-Draft Image Attributes in SDP June 2008 that it wants the sender (Bob) to compensate for a sample aspect ratio of 12:11. 5. Issues Here is listed a few possible issues and observations connected to the use of the imageattr SDP attribute : o Conflict with sprop-parameter-set: H.264 defines the sprop- parameter-set which may define features that are defined in this draft. To avoid conflicts, sessions that use the imageattr attribute MUST NOT specify similar features in sprop-parameter- set. o Image scaling on encoder side: e.g the H.264 codec is not mandated to rescale the video to fit a preferred receiver image size. If this is the case then a possible solution is that only a part of the input, corresponding to the preferred size, is encoded (cropping). Preferably the cropped image is also displayed as a miniature image on the sender display as well to avoid that the important parts drop out of the image. Cropping of the image may also occur if the picture aspect ratio differs between the video camera and the preferred image size on the receiver end. o Sample aspect ratios (SAR): In this draft version table E1 in [H.264] is referenced. Question if it is better to reference to another table. o Image sharpening: In normal cases image sharpening is to be performed after rescaling on the sender side. There may however occur cases where it is preferable to do the sharpening on the receiver side instead. o A high end device may not see any need for imageattr for it as it most likely has the processing capacity to rescale incoming video and may therefore not include the attribute in the offer. The first answer MAY however include an imageattr with incomplete formulation, this will of course lead to extra offer/answer rounds. o As the likelihood for asymmetric properties (different image size) for each participant is quite big, the outcome is likely that it is necessary to specify media in each direction as given in Section 4.6. This in practice means more overhead in terms of both extra offer/answer and increased RTCP traffic. Johansson Expires December 28, 2008 [Page 10] Internet-Draft Image Attributes in SDP June 2008 o What other methods are available to handle asymmetry? o The syntax in this draft may not follow the rules in RFC4566 6. IANA Considerations T.B.D 7. Security Considerations This draft does not add any additional security issues other than those already existing with currently specified offer/answer procedures. 8. Acknowledgements The author would like to thank the people who has contributed with objections and suggestions to this draft. 9. References 9.1. Informative References [H.264] ITU-T, "ITU-T Recommendation H.264, http://www.itu.int/rec/T-REC-H.264-200711-I/en". [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June 2002. [RFC3984] Wenger, S., Hannuksela, M., Stockhammer, T., Westerlund, M., and D. Singer, "RTP Payload Format for H.264 Video", RFC 3984, February 2005. [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. [RFC4587] Even, R., "RTP Payload Format for H.261 Video Streams", RFC 4587, August 2006. [RFC4629] Ott, H., Bormann, C., Sullivan, G., Wenger, S., and R. Even, "RTP Payload Format for ITU-T Rec", RFC 4629, January 2007. Johansson Expires December 28, 2008 [Page 11] Internet-Draft Image Attributes in SDP June 2008 [S4-080144] 3GPP, "Signaling of Image Size: Combining Flexibility and Low Cost, http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/ TSGS4_48/Docs/S4-080144.zip". 9.2. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. Author's Address Ingemar Johansson Ericsson AB Laboratoriegrand 11 SE-971 28 Lulea SWEDEN Phone: +46 73 0783289 Email: ingemar.s.johansson@ericsson.com Johansson Expires December 28, 2008 [Page 12] Internet-Draft Image Attributes in SDP June 2008 Full Copyright Statement Copyright (C) The IETF Trust (2008). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Johansson Expires December 28, 2008 [Page 13]