Network Working Group I. Johansson Internet-Draft Ericsson AB Intended status: Standards Track K. Jung Expires: May 29, 2011 Samsung Electronics Co., Ltd. Nov 25, 2010 Negotiation of Generic Image Attributes in SDP draft-ietf-mmusic-image-attributes-09 Abstract This document proposes a new generic session set up attribute to make it possible to negotiate different image attributes such as image size. A possible use case is to make it possible for a low-end hand- held terminal to display video without the need to rescale the image, something that may consume large amounts of memory and processing power. The draft also helps to maintain an optimal bitrate for video as only the image size that is desired by the receiver is transmitted. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on May 29, 2011. Copyright Notice Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect Johansson & Jung Expires May 29, 2011 [Page 1] Internet-Draft Image Attributes in SDP Nov 2010 to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Requirements . . . . . . . . . . . . . . . . . . . . . . . 4 2. Conventions, Definitions and Acronyms . . . . . . . . . . . . 5 3. Specification of the 'imageattr' SDP Attribute . . . . . . . . 5 3.1. Attribute syntax . . . . . . . . . . . . . . . . . . . . . 5 3.1.1. Overall view of syntax . . . . . . . . . . . . . . . . 5 3.2. Considerations . . . . . . . . . . . . . . . . . . . . . . 11 3.2.1. No imageattr in 1st offer . . . . . . . . . . . . . . 11 3.2.2. Asymmetry . . . . . . . . . . . . . . . . . . . . . . 11 3.2.3. sendonly and recvonly . . . . . . . . . . . . . . . . 11 3.2.4. Sample aspect ratio . . . . . . . . . . . . . . . . . 12 3.2.5. SDPCapNeg support . . . . . . . . . . . . . . . . . . 12 3.2.6. Interaction with codec parameters . . . . . . . . . . 13 3.2.7. Change of display in middle of session . . . . . . . . 15 3.2.8. Use with layered codecs . . . . . . . . . . . . . . . 15 3.2.9. Addition of parameters . . . . . . . . . . . . . . . . 15 4. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4.1. A High-Level Example . . . . . . . . . . . . . . . . . . . 15 4.2. Detailed Examples . . . . . . . . . . . . . . . . . . . . 16 4.2.1. Example 1 . . . . . . . . . . . . . . . . . . . . . . 16 4.2.2. Example 2 . . . . . . . . . . . . . . . . . . . . . . 17 4.2.3. Example 3 . . . . . . . . . . . . . . . . . . . . . . 18 4.2.4. Example 4 . . . . . . . . . . . . . . . . . . . . . . 19 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 6. Security Considerations . . . . . . . . . . . . . . . . . . . 20 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 20 8. Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 9.1. Normative References . . . . . . . . . . . . . . . . . . . 22 9.2. Informative References . . . . . . . . . . . . . . . . . . 23 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 Johansson & Jung Expires May 29, 2011 [Page 2] Internet-Draft Image Attributes in SDP Nov 2010 1. Introduction This document proposes a new attribute to make it possible to negotiate different image attributes such as image size. The term image size is defined here as it may differ from the physical screen size of for instance a hand-held terminal. As an example it may be beneficial to display a video image on a part of the physical screen and leave space on the screen for other features such as menus and other info. There are a number of benefits with a possibility to negotiate the image size: o Less image distortion: Rescaling of images introduces additional distortion, something that can be avoided (at least on the receiver side) if the image size can be negotiated. o Reduced receiver complexity: Image rescaling can be quite computation intensive. For low end devices this can be a problem. o Optimal quality for the given bitrate: The sender does not need to encode an entire CIF (352x288) image if only an image size of 288x256 is displayed on the receiver screen. This alternatively gives a saving in bitrate. o Memory requirement: The receiver device will know the size of the image and can then allocate memory accordingly. o Optimal aspect ratio: The indication of the supported image sizes and aspect ratio allows the receiver to select the most appropriate combination based on its rescaling capabilities and the desired rendering. For example, if a sender proposes three resolutions in its SDP offer, 100x200, 200x100 and 100x100 with sar=1.0 (1:1) etc. then the receiver can select the option that fits the receiver screen best. In cases where rescaling is not implemented (for example, rescaling is not mandatory to implement in H.264 [H.264]), the indication of the image attributes may still provide an optimal use of bandwidth because the attribute will anyway give the encoder a better indication about what image size is preferred and will thus help to avoid wasting bandwidth by encoding with an unnecessarily large resolution. For implementers that are considering rescaling issues, it is worth notice that there are several benefits to do it on the sender side: Johansson & Jung Expires May 29, 2011 [Page 3] Internet-Draft Image Attributes in SDP Nov 2010 o Rescaling on the sender/encoder side is likely to be easier to do as the camera related software/hardware already contains the necessary functionality for zooming/cropping/trimming/sharpening the video signal. Moreover, rescaling is generally done in RGB or YUV domain and should not depend on the codecs used. o The encoder may be able to encode in a number of formats but may not know which format to choose as, without the image attribute, it does not know the receiver's performance or preference. o The quality drop due to digital domain rescaling using interpolation is likely to be lower if it is done before the video encoding rather than after the decoding especially when low bitrate video coding is used. o If low-complexity rescaling operations such as simple cropping must be performed, the benefit with having this functionality on the sender side is that it is then possible to present a miniature "what you send" image on the display to help the user to target the camera correctly. Several of the existing standards ([H.263], [H.264] and [MPEG-4]) have support for different resolutions at different framerates. The purpose of this document is to provide for a generic mechanism and is targeted mainly at the negotiation of the image size but to make it more general the attribute is named 'imageattr'. This document is limited to point-to-point unicast communication scenarios. The attribute may be used in centralized conferencing scenarios as well but due to the abundance of configuration options it may then be difficult to come up with a configuration that fits all parties. 1.1. Requirements The design of the image attribute is based on the following requirements which are listed only for informational purposes: REQ-1: Support the indication of one or more set(s) of image attributes that the SDP endpoint wish to receive or send. Each image attribute set must include a specific image size. REQ-2: Support set up/negotiation of image attributes, meaning that each side in the Offer/Answer should be able to negotiate the image attributes it prefers to send and receive. Johansson & Jung Expires May 29, 2011 [Page 4] Internet-Draft Image Attributes in SDP Nov 2010 REQ-3: Interoperate with codec specific parameters such as sprop- parameter-sets in [H.264] or config in [MPEG-4]. REQ-4: Make the attribute generic with as few codec specific details/tricks as possible in order to be codec agnostic. Besides the above mentioned requirements, the requirement below may be applicable. OPT-1: The image attribute should support the description of image- related attributes for various types of media, including video, pictures, images, etc. 2. Conventions, Definitions and Acronyms The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 3. Specification of the 'imageattr' SDP Attribute This section defines the SDP image attribute 'imageattr', which can be used in an SDP Offer/Answer exchange to indicate various image attribute parameters. In this document, we define the following image attribute parameters: image resolution, sample aspect ratio (sar), allowed range in picture aspect ratio (par) and the preference of a given parameter set over another (q). The attribute is extensible and guidelines for defining additional parameters are provided in Section 3.2.9. 3.1. Attribute syntax In this section the syntax of the 'imageattr' attribute is described. The 'imageattr' attribute is a media-level attribute. The section is split up in two parts, the first gives an overall view of the syntax while the second describes how the syntax is used. 3.1.1. Overall view of syntax The syntax for the image attribute is in ABNF [RFC5234]: Johansson & Jung Expires May 29, 2011 [Page 5] Internet-Draft Image Attributes in SDP Nov 2010 ---- image-attr = "imageattr:" PT 1*2( 1*WSP ( "send" / "recv" ) 1*WSP attr-list ) PT = 1*DIGIT / "*" attr-list = ( set *(1*WSP set) ) / "*" ; WSP and DIGIT defined in [RFC5234] ---- ---- set= "[" "x=" xyrange "," "y=" xyrange *( "," key-value ) "]" ; x is the horizontal image size range (pixel count) ; y is the vertical image size range (pixel count) key-value = ( "sar=" srange ) / ( "par=" prange ) / ( "q=" qvalue ) ; Key-value MAY be extended with other keyword ; parameters. ; At most one instance each of sar, par, or q ; allowed in a set. ; ; sar (sample aspect ratio) is the sample aspect ratio ; associated with the set (optional,MAY be ignored) ; par (picture aspect ratio) is the allowed ; ratio between the display's x and y physical ; size (optional) ; q (optional, range [0.0..1.0], default value 0.5) ; is the preference for the given set, ; a higher value means a higher preference Johansson & Jung Expires May 29, 2011 [Page 6] Internet-Draft Image Attributes in SDP Nov 2010 ---- onetonine = "1" / "2" / "3" / "4" / "5" / "6" / "7" / "8" / "9" ; Digit between 1 and 9 xyvalue = onetonine *5DIGIT ; Digit between 1 and 9 which is ; followed by 0 to 5 other digits step = xyvalue xyrange = ( "[" xyvalue ":" [ step ":" ] xyvalue "]" ) ; Range between a lower and an upper value ; with an optional step, default step = 1 ; The rightmost occurence of xyvalue MUST have a ; higher value than the leftmost occurence. / ( "[" xyvalue 1*( "," xyvalue ) "]" ) ; Discrete values separated by ',' / ( xyvalue ) ; A single value spvalue = ( "0" "." onetonine *3DIGIT ) ; Values between 0.1000 and 0.9999 / ( onetonine "." 1*4DIGIT ) ; Values between 1.0000 and 9.9999 srange = ( "[" spvalue 1*( "," spvalue ) "]" ) ; Discrete values separated by ','. ; Each occurrence of spvalue MUST be ; greater than the previous occurrence. / ( "[" spvalue "-" spvalue "]" ) ; Range between a lower and an upper level (inclusive) ; The second occurence of spvalue MUST have a higher ; value than the first / ( spvalue ) ; A single value prange = ( "[" spvalue "-" spvalue "]" ) ; Range between a lower and an upper level (inclusive) ; The second occurence of spvalue MUST have a higher ; value than the first qvalue = ( "0" "." 1*2DIGIT ) / ( "1" "." 1*2("0") ) ; Values between 0.00 and 1.00 ---- o The attribute typically contains a "send" and a "recv" keyword. These specify the preferences for the media once the session is set up, in the send and receive direction respectively from the point of view of the sender of the session description. o The "send" keyword and corresponding attribute list (attr-list) MUST NOT occur more than once per image attribute. Johansson & Jung Expires May 29, 2011 [Page 7] Internet-Draft Image Attributes in SDP Nov 2010 o The "recv" keyword and corresponding attribute list (attr-list) MUST NOT occur more than once per image attribute. o PT is the payload type number, it MAY be set to * to indicate that the attribute applies to all payload types in the media description. o For sendrecv streams both of the send and recv directions SHOULD BE present. o For inactive streams it is RECOMMENDED that both of the send and recv directions are present. o For sendonly or recvonly streams one of the directions MAY be omitted. See Section 3.2.3. 3.1.1.1. Parameter rules For the parameters the following rules apply. Payload type number (PT): The image attribute is bound to a specific codec by means of the payload type number. A wild card (*) can be specified for the payload type number to indicate that it applies to all payload types in the media description. Several image attributes can be defined for instance for different video codec alternatives conditioned that the payload type number differs. Note that the attribute is associated to the codec(s), for instance an SDP offer may specify payload type number 101 while the answer may specify 102, this may make it troublesome to specify a payload type number with the 'imageattr' attribute. The only solution that works in this case is to use a wild card (*) and thus specify that the image attribute applies to all codecs in a media description. Preference (q): The preference for each set is 0.5 by default, setting the optional q parameter to another value makes it possible to set different preferences for the sets. A higher value gives a higher preference for the given set. sar: The sar parameter specifies the sample aspect ratio associated to the given range of x and y values. The sar parameter is defined as dx/dy where dx and dy is the physical size of the pixels. Square pixels gives a sar=1.0. The parameter sar MAY be expressed as a range or as a single value. If this parameter is not present a default sar value of 1.0 is assumed. The interpretation of sar differs between the send and the receive directions. Johansson & Jung Expires May 29, 2011 [Page 8] Internet-Draft Image Attributes in SDP Nov 2010 * In the send direction it defines a specific sample aspect ratio associated to a given x and y image size (range). * In the recv direction sar expresses that the receiver of the given medium prefers to receive a given x and y resolution with a given sample aspect ratio. See Section 3.2.4 for a more detailed discussion. The sar parameter will likely not solve all the issues that are related to different sample aspect ratios but it can help to solve them and reduce aspect ratio distortion. The response MUST NOT include a sar parameter if there is no acceptable value given. The reason to this is that if the response includes a sar parameter it is interpreted as "sar parameter accepted" while removal of the sar parameter is treated as "sar parameter not accepted", for this reason it is safer to remove an unacceptable sar parameter altogether. par: The par (width/height = x/y ratio) parameter indicates a range of allowed ratios between x and y physical size (picture aspect ratio). This is used to limit the number of x and y image size combinations, par is given as ---- par=[ratio_min-ratio_max] ---- Where ratio_min and ratio_max are the min and max allowed picture aspect ratios. If sar and the sample aspect ratio that the receiver actually uses in the display are the same (or close), the relation between the x and y pixel resolution and the physical size of the image is straightforward. If however sar differs from the sample aspect ratio of the receiver display this must be taken into consideration when the x and y pixel resolution alternatives are sorted out. 3.1.1.2. Offer/answer rules In accordance with [RFC3264], offer answer exchange of the image attribute is as follows. o Offerer sending the offer: * The offerer must be able to support the image attributes that it offers. The exception is if the offerer has expressed a wild card (*) in the attribute list. * It is recommended that a device which sees no reason to use the image attribute, anyway includes the attribute with wild cards Johansson & Jung Expires May 29, 2011 [Page 9] Internet-Draft Image Attributes in SDP Nov 2010 (*) in the attribute lists for the send and recv directions. An example of this looks like: ---- a=imageattr:97 send * recv * ---- This gives the answerer the possibility to express its preferences. The use of wild cards introduces a risk that the message size can increase in an uncontrolled way. To reduce this risk these wild cards SHOULD only be replaced by an as small set as possible. o Answerer receiving the offer and sending the answer: * The answerer may choose to keep the image attribute but is not required to do so. * The answerer may, for its receive and send direction, include one or more entries that it can support from the set of entries proposed in the offer. * The answerer may also, for its receive and send direction, replace the entries with a complete new set of entries different from the original proposed by the offerer. The implementor of this feature should however be aware that this may cause extra offer/answer exchanges. * The answerer may also remove its send direction completely if it is deemed that it cannot support any of the proposed entries. * The answerer should not include an image attribute in the answer if it was not present in the offer. o Offerer receiving the answer: * If the image attribute is not included in the SDP answer the offerer SHOULD continue to process the answer as if this mechanism had not been offered. * If the image attribute is included in the SDP answer but none of the entries are usable or acceptable, the offerer MUST resort to other methods to determine the appropriate image size. In this case the offerer must also issue a new offer/ answer without the image attribute to avoid misunderstandings between offerer and answerer. This will avoid the risk on infinite negotiations. Johansson & Jung Expires May 29, 2011 [Page 10] Internet-Draft Image Attributes in SDP Nov 2010 3.2. Considerations 3.2.1. No imageattr in 1st offer When the initial offer does not contain the 'imageattr' attribute, the rules in Section 3.1.1.2 require the attribute to be absent in the answer. The reasons for this are: o The offerer of the initial SDP is not likely to understand the image attribute if it did not include it in the offer, bearing in mind that Section 3.1.1 recommends that the offerer provide the attribute with wildcarded parameters if it has no preference. o Inclusion of the image attribute in the answer may come in conflict with the rules in Section 3.1.1.2 esp. the rules that apply to "offerer receiving the answer". For the above reasons it is RECOMMENDED that a device which sees no reason to use the image attribute, anyway includes the attribute with wild cards (*) in the attribute lists for the send and recv directions. 3.2.2. Asymmetry While the image attribute supports asymmetry there are some limitations to this. One important limitation is that the codec being used can only support up to a given maximum resolution for a given profile level. As an example H.264 [H.264] with profile level 1.2 does not support higher resolution than 352x288 (CIF). The offer/answer rules imply that the same profile level must be used in both directions. This means that in an asymmetric scenario where Alice wants an image size of 580x360 and Bob wants 150x120, profile level 2.2 is needed in both directions even though profile level 1 would have been sufficient in one direction. Currently, the only solution to this problem is to specify two unidirectional media descriptions. Note however that the asymmetry issue for the H.264 codec is solved by means of the level-asymmetry- allowed parameter in [RFC3984bis]. 3.2.3. sendonly and recvonly If the directional attributes a=sendonly or a=recvonly are given for a medium, there is of course no need to specify the image attribute for both directions. Therefore one of the directions in the attribute may be omitted. However it may be good to do the image Johansson & Jung Expires May 29, 2011 [Page 11] Internet-Draft Image Attributes in SDP Nov 2010 attribute negotiation in both directions in case the session is updated for media in both directions at a later stage. 3.2.4. Sample aspect ratio The relationship between the sar parameter and the x and y pixel resolution deserves some extra discussion. Consider the offer from Alice to Bob (we set the recv direction aside for the moment): ---- a=imageattr:97 send [x=720,y=576,sar=1.1] ---- If the receiver display has square pixels the 720x576 image would need to be rescaled to for example 792x576 or 720x524 to ensure a correct image aspect ratio. This in practice means that rescaling would need to be performed on the receiver side, something that is contrary to the spirit of this draft. To avoid this problem Alice may specify a range of values for the sar parameter like: ---- a=imageattr:97 send [x=720,y=576,sar=[0.91,1.0,1.09,1.45]] ---- Meaning that Alice can encode with any of the mentioned sample aspect ratios, leaving to Bob to decide which one he prefers. 3.2.5. SDPCapNeg support The image attribute can be used within the SDP Capability Negotiation [RFC5939] framework and its use is then specified using the "a=acap" parameter. An example is ---- a=acap:1 imageattr:97 send [x=720,y=576,sar=[0.91,1.0,1.09,1.45]] ---- For use with SDP Media Capability Negotiation extension [SDPMedCapNeg], where it is no longer possible to specify payload type numbers, it is possible to use the parameter substitution rule, an example of this is. ---- ... a=mcap:1 video H264/90000 a=acap:1 imageattr:%1% send [x=720,y=576,sar=[0.91,1.0,1.09,1.45]] ... ---- Where %1% maps to media capability number 1. It is also possible to use the a=mscap attribute like in the example below. Johansson & Jung Expires May 29, 2011 [Page 12] Internet-Draft Image Attributes in SDP Nov 2010 ---- ... a=mcap:1 video H264/90000 a=mscap:1 imageattr send [x=720,y=576,sar=[0.91,1.0,1.09,1.45]] ... ---- 3.2.6. Interaction with codec parameters As the SDP for most codecs already specifies some kind of indication of, for example, the image size, at session set up, measures must be taken to avoid conflicts between the image attribute and this already existing information. The following subsections describe the most well known codecs and how they define image-size related information. Section 3.2.6.4 outlines a few possible solutions, but this document does not make any recommendation for any of them. 3.2.6.1. H.263 The payload format for H.263 [H.263] is described in [RFC4629]. H.263 defines (on the fmtp line) a list of image sizes and their maximum frame rates (profiles) that the offerer can receive. The answerer is not allowed to modify this list and must reject a payload type that contains an unsupported profile. The CUSTOM profile may be used for image size negotiation but support for asymmetry requires the specification of two unidirectional media descriptions using the sendonly/recvonly attributes. 3.2.6.2. H.264 The payload format for H.264 [H.264] is described in [RFC3984] and updated in [RFC3984bis]. H.264 defines image size related information in the fmtp line by means of sprop-parameter-sets. According to the specification several sprop-parameter-sets may be defined for one payload type. The sprop-parameter-sets describe the image size (+ more) that the offerer sends in the stream and need not be complete. This means that this does not represent any negotiation. Moreover an answer is not allowed to change the sprop-parameter-sets. This configuration may be changed later inband if for instance image sizes need to be changed or added. Johansson & Jung Expires May 29, 2011 [Page 13] Internet-Draft Image Attributes in SDP Nov 2010 3.2.6.3. MPEG-4 The payload format for MPEG-4 [MPEG-4] is described in [RFC3016]. MPEG-4 defines a config parameter on the fmtp line which is a hexadecimal representation of the MPEG-4 visual configuration information. This configuration does not represent any negotiation and the answer is not allowed to change the parameter. It is not possible to change the configuration using inband signaling. 3.2.6.4. Possible solutions The subsections above clearly indicate that this kind of information must be aligned well with the image attribute to avoid conflicts. There are a number of possible solutions, listed below without any preference: o Ignore payload format parameters: This may not work well in the presence of bad channel conditions especially in the beginning of a session. Moreover this is not a good option for MPEG-4. o 2nd session-wide offer/answer round: In the 2nd offer/answer the codec payload format specific parameters are defined based on the outcome of the 'imageattr' negotiation. The drawback with this is that set up of the entire session (including audio) may be delayed considerably, especially as the 'imageattr' negotiation can already itself cost up to two offer/answer rounds. Also the conflict between the 'imageattr' negotiation and the payload format specific parameters is still present after the first offer/ anser round and a fuzzy/buggy implementation may start media before the second offer/answer is completed with unwanted results. o 2nd session-wide offer/answer round only for video: This is similar to the alternative above with the exception that set up time for audio is not increased, moreover the port number for video is set to 0 during the 1st offer answer round to avoid that media flows. This has the effect that video will blend in some time after the audio is started (up to 2 seconds delay). This alternative is likely the most clean-cut and failsafe alternative. The drawback is, as the port number in the first offer is always zero, the media startup will always be delayed even though it would in fact have been possible to start media already after the first offer/ answer round. Note that according to [RFC3264], a port number of zero means that the whole media line is rejected meaning that a new offer for the Johansson & Jung Expires May 29, 2011 [Page 14] Internet-Draft Image Attributes in SDP Nov 2010 same port number should be treated as a completely new stream and not as an update. The most safe way to solve this problem is to use preconditions, this is however outside the scope of this document. 3.2.7. Change of display in middle of session A very likely scenario is that a user switches to another phone during a video telephony call or plugs a cellphone into an external monitor. In both cases it is very likely that a renegotiation is initiated using the SIP-REFER or SIP-UPDATE methods. It is RECOMMENDED to negotiate the image size during this renegotiation. 3.2.8. Use with layered codecs As the image attribute is a media level attribute, its use with layered codecs cause some concern. If the layers are transported in different RTP streams the layers are specified on different media descriptions and the relation is specified using the grouping framework [RFC5888] and the depend attribute [RFC5583]. As it is not possible to specify only one image attribute for several media descriptions the solution is either to specify the same image attribute for each media description, or to only specify the image attribute for the base layer. 3.2.9. Addition of parameters The image attribute allows for the addition of parameters in the future. To make backwards adaptation possible; an entity that processes the attribute MUST ignore any unknown parameters in the offer and MUST NOT include them in the answer it generates. Addition of future parameters that are not understood by the receiving endpoint may lead to ambiguities if mutual dependencies between parameters exist, therefore addition of parameters must be done with great care. 4. Examples This section gives some more information on how to use the attribute by means of a high-level example and a few detailed examples. 4.1. A High-Level Example Assume that Alice wishes to set up a session with Bob and that Alice takes the first initiative. The syntactical white-space delimiters (1*WSP) and double-quotes are removed to make reading easier. Johansson & Jung Expires May 29, 2011 [Page 15] Internet-Draft Image Attributes in SDP Nov 2010 In the offer Alice provides information for both the send and receive (recv) directions. For the send direction Alice provides a list that the answerer can select from. For the receive direction Alice may either specify a desired image size range right away or a * to instruct Bob to reply with a list of image sizes that Bob can support for sending. Using the overall high level syntax the image attribute may then look like ---- a=imageattr:PT send attr-list recv attr-list ---- or ---- a=imageattr:PT send attr-list recv * ---- In the first alternative the recv direction may be a full list of desired image size formats. It may however (and most likely) just be a list with one alternative for the preferred x and y resolution. If Bob supports an x and y resolution in at least one of the X and Y ranges given in the send attr-list and in the recv attr-list of the offer, the answer from Bob will look like: ---- a=imageattr:PT send attr-list recv attr-list ---- And the offer answer negotiation is done. Worth notice here is that the attr-list will likely be pruned in the answer. While it may contain many different alternatives in the offer it may in the end contain just one or two alternatives. If Bob does not support any x and y resolution in one of the provided send or recv ranges given in the send attr-list or in the recv attr- list, the corresponding part is removed completely. For instance, if Bob doesn't support any of the offered alternatives in the recv attr- list in the offer, the answer from Bob would look like: ---- a=imageattr:PT recv attr-list ---- 4.2. Detailed Examples This section gives a few detailed examples, it is assumed where needed that Alice initiates a session with Bob 4.2.1. Example 1 Two image resolution alternatives are offered with 800x640 with sar=1.1 having the highest preference Johansson & Jung Expires May 29, 2011 [Page 16] Internet-Draft Image Attributes in SDP Nov 2010 It is also indicated that Alice wish to display video with a resolution of 330x250 on her display ---- a=imageattr:97 send [x=800,y=640,sar=1.1,q=0.6] [x=480,y=320] \ recv [x=330,y=250] ---- In case Bob accepts the "recv [x=330,y=250]" the answer may look like ---- a=imageattr:97 recv [x=800,y=640,sar=1.1] \ send [x=330,y=250] ---- Indicating that the receiver (Bob) wish the encoder (on Alice's side) to compensate for a sample aspect ratio of 1.1 (11:10) and desires an image size on its screen of 800x640. There is however a possibility that "recv [x=330,y=250]" is not supported. If the case, Bob may completely remove this part or replace it with a list of supported image sizes. ---- a=imageattr:97 recv [x=800,y=640,sar=1.1] \ send [x=[320:16:640],y=[240:16:480],par=[1.2-1.3]] ---- Alice can then select a valid image size which is closest to the one that was originally desired (336x256) and performs a second offer/ answer ---- a=imageattr:97 send [x=800,y=640,sar=1.1] \ recv [x=336,y=256] ---- Bob replies with: ---- a=imageattr:97 recv [x=800,y=640,sar=1.1] \ send [x=336,y=256] ---- 4.2.2. Example 2 Two image resolution sets are offered with the first having a higher preference (q=0.6). Johansson & Jung Expires May 29, 2011 [Page 17] Internet-Draft Image Attributes in SDP Nov 2010 ---- a=imageattr:97 \ send [x=[480:16:800],y=[320:16:640],par=[1.2-1.3],q=0.6] \ [x=[176:8:208],y=[144:8:176],par=[1.2-1.3]] \ recv * ---- The x-axis resolution can take the values 480 to 800 in 16 pixels steps and 176 to 208 in 8 pixels steps. The par parameter limits the set of possible x and y screen resolution combinations such that 800x640 (ratio=1.25) is a valid combination while 720x608 (ratio=1.18) or 800x608 (ratio=1.31) are invalid combinations. For the recv direction (Bob->Alice) Bob is requested to provide with a list of supported image sizes 4.2.3. Example 3 In this example a more of the SDP offer is shown. ---- m=video 49154 RTP/AVP 99 a=rtpmap:99 H264/90000 a=fmtp:99 packetization-mode=0;profile-level-id=42e011; \ sprop-parameter-sets=Z0LgC5ZUCg/I,aM4BrFSAa a=imageattr:99 \ send [x=176,y=144] [x=224,y=176] [x=272,y=224] [x=320,y=240] \ recv [x=176,y=144] [x=224,y=176] [x=272,y=224,q=0.6] [x=320,y=240] ---- In the send direction, sprop-parameter-sets is defined for a resolution of 320x240 which is the largest image size offered in the send direction. This means that if 320x240 is selected, no additional offer/answer is necessary. In the receive direction four alternative image sizes are offered with 272x224 being the preferred choice. The answer may look like: ---- m=video 49154 RTP/AVP 99 a=rtpmap:99 H264/90000 a=fmtp:99 packetization-mode=0;profile-level-id=42e011; \ sprop-parameter-sets=Z0LgC5ZUCg/I,aM4BrFSAa a=imageattr:99 send [x=320,y=240] recv [x=320,y=240] ---- Indicating (in this example) that the image size is 320x240 in both directions. Although the offerer preferred 272x224 for the receive direction, the answerer might not be able to offer 272x224 or not Johansson & Jung Expires May 29, 2011 [Page 18] Internet-Draft Image Attributes in SDP Nov 2010 allow encoding and decoding of video of different image sizes simultaneously. The answerer sets new sprop-parameter-sets, constructed for both send and receive directions at the restricted conditions and image size of 320x240. 4.2.4. Example 4 This example illustrates in more detail how compensation for different sample aspect ratios can be negotiated with the image attribute. We set up a session between Alice and Bob, Alice is the offerer of the session. The offer (from Alice) contains the image attribute below: ---- a=imageattr:97 \ send [x=400:16:800],y=[320:16:640],sar=[1.0-1.3],par=[1.2-1.3]] \ recv [x=800,y=600,sar=1.1] ---- First we consider the recv direction: The offerer (Alice) explicitly states that she wish to receive the screen resolution 800x600, however she also indicates that the screen on her display does not use square pixels, the sar value=1.1 means that Bob must (preferably) compensate for this. So.. If Bob's video camera produces square pixels, and wish to satisfy Alice's sar requirement, the image processing algorithm must rescale a 880x600 pixel image (880=800*1.1) to 800x600 pixels (could be done other ways). ... and now the send direction: Alice indicates that she can (in the image processing algorithms) rescale the image for sample aspect ratios in the range 1.0 to 1.3. She can also provide with a number of different image sizes (in pixels) ranging from 400x320 to 800x640. Bob inspects the offered sar and image sizes and responds with the modified image attribute ---- a=imageattr:97 \ recv [x=464,y=384,sar=1.15] \ send [x=800,y=600,sar=1.1] ---- Alice will, in order to satisfy Bob's request, need to rescale the image from her video camera from 534x384 (534=464*1.15) to 464x384. Neither part is required to rescale like this (sar may be ignored), the consequence will of course be a distorted image. Johansson & Jung Expires May 29, 2011 [Page 19] Internet-Draft Image Attributes in SDP Nov 2010 5. IANA Considerations Following the guidelines in [RFC4566], the IANA is requested to register one new SDP attribute: Attribute name: imageattr Long form name: Image attribute Type of attribute: Media-level Subject to charset: No Purpose: This attribute defines the ability to negotiate various image attributes such as image sizes. The attribute contains a number of parameters which can be modified in and offer/answer exchange. Appropriate values: See Section 3.1.1 of RFCXXXX Contact name: Authors of RFCXXXX Note to RFC Editor: please replace "RFCXXXX" above with the RFC number of this document, and remove this note. 6. Security Considerations This draft does not add any additional security issues other than those already existing with currently specified offer/answer procedures. 7. Acknowledgements The authors would like to thank the people who has contributed with objections and suggestions to this draft and provided with valuable guidance in the amazing video-coding world. Special thanks go to Clinton Priddle, Roni Even, Randell Jesup, and Dan Wing. 8. Changes Note to RFC Editor: please replace remove this section in its entirety before publication. The main changes are: Johansson & Jung Expires May 29, 2011 [Page 20] Internet-Draft Image Attributes in SDP Nov 2010 From WG -08 to WG -09 * Clarified the issue that offer and answer may use different PT numbers * Clarified that wild cards in send and recv direction should be used with care and with total message size in mind * Typos and unclear language fixe From WG -07 to WG -08 * SHOULD changed to MUST in section "Offer/answer rules" From WG -05 to WG -06 & -07 * Update based on AD review comments, no changes to fix issue related to RFC2119 keywords * Minor editorial changes * Added extra example to use of attribute with SDPCapNeg From WG -04 to WG -05 * Review based on WGLC comments * ABNF improved * Change use of RFC2119 keywords (MUST, SHOULD, MAY) to lowercase in some sections * Clarification on the directions send and recv in sendrecv, inactive modes * Clarification around sar parameter added From WG -03 to WG -04 * Rearrangement of text * Clarification of offer/answer rules * Cleaned IANA section Johansson & Jung Expires May 29, 2011 [Page 21] Internet-Draft Image Attributes in SDP Nov 2010 From WG -02 to WG -03 * Partial update based on review comments from Jean-Francois Mule From WG -01 to WG -02 * Added extra example that highlights the negotiation of sar From WG -00 to WG -01 * Added info about future addition of parameters and backwards compatibility * Added IANA considerations From individual -02 to WG -00 * Cleanup of syntax, ABNF form * Additional example From -01 to -02 * Cleanup of the sar and par parameters to make them match the established conventions * Requirement specification added * New bidirectional syntax * Interoperability considerations with well known video codecs discussed 9. References 9.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June 2002. [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. Johansson & Jung Expires May 29, 2011 [Page 22] Internet-Draft Image Attributes in SDP Nov 2010 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008. [RFC5583] Schierl, T. and S. Wenger, "Signaling Media Decoding Dependency in the Session Description Protocol (SDP)", RFC 5583, July 2009. [RFC5888] Camarillo, G. and H. Schulzrinne, "The Session Description Protocol (SDP) Grouping Framework", RFC 5888, June 2010. 9.2. Informative References [H.263] ITU-T, "ITU-T Recommendation H.263 (2005): "Video coding for low bit rate communication"". [H.264] ITU-T, "ITU-T Recommendation H.264, http://www.itu.int/rec/T-REC-H.264-200711-I/en". [MPEG-4] ISO/IEC, "ISO/IEC 14496-2:2004: "Information technology - Coding of audio-visual objects - Part 2: Visual"". [RFC3016] Kikuchi, Y., Nomura, T., Fukunaga, S., Matsui, Y., and H. Kimata, "RTP Payload Format for MPEG-4 Audio/Visual Streams", RFC 3016, November 2000. [RFC3984] Wenger, S., Hannuksela, M., Stockhammer, T., Westerlund, M., and D. Singer, "RTP Payload Format for H.264 Video", RFC 3984, February 2005. [RFC3984bis] IETF, "RTP Payload Format for H.264 Video, http:// tools.ietf.org/wg/avt/draft-ietf-avt-rtp-rfc3984bis/". [RFC4629] Ott, H., Bormann, C., Sullivan, G., Wenger, S., and R. Even, "RTP Payload Format for ITU-T Rec", RFC 4629, January 2007. [RFC5939] Andreasen, F., "Session Description Protocol (SDP) Capability Negotiation", RFC 5939, September 2010. [SDPMedCapNeg] IETF, "SDP media capabilities Negotiation, http:// tools.ietf.org/wg/mmusic/ draft-ietf-mmusic-sdp-media-capabilities". Johansson & Jung Expires May 29, 2011 [Page 23] Internet-Draft Image Attributes in SDP Nov 2010 Authors' Addresses Ingemar Johansson Ericsson AB Laboratoriegrand 11 SE-971 28 Luleae SWEDEN Phone: +46 73 0783289 Email: ingemar.s.johansson@ericsson.com Kyunghun Jung Samsung Electronics Co., Ltd. Dong Suwon P.O. Box 105 416, Maetan-3Dong, Yeongtong-gu Suwon-city, Gyeonggi-do Korea 442-600 Phone: +82 10 9909 4743 Email: kyunghun.jung@samsung.com Johansson & Jung Expires May 29, 2011 [Page 24]