INTERNET-DRAFT R. Huang Intended Status: Standard Track R. Even Expires: January 7, 2016 Huawei July 6, 2015 Real-time Transport Control Protocol (RTCP) Feedback Message Extension for Image Control Message draft-huang-avtext-feedback-image-control-00 Abstract This document introduces a method to let receivers control the part of the full video image that they wish to watch. It specifies an extension to the messages defined in the Audio-Visual Profile with Feedback (AVPF), and it can be applied to any RTP video applications. Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Copyright and License Notice Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Expires January 7, 2016 [Page 1] INTERNET DRAFT July 6, 2015 Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 3 4. RTCP Receiver Report Extensions . . . . . . . . . . . . . . . . 4 4.1. Payload-Specific Feedback Messages . . . . . . . . . . . . 5 4.1.1. Image Control Request (ICR) . . . . . . . . . . . . . . 5 4.1.1.1 Message Format . . . . . . . . . . . . . . . . . . . 5 4.1.1.2. Semantics . . . . . . . . . . . . . . . . . . . . . 6 4.1.1.3. Timing Rules . . . . . . . . . . . . . . . . . . . 6 4.1.1.4 Handling of Message in Mixers and Translators . . . 8 4.1.1.5 Remarks . . . . . . . . . . . . . . . . . . . . . . 8 4.1.2 Image Control Notification (ICN) . . . . . . . . . . . 7 4.1.2.1 Message Format . . . . . . . . . . . . . . . . . . . 7 4.1.2.2 Semantics . . . . . . . . . . . . . . . . . . . . . 8 4.1.2.3. Timing Rules . . . . . . . . . . . . . . . . . . . 8 4.1.1.4 Handling of Message in Mixers and Translators . . . 8 4.1.1.5 Remarks . . . . . . . . . . . . . . . . . . . . . . 8 5 Security Considerations . . . . . . . . . . . . . . . . . . . . 8 6 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 8 7 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 8 8 References . . . . . . . . . . . . . . . . . . . . . . . . . . 9 8.1 Normative References . . . . . . . . . . . . . . . . . . . 9 8.2 Informative References . . . . . . . . . . . . . . . . . . 9 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 9 Expires January 7, 2016 [Page 2] INTERNET DRAFT July 6, 2015 1 Introduction 4K Ultra HD technology is by itself a very new trend in the overall electronics landscape, and the impact of it is growing month by month. The new resolution format itself is slowly starting to remake perceptions of where the entire visual media industry will be going over the next few years. However, there are not many 4K display devices out yet, and 99.9% of all final visual media productions are only 1080p HD. And sometimes due to the bandwidth limitation, application providers are unable to provide real 4K services to end users. This document introduces a method to let receivers control the part of the full video image that they wish to watch. It specifies an extension to the messages defined in the Audio-Visual Profile with Feedback (AVPF), and it can be applied to any RTP video applications. 2 Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 3. Motivation This document considers following use cases: 1. In a real-time monitoring application, the user who watches the monitor screen may want to get a closer view of some detail of the video image. He will want an option to ask for a specific part of the full picture allowing the sender to either zoom in to this part or send part of the full picture (may be a 4K resolution) to reduce the used bandwidth. 2. In a video conferencing scenario, one user may prefer a detailed image of one specific person, e.g., the facial expression of the person, while another user may be fine with the initial image. Changing the image can be also done by delivering the original encoder image resolution to the receivers and changing it locally, or by the remote side using camera zoom function. However, that's not always feasible for all cases. In a UHD environment, the issue of bandwidth manifests itself in all of the same processes that are in effect for those files in 1080p resolution. Even after the 4K streams Expires January 7, 2016 [Page 3] INTERNET DRAFT July 6, 2015 have been optimized at the source, they'll still require at least two to three times the bandwidth you'd need today to watch a 1080p HD feed. And in case 2, it's also not feasible to use camera zoom because not all the receivers want to enlarge their pictures. In this document, a new mechanism is introduced which allows applications to signal to the remote encoder the desired image area based on the original one. It is useful for cases where higher definition resolution is available at the source while only streaming of lower definition resolution is supported due to the issues of bandwidth or user presenting devices limitation. The basic idea is to use a new feedback message to signal the area that the receiver requests to enlarge. The specific area is a rectangle indicated by the coordinate of the upper left point and the length and the width. The unit of the coordinate is based on the original picture sent by the sender. It can be illustrated in the following figure 1. 2160 @(x,y) is the start point (upper left point) | | | x Length |.........@**************** | * * | * *width | y * * | ***************** | . |_________._________________________________3840 Figure 1 The illustration of enlarged image area The motivation for using the media path instead of signaling path is similar to [RFC5104]. Media and signaling paths are sometimes separated in systems employing middle box which has the image control ability. As the messages are intended for the media part rather than the signaling part, using media path avoid the transmission across interfaces and unnecessary control traffic between signaling and processing. Moreover, the control handled in media path is faster than signaling path as explained in [RFC5104]. 4. RTCP Receiver Report Extensions The new feedback messages are defined in the following subsections, following a similar structure to that in sections 6.3 of the AVPF specification [RFC4585]. Expires January 7, 2016 [Page 4] INTERNET DRAFT July 6, 2015 4.1. Payload-Specific Feedback Messages As specified by Section 6.1 of [RFC4585], Payload-Specific Feedback Messages are identified by the RTCP packet type value PSFB (206). The following subsections define 2 new FCI formats for the payload- specific feedback messages to extend those defined in [RFC4585] and [RFC5104]. 4.1.1. Image Control Request (ICR) The ICR feedback message is identified by RTCP packet type value PT=PSFB and FMT=ICR. [Note to RFC Editor: Please replace ICR with the IANA provided feedback message type for this message.] The FCI field MUST contain one ICR FCI entries. 4.1.1.1 Message Format The content of the FCI entry for the Image Control Request is showed in Figure 2. The length of the feedback message MUST be set to 4. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seq nr. | x axis of start point | y axis of start | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | point | length | width | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 2: Syntax of an FCI Entry in the ICN Message Seq nr.: 8 bits Request sequence number. The sequence number space is unique for pairing of the SSRC of request source and the SSRC of the request target. The sequence number SHALL be increased by 1 modulo 256 for new command. The initial value is arbitrary. A repetition message SHOULD NOT increase the sequence number. x axis of start point: 14 bits The start point is the upper left point of the rectangle requested to be enlarged. X axis of start point is the value of the horizontal axis of this point. A value of 16383 indicates the point is invalid. Expires January 7, 2016 [Page 5] INTERNET DRAFT July 6, 2015 y axis of start point: 14 bits Y axis of start point is the value of the vertical axis of this point. A value of 16383 indicates the point is invalid. length: 14 bits Length indicates the horizontal side of the rectangle requested to be enlarged. A value of 16383 indicates the rectangle is invalid. width: 14 bits Width indicates the vertical side of the rectangle requested to be enlarged. A value of 16383 indicates the rectangle is invalid. 4.1.1.2. Semantics A decoder can request a image control by sending a ICR message to an encoder. If the encoder has the ability to do such kind of change, it SHOULD take into account the received ICR messages. If one of the values in the message is not valid, the encoder SHOULD give up the attempt of changing and give a notice. The rectangle requested in the ICR message may not suit the final presenting of the image quite well. In that case, the decoder is allowed to adjust according to the final presenting devices. Within the common packet header for feedback messages (as defined in section 6.1 of [RFC4585]),the "SSRC of packet sender" field indicates the source of the request, and the "SSRC of media source" indicates the media source required to be applied the change to. 4.1.1.3. Timing Rules The timing follows the rules outlined in section 3 of [RFC4585]. This request message is not time critical and SHOULD be sent using regular RTCP timing. Only if it is known that the user interface requires quick feedback, the message MAY be sent with early or immediate feedback timing. 4.1.1.4 Handling of Message in Mixers and Translators A mixer of media translator that encodes content sent to the session participant issuing the ICR SHALL consider the request to determine if it can fulfill it. A media translator unable to fulfill the request MAY forward the request unchanged towards the media sender. 4.1.1.5 Remarks Expires January 7, 2016 [Page 6] INTERNET DRAFT July 6, 2015 None. 4.1.2 Image Control Notification (ICN) The ICN feedback message is identified by RTCP packet type value PT=PSFB and FMT=ICN. [Note to RFC Editor: Please replace ICN with the IANA provided feedback message type for this message.] The FCI field MUST contain one ICR FCI entries. 4.1.2.1 Message Format The content of the FCI entry for the Image Control Notification is showed in Figure 3. The length of the feedback message MUST be set to 3. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seq nr. | R | reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 3: Syntax of an FCI Entry in the ICN Message Seq nr.: 8 bits The corresponding request sequence number from the ICR that resulted in this notification. Result (R): 2 bits This field indicates the result that the media sender deals with the ICR message. R=0: the media sender applies the image change successfully. R=1: The values of the ICR message are invalid. R=2: The media sender does not support the image change. R=3: Reserved. Reserved: 22 bits All bits SHALL be set to 0 by the sender and SHALL be ignored on Expires January 7, 2016 [Page 7] INTERNET DRAFT July 6, 2015 reception. 4.1.2.2 Semantics This feedback message is used to acknowledge the reception of a ICR. For each ICR received at the media source, a ICN message SHALL be sent. The notification message SHALL also be sent in response to ICR repetitions received. If the request receiver has received ICR with several different sequence numbers from a single requestor, it SHALL only respond to the request with the highest (modulo 265) sequence number which may be a smaller integer value due to the wrapping of the field. Within the common packet header for feedback messages (as defined in section 6.1 of [RFC4585]),the "SSRC of packet sender" field indicates the source of the notification, and the "SSRC of media source" indicates the media source applying the change to. 4.1.2.3. Timing Rules The timing follows the rules outlined in section 3 of [RFC4585]. This request message is not time critical and SHOULD be sent using regular RTCP timing. 4.1.1.4 Handling of Message in Mixers and Translators A mixer of media translator that acts upon a ICR SHALL send the corresponding ICN. In cases where it needs to forward a ICR itself, the notification message MAY need to be delayed until the ICRhas been responded to. 4.1.1.5 Remarks None. 5 Security Considerations TBD 6 IANA Considerations TBD 7 Acknowledgments TBD Expires January 7, 2016 [Page 8] INTERNET DRAFT July 6, 2015 8 References 8.1 Normative References [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003. [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, "Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, July 2006. [RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman, "Codec Control Messages in the RTP Audio-Visual Profile with Feedback (AVPF)", RFC 5104, February 2008. 8.2 Informative References TBD Authors' Addresses Rachel Huang Huawei 101 Software Avenue, Yuhua District Nanjing 210012 China Email: rachel.huang@huawei.com Roni Even Huawei 14 David Hamelech Tel Aviv 64953 Israel Email: roni.even@mail01.huawei.com Expires January 7, 2016 [Page 9]