Internet DRAFT - draft-huang-avtext-feedback-image-control
draft-huang-avtext-feedback-image-control
INTERNET-DRAFT R. Huang
Intended Status: Standard Track R. Even
Expires: January 7, 2016 Huawei
July 6, 2015
Real-time Transport Control Protocol (RTCP) Feedback Message Extension
for Image Control Message
draft-huang-avtext-feedback-image-control-00
Abstract
This document introduces a method to let receivers control the part
of the full video image that they wish to watch. It specifies an
extension to the messages defined in the Audio-Visual Profile with
Feedback (AVPF), and it can be applied to any RTP video
applications.
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/1id-abstracts.html
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
Copyright and License Notice
Copyright (c) 2015 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
<R. Huang, et al> Expires January 7, 2016 [Page 1]
INTERNET DRAFT <RTCP Feedback Image Control> July 6, 2015
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 3
4. RTCP Receiver Report Extensions . . . . . . . . . . . . . . . . 4
4.1. Payload-Specific Feedback Messages . . . . . . . . . . . . 5
4.1.1. Image Control Request (ICR) . . . . . . . . . . . . . . 5
4.1.1.1 Message Format . . . . . . . . . . . . . . . . . . . 5
4.1.1.2. Semantics . . . . . . . . . . . . . . . . . . . . . 6
4.1.1.3. Timing Rules . . . . . . . . . . . . . . . . . . . 6
4.1.1.4 Handling of Message in Mixers and Translators . . . 8
4.1.1.5 Remarks . . . . . . . . . . . . . . . . . . . . . . 8
4.1.2 Image Control Notification (ICN) . . . . . . . . . . . 7
4.1.2.1 Message Format . . . . . . . . . . . . . . . . . . . 7
4.1.2.2 Semantics . . . . . . . . . . . . . . . . . . . . . 8
4.1.2.3. Timing Rules . . . . . . . . . . . . . . . . . . . 8
4.1.1.4 Handling of Message in Mixers and Translators . . . 8
4.1.1.5 Remarks . . . . . . . . . . . . . . . . . . . . . . 8
5 Security Considerations . . . . . . . . . . . . . . . . . . . . 8
6 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 8
7 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 8
8 References . . . . . . . . . . . . . . . . . . . . . . . . . . 9
8.1 Normative References . . . . . . . . . . . . . . . . . . . 9
8.2 Informative References . . . . . . . . . . . . . . . . . . 9
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 9
<R. Huang, et al> Expires January 7, 2016 [Page 2]
INTERNET DRAFT <RTCP Feedback Image Control> July 6, 2015
1 Introduction
4K Ultra HD technology is by itself a very new trend in the overall
electronics landscape, and the impact of it is growing month by
month. The new resolution format itself is slowly starting to remake
perceptions of where the entire visual media industry will be going
over the next few years.
However, there are not many 4K display devices out yet, and 99.9% of
all final visual media productions are only 1080p HD. And sometimes
due to the bandwidth limitation, application providers are unable to
provide real 4K services to end users.
This document introduces a method to let receivers control the part
of the full video image that they wish to watch. It specifies an
extension to the messages defined in the Audio-Visual Profile with
Feedback (AVPF), and it can be applied to any RTP video
applications.
2 Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
3. Motivation
This document considers following use cases:
1. In a real-time monitoring application, the user who watches the
monitor screen may want to get a closer view of some detail of the
video image. He will want an option to ask for a specific part of
the full picture allowing the sender to either zoom in to this
part or send part of the full picture (may be a 4K resolution) to
reduce the used bandwidth.
2. In a video conferencing scenario, one user may prefer a
detailed image of one specific person, e.g., the facial expression
of the person, while another user may be fine with the initial
image.
Changing the image can be also done by delivering the original
encoder image resolution to the receivers and changing it locally, or
by the remote side using camera zoom function. However, that's not
always feasible for all cases. In a UHD environment, the issue of
bandwidth manifests itself in all of the same processes that are in
effect for those files in 1080p resolution. Even after the 4K streams
<R. Huang, et al> Expires January 7, 2016 [Page 3]
INTERNET DRAFT <RTCP Feedback Image Control> July 6, 2015
have been optimized at the source, they'll still require at least two
to three times the bandwidth you'd need today to watch a 1080p HD
feed. And in case 2, it's also not feasible to use camera zoom
because not all the receivers want to enlarge their pictures.
In this document, a new mechanism is introduced which allows
applications to signal to the remote encoder the desired image area
based on the original one. It is useful for cases where higher
definition resolution is available at the source while only streaming
of lower definition resolution is supported due to the issues of
bandwidth or user presenting devices limitation.
The basic idea is to use a new feedback message to signal the area
that the receiver requests to enlarge. The specific area is a
rectangle indicated by the coordinate of the upper left point and the
length and the width. The unit of the coordinate is based on the
original picture sent by the sender. It can be illustrated in the
following figure 1.
2160 @(x,y) is the start point (upper left point)
|
|
| x Length
|.........@****************
| * *
| * *width
| y * *
| *****************
| .
|_________._________________________________3840
Figure 1 The illustration of enlarged image area
The motivation for using the media path instead of signaling path is
similar to [RFC5104]. Media and signaling paths are sometimes
separated in systems employing middle box which has the image control
ability. As the messages are intended for the media part rather than
the signaling part, using media path avoid the transmission across
interfaces and unnecessary control traffic between signaling and
processing. Moreover, the control handled in media path is faster
than signaling path as explained in [RFC5104].
4. RTCP Receiver Report Extensions
The new feedback messages are defined in the following subsections,
following a similar structure to that in sections 6.3 of the AVPF
specification [RFC4585].
<R. Huang, et al> Expires January 7, 2016 [Page 4]
INTERNET DRAFT <RTCP Feedback Image Control> July 6, 2015
4.1. Payload-Specific Feedback Messages
As specified by Section 6.1 of [RFC4585], Payload-Specific Feedback
Messages are identified by the RTCP packet type value PSFB (206).
The following subsections define 2 new FCI formats for the payload-
specific feedback messages to extend those defined in [RFC4585] and
[RFC5104].
4.1.1. Image Control Request (ICR)
The ICR feedback message is identified by RTCP packet type value
PT=PSFB and FMT=ICR.
[Note to RFC Editor: Please replace ICR with the IANA provided
feedback message type for this message.]
The FCI field MUST contain one ICR FCI entries.
4.1.1.1 Message Format
The content of the FCI entry for the Image Control Request is showed
in Figure 2. The length of the feedback message MUST be set to 4.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Seq nr. | x axis of start point | y axis of start |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| point | length | width |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2: Syntax of an FCI Entry in the ICN Message
Seq nr.: 8 bits
Request sequence number. The sequence number space is unique for
pairing of the SSRC of request source and the SSRC of the request
target. The sequence number SHALL be increased by 1 modulo 256 for
new command. The initial value is arbitrary. A repetition message
SHOULD NOT increase the sequence number.
x axis of start point: 14 bits
The start point is the upper left point of the rectangle requested
to be enlarged. X axis of start point is the value of the
horizontal axis of this point. A value of 16383 indicates the
point is invalid.
<R. Huang, et al> Expires January 7, 2016 [Page 5]
INTERNET DRAFT <RTCP Feedback Image Control> July 6, 2015
y axis of start point: 14 bits
Y axis of start point is the value of the vertical axis of this
point. A value of 16383 indicates the point is invalid.
length: 14 bits
Length indicates the horizontal side of the rectangle requested to
be enlarged. A value of 16383 indicates the rectangle is invalid.
width: 14 bits
Width indicates the vertical side of the rectangle requested to be
enlarged. A value of 16383 indicates the rectangle is invalid.
4.1.1.2. Semantics
A decoder can request a image control by sending a ICR message to an
encoder. If the encoder has the ability to do such kind of change, it
SHOULD take into account the received ICR messages. If one of the
values in the message is not valid, the encoder SHOULD give up the
attempt of changing and give a notice. The rectangle requested in the
ICR message may not suit the final presenting of the image quite
well. In that case, the decoder is allowed to adjust according to the
final presenting devices.
Within the common packet header for feedback messages (as defined in
section 6.1 of [RFC4585]),the "SSRC of packet sender" field indicates
the source of the request, and the "SSRC of media source" indicates
the media source required to be applied the change to.
4.1.1.3. Timing Rules
The timing follows the rules outlined in section 3 of [RFC4585]. This
request message is not time critical and SHOULD be sent using regular
RTCP timing. Only if it is known that the user interface requires
quick feedback, the message MAY be sent with early or immediate
feedback timing.
4.1.1.4 Handling of Message in Mixers and Translators
A mixer of media translator that encodes content sent to the session
participant issuing the ICR SHALL consider the request to determine
if it can fulfill it. A media translator unable to fulfill the
request MAY forward the request unchanged towards the media sender.
4.1.1.5 Remarks
<R. Huang, et al> Expires January 7, 2016 [Page 6]
INTERNET DRAFT <RTCP Feedback Image Control> July 6, 2015
None.
4.1.2 Image Control Notification (ICN)
The ICN feedback message is identified by RTCP packet type value
PT=PSFB and FMT=ICN.
[Note to RFC Editor: Please replace ICN with the IANA provided
feedback message type for this message.]
The FCI field MUST contain one ICR FCI entries.
4.1.2.1 Message Format
The content of the FCI entry for the Image Control Notification is
showed in Figure 3. The length of the feedback message MUST be set to
3.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Seq nr. | R | reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3: Syntax of an FCI Entry in the ICN Message
Seq nr.: 8 bits
The corresponding request sequence number from the ICR that
resulted in this notification.
Result (R): 2 bits
This field indicates the result that the media sender deals with
the ICR message.
R=0: the media sender applies the image change successfully.
R=1: The values of the ICR message are invalid.
R=2: The media sender does not support the image change.
R=3: Reserved.
Reserved: 22 bits
All bits SHALL be set to 0 by the sender and SHALL be ignored on
<R. Huang, et al> Expires January 7, 2016 [Page 7]
INTERNET DRAFT <RTCP Feedback Image Control> July 6, 2015
reception.
4.1.2.2 Semantics
This feedback message is used to acknowledge the reception of a ICR.
For each ICR received at the media source, a ICN message SHALL be
sent. The notification message SHALL also be sent in response to ICR
repetitions received. If the request receiver has received ICR with
several different sequence numbers from a single requestor, it SHALL
only respond to the request with the highest (modulo 265) sequence
number which may be a smaller integer value due to the wrapping of
the field.
Within the common packet header for feedback messages (as defined in
section 6.1 of [RFC4585]),the "SSRC of packet sender" field indicates
the source of the notification, and the "SSRC of media source"
indicates the media source applying the change to.
4.1.2.3. Timing Rules
The timing follows the rules outlined in section 3 of [RFC4585]. This
request message is not time critical and SHOULD be sent using regular
RTCP timing.
4.1.1.4 Handling of Message in Mixers and Translators
A mixer of media translator that acts upon a ICR SHALL send the
corresponding ICN. In cases where it needs to forward a ICR itself,
the notification message MAY need to be delayed until the ICRhas been
responded to.
4.1.1.5 Remarks
None.
5 Security Considerations
TBD
6 IANA Considerations
TBD
7 Acknowledgments
TBD
<R. Huang, et al> Expires January 7, 2016 [Page 8]
INTERNET DRAFT <RTCP Feedback Image Control> July 6, 2015
8 References
8.1 Normative References
[KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003.
[RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
"Extended RTP Profile for Real-time Transport Control
Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, July
2006.
[RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman,
"Codec Control Messages in the RTP Audio-Visual Profile
with Feedback (AVPF)", RFC 5104, February 2008.
8.2 Informative References
TBD
Authors' Addresses
Rachel Huang
Huawei
101 Software Avenue, Yuhua District
Nanjing 210012
China
Email: rachel.huang@huawei.com
Roni Even
Huawei
14 David Hamelech
Tel Aviv 64953
Israel
Email: roni.even@mail01.huawei.com
<R. Huang, et al> Expires January 7, 2016 [Page 9]
