Internet DRAFT - draft-realvnc-rtcp-codec
draft-realvnc-rtcp-codec
Internet Engineering Task Force N. Wilson, Ed.
Internet-Draft RealVNC Ltd
Intended status: Informational September 24, 2018
Expires: March 28, 2019
Codec-Layer Feedback for the Real-time Transport Control Protocol (RTCP)
draft-realvnc-rtcp-codec-00
Abstract
This document specifies an extension to the messages defined in the
Audio-Visual Profile with Feedback (AVPF), allowing for an RTP codec
to send codec-specific feedback via RTCP. Currently, RTCP feedback
messages have been defined only for generic capabilities which may be
defined in the context of individual codecs, and some codecs have
their own RTCP feedback messages. There is a need for a feedback
message at the codec-layer, allowing other codecs to send codec-
specific feedback.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on March 28, 2019.
Copyright Notice
Copyright (c) 2018 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
Wilson Expires March 28, 2019 [Page 1]
Internet-Draft Codec-Layer Feedback for RTCP September 2018
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1. Glossary . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2. Requirements Language . . . . . . . . . . . . . . . . . . 3
3. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 3
4. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . 4
5. Format of RTCP Feedback Message . . . . . . . . . . . . . . . 5
6. Semantics of the RTCP Feedback Message . . . . . . . . . . . 5
7. SDP Definition . . . . . . . . . . . . . . . . . . . . . . . 6
7.1. Extension of the rtcp-fb attribute . . . . . . . . . . . 6
7.2. Offer/Answer . . . . . . . . . . . . . . . . . . . . . . 7
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7
9. Security Considerations . . . . . . . . . . . . . . . . . . . 8
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 8
10.1. Normative References . . . . . . . . . . . . . . . . . . 8
10.2. Informative References . . . . . . . . . . . . . . . . . 9
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 9
1. Introduction
The Extended RTP Profile for RTCP-Based Feedback (RTP/AVPF,
[RFC4585]) introduced the concept of feedback messages, which may be
used by RTP peers to exchange information controlling an RTP stream.
The framework provided by RTP/AVPF divides such feedback into
Transport-Layer (RTPFB) and Payload-Specific feedback (PSFB)
messages. These include generic capabilities such as "Slice Loss
Indication" (SLI), which are widely applicable to many different
codecs, yet require interpretation in the context of each codec,
since the notion of "slice" is codec-specific. Since then, further
Codec-Control messages have been defined in the same framework, which
further expand the generic messages defined for the RTP/AVPF control
layer, for example "Full Intra Request" from [RFC5104], which follows
the same model as SLI.
However, these generic messages do not enable truly codec-specific
feedback to be exchanged. There is currently no payload-specific
feedback number assigned for codec-defined feedback which is not
covered by the existing mechanisms (SLI, FIR, and others). Some
codecs have resorted to defining their own top-level feedback
messages (such as the H.271 Video Back Channel Message, [RFC5104]),
yet the space of assigned numbers for PSFB messages is scarce, and it
Wilson Expires March 28, 2019 [Page 2]
Internet-Draft Codec-Layer Feedback for RTCP September 2018
is not reasonable to allocate further numbers for messages that are
strongly tied to specific codecs.
Some applications may make use of the Application-Layer Feedback
capability from [RFC4585], however this is unsatisfactory for
widespread use, since a codec may be used by more than one
application, at which point collisions may occur in the use of the
application-layer feedback.
This document therefore requests that a PSFB number be reserved for
codec-layer feedback. Such feedback is specific to an individual
codec, as identified by the payload type identifier carried in the
feedback message. The contents and semantics of such a feedback
message are entirely defined by the codec, in the same way that
codecs have free control over the contents of the RTP payload.
2. Definitions
2.1. Glossary
AVPF - The extended RTP profile for RTCP-based feedback
CCM - Codec Control Message [RFC5104]
CLF - Codec-Layer Feedback
FCI - Feedback Control Information [RFC4585]
FIR - Full Intra Request
PSFB - Payload-Specific Feedback [RFC4585]
SLI - Slice Loss Indication
2.2. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
3. Use Cases
We briefly outline here some use-cases for codec-specific feedback
messages. In each case, there is no clear existing RTCP message
which may be used.
Codec-specific quality control. Some codecs may wish to allow
control of the quality level via feedback from the receiver. Such
Wilson Expires March 28, 2019 [Page 3]
Internet-Draft Codec-Layer Feedback for RTCP September 2018
quality control would not fit well into a generic message, since
the codec's control parameters (quantization tables, dithering
level, text quality threshold) may differ widely from the control
parameters for other codecs. The ability for the codec to define
its own PSFB messages, without requiring a codec-specific
allocation of a new PSFB message, would be helpful.
Codec-specific picture control. Some codecs may wish to allow the
receiver to specify to the sender a preferred color palette to use
for the transmitted picture, perhaps to ensure the image is usable
in a constrained viewing environment. Different codecs however
encode color data in different ways (sRGB, YUV, YDbDr, and many
more). In the case of a codec designed to support palettized
encodings for constrained viewing environments, it would be
helpful for the codec to be able to define its own feedback
messages to provide such a capability.
Codec-specific decoder control. There are codecs in which the
decoder requires access to certain control data in order to begin
decoding, for example the RTP embedding of H.264 [RFC6184] has
special considerations in the transmission of Picture Parameter
Set (PPS) NALUs, with the recommendation that these units be
exchanged reliably over an out-of-band channel (such as SDP
exchange). However, for other codecs, the decoder control data
may be less easily exchanged over such a channel: either because
it is larger in size, or changes more frequently, or may be re-
ordered with respect to the RTP stream (for codecs where the
decoder does not need to block decoding until the very latest
control information is received). In these cases, it may be
preferable to exchange the decoder-control data over RTCP, so that
it does not participate in the generic RTP control such as
retransmissions. For a codec with specific decoder-control data,
it may be helpful to enable the codec to make use of RTCP as well
as RTP messages.
4. Protocol Overview
This document extends the RTCP feedback messages defined in the RTP/
AVPF [RFC4585] by defining an RTCP Codec-Layer Feedback (CLF)
message. The RTCP CLF message contains a header identifying the
codec by means of the payload type identifier, and the remaining
contents and semantics of the message are defined by the codec.
A CLF RTCP message may be sent from the sender to the receiver, or
from the receiver to the sender, depending on the codec.
Wilson Expires March 28, 2019 [Page 4]
Internet-Draft Codec-Layer Feedback for RTCP September 2018
5. Format of RTCP Feedback Message
As specified by section 6.1 of [RFC4585], Payload-Specific FB
messages are identified by the RTCP packet type value PSFB (206).
This memo defines the CLF message, which is identified by RTCP packet
type values PT=PSFB and FMT=XXX.
The Feedback Control Information (FCI) for the CLF is of variable-
length, as determined by the codec which defines the CLF message. At
least four octets of FCI must be provided. The length of the CLF
feedback message MUST be set to 2+N/4, where N is the length in
octets of the CLF message.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|P| Payload Type| CLF data, defined by the codec... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | Padding (opt) | Padding count |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Syntax of the Codec-Layer Feedback (CLF) message
P: 1 bit
Indicates whether the CLF message is padded. All CLF messages
must be a multiple of 4 octets long; if the P bit is set to 1,
then the final octet contains a padding count of between 1 and 3,
indicating the number of final padding octets in the message
(including the padding count).
Payload Type: 7 bits
Indicates the RTP payload type in the context of which the native
CLF data MUST be interpreted.
6. Semantics of the RTCP Feedback Message
Within the common packet header for feedback messages (as defined in
Section 6.1 of [RFC4585]), the "SSRC of packet sender" field
indicates the source of the feedback message, and the "SSRC of media
source" indicates the SSRC of the sender end of the stream.
A CLF message MUST NOT be used for codecs that do not define the
contents and semantics of the CLF message.
A CLF message MUST NOT be sent in a direction which is not defined by
the codec: a CLF message MUST NOT be sent by the sender to the
receiver unless the codec defines the CLF message in that direction,
Wilson Expires March 28, 2019 [Page 5]
Internet-Draft Codec-Layer Feedback for RTCP September 2018
and a CLF message MUST NOT be sent by the receiver to the sender
unless the codec defines the CLF message in that direction.
CLF messages are sent according to timing rules defined by the codec.
A mixer or translator MAY choose to forward the message, but SHOULD
ensure that such forwarded messages are not sent at a rate greater
than the receiver's bandwidth (for example, a mixer which receives
feedback messages from a unicast peer, and forwards those messages to
a multicast session may be forced to drop packets); such a mixer
SHOULD understand the semantics of the CLF message for the payload
being forwarded, to ensure that the dropped packets do not prevent
correct operation of the codec. A mixer SHOULD NOT drop CLF packets
which it does not understand. A mixer MAY choose not to forward CLF
messages at all, by declaring that it does not support CLF messages
(for example, by using an SDP declaration to state that it does not
accept CLF messages for the given codec).
Since mixers may not support CLF messages for any particular codec,
and may not support CLF messages at all, codecs SHOULD NOT mandate
that CLF support is required for the correct operation of the codec.
7. SDP Definition
Section 4 of [RFC4585] defines a new SDP [RFC4566] attribute, rtcp-
fb, that may be used to negotiate the capability to handle specific
AVPF commands and indications. The ABNF for rtcp-fb is described in
section 4.2 of [RFC4585]. In this section, we extend the rtcp-fb
attribute to include the CLF message.
7.1. Extension of the rtcp-fb attribute
AVPF specifies that the rtcp-fb attribute must only be used as a
media level attribute and must not be provided at session level. All
the rules described in [RFC4585] for rtcp-fb attribute relating to
payload type and to multiple rtcp-fb attributes in a session
description also apply to the new feedback message defined in this
memo.
In [RFC5104], a feedback value "ccm" is defined, which indicates the
support of codec control using RTCP feedback messages. The "ccm"
feedback value is used with a new parameter defined in this memo,
"clf", to indicate support for Codec-Layer Feedback (CLF).
In the ABNF for rtcp-fb-ccm-param defined in [RFC5104], there is a
placeholder token to allow defining new CCM message parameters.
"clf" is defined as a new CCM parameter in this document.
Wilson Expires March 28, 2019 [Page 6]
Internet-Draft Codec-Layer Feedback for RTCP September 2018
Inclusion of the CLF message type in an SDP description indicates
support for sending and receiving the CLF message, depending on
whether the codec defines the messages in each direction.
The "clf" parameter SHOULD NOT be used with rtcp-fb-pt set to "*", to
indicate support for the CLF message for all codecs; instead support
for CLF messages SHOULD be declared for each supported codec
individually.
For example, to indicate support for receiving CLF messages, the
sender may declare the following in SDP:
v=0
o=alice 12345 12345 IN IP4 host.example.com
s=Media with CLF feedback
t=0 0
c=IN IP4 host.example.com
m=video 23456 RTP/AVPF 98
a=rtpmap:98 urn:codec-name/90000
a=rtcp-fb:98 ccm clf
7.2. Offer/Answer
The Offer/Answer [RFC3264] implications for codec layer feedback
feedback messages are similar to those described in [RFC5104]. The
offerer MAY indicate the capability to support selected CLF messages.
The answerer MUST remove all CLF parameters corresponding to the CLF
message and codec combinations that it does not wish to support in
this particular media session (for example, because it does not
implement the CLF message for the codec in question). The answerer
MUST NOT add new clf parameters in addition to what has been offered.
As in [RFC5104], only if CLF is jointly declared for a codec by both
offer and the answer, may CLF messages be used.
Including a CLF parameter in an offer or answer indicates that the
party (offerer or answerer) is at least capable of receiving the
corresponding CLF message and acting upon it. It is not mandated to
send CLF messages of any negotiated type, unless the acknowledgement
or response is a mandatory part of receiving the CLF message.
8. IANA Considerations
A new value "clf" must be registered with IANA in the "Codec Control
Messages" registry, located at time of publication at:
http://www.iana.org/assignments/sdp-parameters
The new value is as follows:
Wilson Expires March 28, 2019 [Page 7]
Internet-Draft Codec-Layer Feedback for RTCP September 2018
Value name: clf
Long name: Codec-Layer Feedback
Usable with: ccm
Reference: RFC-realvnc-codec-layer-feedback
The following value must be registered as a FMT value in the "FMT
Values for PSFB Payload Types" registry located at the time of
publication at: http://www.iana.org/assignments/rtp-parameters
PSFB range
Name Long Name Value Reference
---- -------------------- ----- ----------------------------------
CLF Codec Layer Feedback XXX [RFC-realvnc-codec-layer-feedback]
9. Security Considerations
The CLF message is defined within the context of the existing
security model for RTP/RTCP sessions. As such, it does not enable
significant changes to the existing security model.
Since the behaviour of CLF messages is undefined by this
specification, detailed considerations must be provided by each codec
specification which defines such a message.
To guard against malicious injection of CLF data, the CLF messages
should be treated as untrusted external data, in the same way that an
RTP payload itself would be. Authentication and integrity checks may
be used where appropriate to ensure that third-parties may not inject
or modify CLF messages, for example using SRTP and the SAVFP profile
to provide security.
The CLF message format itself is not believed to carry security risks
additional to those already incurred by RTP/RTCP peers.
10. References
10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
with Session Description Protocol (SDP)", RFC 3264,
DOI 10.17487/RFC3264, June 2002,
<https://www.rfc-editor.org/info/rfc3264>.
Wilson Expires March 28, 2019 [Page 8]
Internet-Draft Codec-Layer Feedback for RTCP September 2018
[RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, DOI 10.17487/RFC4566,
July 2006, <https://www.rfc-editor.org/info/rfc4566>.
[RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
"Extended RTP Profile for Real-time Transport Control
Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585,
DOI 10.17487/RFC4585, July 2006,
<https://www.rfc-editor.org/info/rfc4585>.
[RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman,
"Codec Control Messages in the RTP Audio-Visual Profile
with Feedback (AVPF)", RFC 5104, DOI 10.17487/RFC5104,
February 2008, <https://www.rfc-editor.org/info/rfc5104>.
10.2. Informative References
[RFC6184] Wang, Y., Even, R., Kristensen, T., and R. Jesup, "RTP
Payload Format for H.264 Video", RFC 6184,
DOI 10.17487/RFC6184, May 2011,
<https://www.rfc-editor.org/info/rfc6184>.
Author's Address
Nicholas Wilson (editor)
RealVNC Ltd
Betjeman House, 104 Hills Road
Cambridge CB2 1LQ
UK
Phone: +44 1223 310411
Email: ncw@realvnc.com
Wilson Expires March 28, 2019 [Page 9]
