| Network Working Group | J. Lennox |
| Internet-Draft | Vidyo |
| Intended status: Informational | K. Gross |
| Expires: January 16, 2014 | AVA |
| S. Nandakumar | |
| G. Salgueiro | |
| Cisco Systems | |
| B. Burman | |
| Ericsson | |
| July 15, 2013 |
A Taxonomy of Grouping Semantics and Mechanisms for Real-Time Transport Protocol (RTP) Sources
draft-lennox-raiarea-rtp-grouping-taxonomy-01
The terminology about, and associations among, Real-Time Transport Protocol (RTP) sources can be complex and somewhat opaque. This document describes a number of existing and proposed relationships among RTP sources, and attempts to define common terminology for discussing protocol entities and their relationships.
This document is still very rough, but is submitted in the hopes of making future discussion productive.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 16, 2014.
Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
The existing taxonomy of sources in RTP is often regarded as confusing and inconsistent. Consequently, a deep understanding of how the different terms relate to each other becomes a real challenge. Frequently cited examples of this confusion are (1) how different protocols that make use of RTP use the same terms to signify different things and (2) how the complexities addressed at one layer are often glossed over or ignored at another.
This document attempts to provide some clarity by reviewing the semantics of various aspects of sources in RTP. As an organizing mechanism, it approaches this by describing various ways that RTP sources can be grouped and associated together.
This section defines concepts that serve to identify various components in a given RTP usage. For each concept an attempt is made to list any alternate definitions and usages that co-exist today along with various characteristics that further describes the concept.
All references to ControLling mUltiple streams for tElepresence (CLUE) in this document map to [I-D.ietf-clue-framework] and all references to Web Real-Time Communications (WebRTC) map to [I-D.ietf-rtcweb-overview].
A single entity sending or receiving RTP packets. It may be decomposed into several functional blocks, but as long as it behaves as a single RTP stack entity it is classified as a single "End Point".
The CLUE Working Group (WG) uses the terms "Media Provider" and "Media Consumer" to describes aspects of End Point pertaining to sending and receiving functionalities.
End Points can be identified in several different ways. While RTCP Canonical Names (CNAMEs) [RFC3550] provide a globally unique and stable identification mechanism for the duration of the Communication Session (See Section 2.12), their validity applies exclusively within a synchronization context. Therefore, a mechanisms outside the scope of RTP, such as an application defined mechanisms, must be depended upon to ensure End Point identification when outside this synchronization context.
The physical source of stream of media data of one type such as camera or microphone.
The CLUE WG uses the term "Capture Device" to identify a physical capture device.
WebRTC WG uses the term "Recording Device" to refer to the locally available capture devices in an end-system.
A Media Source logically defines the source of a raw stream of media data as generated either by a single capture device or by a conceptual source. A Media Source represents an Audio Source or a Video Source.
The CLUE WG uses the term "Media Capture" for this purpose. A CLUE Media Capture is identified via indexed notation. The terms Audio Capture and Video Capture are used to identify Audio Sources and Video Sources respectively. Concepts such as "Capture Scene", "Capture Scene Entry" and "Capture" provide a flexible framework to represent media captured spanning spatial regions.
The WebRTC WG defines the term "RtcMediaStreamTrack" to refer to a Media Source. An "RtcMediaStreamTrack" is identified by the ID attribute on it.
Typically a Media Source is mapped to a single m=line via the Session Description Protocol (SDP) [RFC4566] unless mechanisms such as Source-Specific attributes are in place [RFC5576]. In the latter cases, an m=line can represent either multiple Media Sources or multiple Media Streams (See Section 2.4 for a definition of this term).
Media from a Media Source is encoded and packetized to produce one or more Media Streams representing a sequence of RTP packets.
The term "Stream" is used by the CLUE WG to define a encoded Media Source sent via RTP. "Capture Encoding", "Encoding Groups" are defined to capture specific details of the encoding scheme.
RFC3550 [RFC3550] uses the term Source for this purpose.
The equivalent mapping of Media Stream in SDP [RFC4566] is defined per usage. For example, each m=line can describe one Media Stream and hence one Media Source OR a single m=line can describe properties for multiple Media Streams (via [RFC5576] mechanisms for example).
A Media Provider is a logical component within the RTP Stack that is responsible for encoding the media data from one or more Media Sources to generate RTP Payload for the outbound Media Streams.
Within the SDP usage, an m=line describes the necessary configuration required for encoding purposes.
CLUE's "Capture Encoding" provides specific encoding configuration for this purpose.
WebRTC WG uses the term "RtcMediaStreamTrack" to qualify as source of the media data that is encoded via the Media Provider.
An RTP session is an association among a group of participants communicating with RTP. It is a group communications channel which can potentially carry a number of Media Streams. Within an RTP session, every participant finds out meta-data and control information (over RTCP) about all the Media Streams in the RTP session. The bandwidth of the RTCP control channel is shared within an RTP Session.
Within the context of SDP a singe m=line can map to a single RTP Session or multiple m=lines can map to a single RTP Session. The latter is enabled via multiplexing schemes such as BUNDLE [I-D.ietf-mmusic-sdp-bundle-negotiation], for example, that allows mapping of multiple m=lines to a single RTP Session.
A Media Transport defines an end-to-end transport association for carrying one or more RTP Sessions. The combination of a network address and port uniquely identifies such a transport association, for example an IP address and a UDP port.
Represents a physical rendering device such display or speaker.
A Media Renderer is a logical component within the RTP Stack that is responsible for decoding the RTP Payload within the incoming Media Streams to generate media data suitable for eventual rendering.
Within the context of SDP, an m=line describes the necessary configuration required to decode either one or more incoming Media Streams.
The WebRTC WG uses the term "RtcMediaStreamTrack" to qualify the media data decoded via the Media Renderer corresponding to the incoming Media Stream.
A participant is an entity reachable by a single signaling address, and is thus related more to the signaling context than to the media context.
A multimedia session is an association among a group of participants engaged in the conversation via one or more RTP Sessions. It defines logical relationships among Media Sources that appear in multiple RTP Sessions.
RFC4566 [RFC4566] defines a multimedia session as a set of multimedia senders and receivers and the data streams flowing from senders to receivers.
RFC3550 [RFC3550] defines it as set of concurrent RTP sessions among a common group of participants. For example, a videoconference (which is a multimedia session) may contain an audio RTP session and a video RTP session.
A communication session is an association among group of participants communicating with each other via a set of multimedia sessions.
The Session Description Protocol RFC4566 [RFC4566]defines a multimedia session as a set of multimedia senders and receivers and the data streams flowing from senders to receivers. In that definition it is however not clear if a multimedia session includes both the sender's and the receiver's view of the same RTP Stream.
For example, in a full mesh communication, the Communication Session consists of a set of separate Multimedia Sessions between each pair of Participants. Another example is a centralized conference, where the Communication Session consists of a set of Multimedia Sessions between each Participant and the conference handler.
This section provides various relationships that can co-exist between the aforementioned concepts in a given RTP usage. Using Unified Modeling Language (UML) class diagrams [UML], Figure 1 below depicts general relations between a Media Source, its Media Provider(s) and the resulting Media Stream(s).
+--------------+ <<uses>> +-------------------------+
| Media Source |- - - - - ->| Synchronization Context |
+--------------+ +-------------------------+
< > 1..*
|
| 0..*
+--------------+
| |<>-+ 0..*
| Media | |
| Provider | |
| |---+ 0..*
+--------------+
< > 1
|
| 0..*
+----------------+ 0..* 1 +-------------+
| Media Stream |----------<>| RTP Session |
+----------------+ +-------------+
Figure 1: Media Source Relations
Media sources can have a large variety of relationships among them. These relationships can apply both between sources within a single RTP Session, and between Media Sources that occur in multiple RTP Session. Ways of relating them typically involve groups: a set of Media Sources has some relationship that applies to all those in the group, and no others. (Relationships that involve arbitrary non-grouping associations among Media sources, such that e.g., A relates to B and B to C, but A and C are unrelated, are uncommon if not nonexistent.) In many cases, the semantics of groups are not simply that the the members form an undifferentiated group, but rather that members of the group have certain roles.
A synchronization context defines requirement on a strong timing relationship between the related entities, typically requiring alignment of clock sources. Such relationship can be identified in multiple ways as listed below. A single Media Source can only belong to a single Synchronization Context, since it is assumed that a single Media Source can only have a single media clock and requiring alignment to several Synchronization Contexts will effectively merge those into a single Synchronization Context.
A single Multimedia session can contain media from one or more Synchronization Contexts. An example of that is a Multimedia Session containing one set of audio and video for communication purposes belonging to one Synchronization context, and another set of audio and video for presentation purposes (like playing a video file) that has no strong timing relationship and need not be strictly synchronized with the audio and video used for communication.
RFC3550 [RFC3550] describes Inter-media synchronization between RTP Sessions based on RTCP CNAME, RTP and Network Time Protocol (NTP) [RFC5905] timestamps.
[I-D.ietf-avtcore-clksrc] provides a mechanism to signal the clock source in SDP, thus allowing a synchronized context to be defined.
In CLUE "Capture Scene", "Capture Scene Entry" and "Captures" define an implied synchronization context.
The WebRTC WG defines "RtcMediaStream" with one or more "RtcMediaStreamTracks". All tracks in a "RTCMediaStream" are intended to be synchronized when rendered.
RFC5888 [RFC5888] defines m=line grouping mechanism called "Lip Synchronization (LS)" for establishing the synchronization requirement across m=lines when they map to individual sources.
RFC5576 [RFC5576] extends the above mechanism when multiple media sources are described by a single m=line.
A containment relationship allows composing of multiple concepts into a larger concept.
Multiple Media Streams can be contained within a single RTP Session via unique SSRC per Media Stream. [I-D.ietf-mmusic-sdp-bundle-negotiation] provides SDP based signaling mechanism to enable this across several m=lines.
RFC5576 [RFC5576] enables the same for multiple Media Sources described in a single m=line.
[I-D.westerlund-avtcore-transport-multiplexing], for example, describes a mechanism that allow several RTP Sessions to be carried over a single underlying Media Transport.
The WebRTC WG defines a containment object named "RTCPeerConnection" that can potentially contain several Media Sources mapped to a single RTP Session or spread across several RTP Sessions.
In this relationship different instances of a concept are treated to be equivalent for the purposes of relating them to the Media Source.
Figure 2 below depicts in UML notation the general relation between a Media Provider and its Media Stream(s), including the Media Stream specializations Source Stream and RTP Repair Stream.
+--------------+
| |<>-+ 0..*
| Media | |
| Provider | |
| |---+ 0..*
+--------------+
< > 1
|
| 0..*
+--------------+ 0..* 1 +-----------------+
| Media Stream |<>-------| Media Transport |
+--------------+ +-----------------+
/\ /\
+--+ +--+
| |
+-------+ +-------+
| |
+--------------+ +--------------+ 1
| Primary |<>----------| Repair |<>-+
| Stream | 1..* 0..* | Stream |---+
+--------------+ +--------------+ 0..*
Figure 2: Media Stream Relations
This relation can in combination with Figure 1 be used to achieve a set of functionalities, described below.
A Media Source represented as multiple independent Encodings constitutes a simulcast of that Media Source. The figure below represents an example of a Media Source that is encoded into three separate simulcast streams that are in turn sent on the same transport flow.
+----------------+
| Media Source |
+----------------+
< > < > < >
| | |
+------------+ | +--------------+
| | |
+----------------+ +----------------+ +----------------+
| Media Provider | | Media Provider | | Media Provider |
+----------------+ +----------------+ +----------------+
< > < > < >
| | |
| | |
+----------------+ +----------------+ +----------------+
| Media Stream | | Media Stream | | Media Stream |
+----------------+ +----------------+ +----------------+
< > < > < >
| | |
+---------------+ | +----------------+
| | |
+-------------------+
| Media Transport |
+-------------------+
Figure 3: Example of Media Source Simulcast
Multi-stream transmission (MST) is a mechanism by which different portions of a layered encoding of a media stream are sent using separate Media Streams (sometimes in separate RTP sessions). MSTs are useful for receiver control of layered media.
A Media Source represented as multiple dependent Encodings constitutes a Media Source that has layered dependency. The figure below represents an example of a Media Source that is encoded into three dependent layers, where two layers are sent on the same transport flow and the third layer is sent on a separate transport flow.
+----------------+
| Media Source |
+----------------+
< > < > < >
| | |
+--------------+ | +--------------+
| | |
+----------------+ +----------------+ +---------------+
| Media Provider |<>-| Media Provider |<>-| Media Provider|
+----------------+ +----------------+ +---------------+
< > < > < >
| | |
| | |
+----------------+ +----------------+ +----------------+
| Media Stream | | Media Stream | | Media Stream |
+----------------+ +----------------+ +----------------+
< > < > < >
| | |
+------+ +------+ |
| | |
+-----------------+ +-----------------+
| Media Transport | | Media Transport |
+-----------------+ +-----------------+
Figure 4: Example of Media Source Layered Dependency
A Media Source may be protected by repair streams during transport. Several approaches listed below can achieve the same result
The figure below represents an example where a Media Source is protected by a retransmission (RTX) flow. In this example the primary Media Stream and the RTP RTX Stream share the same Media Transport.
+----------------+
| Media Source |
+----------------+
< >
|
+----------------+
| Media Provider |
+----------------+
< >
|
+---------------+ +-----------+
| Primary Media |<>-| RTX Media |
| Stream | | Stream |
+---------------+ +-----------+
< > < >
| |
+------+ +------+
| |
+-----------------+
| Media Transport |
+-----------------+
Figure 5: Example of Media Source Retransmission Flows
The figure below represents an example where two Media Sources are protected by individual FEC flows as well as one additional FEC flow that protects the set of both Media Sources (a FEC group). There are several possible ways to map those Media Streams to one or more Media Transport, but that is omitted from the figure for clarity.
+----------+ +----------+
| Media | | Media |
| Source | | Source |
+----------+ +----------+
< > < >
| |
+----------+ +----------+
| Media | | Media |
| Provider | | Provider |
+----------+ +----------+
< > +-------------------+ +-------------------+ < >
| | | | | |
| | < > < > | |
+---------+ +--------+ +--------+ +--------+ +---------+
| Primary | | RTP | | RTP | | RTP | | Primary |
| Media |<>-| FEC |-<>| FEC |<>-| FEC |-<>| Media |
| Stream | | Stream | | Stream | | Stream | | Stream |
+---------+ +--------+ +--------+ +--------+ +---------+
Figure 6: Example of Media Source FEC Flows
RFC5888 [RFC5888] defines m=line grouping mechanism called "FID" for establishing the equivalence of Media Streams across the m=lines under grouping.
RFC5576 [RFC5576] extends the above mechanism when multiple media sources are described by a single m=line.
There are different ways to construct a Communication Session. The general relation in UML notation between a Communication Session, Participants, Multimedia Sessions and RTP Sessions is outlined below.
+---------------+
| Communication |
| Session |
+---------------+
0..* < > < > 1..*
| |
+----------+ +--------+
1..* | | 1..*
+-------------+ 1 0..* +--------------------+
| Participant |<>----------| Multimedia Session |
+-------------+ +--------------------+
< > 1 < > 1
| | 0..*
| +-------------+
| | RTP Session |
| +-------------+
| < > 1
| 0..* | 0..*
+-----------------+ 1 0..* +--------------+
| Media Transport |--------<>| Media Stream |
+-----------------+ +--------------+
Figure 7: Session Relations
Several different flavors of Session can be possible. A few typical examples are listed in the below sub-sections, but many other are possible to construct.
In this example, a single Multimedia Session is shared between the two Participants. That Multimedia Session contains a single RTP Session with two Media Streams from each Participant. Each Participant has only a single Media Transport, carrying those Media Streams, which is the main reason why there is only a single RTP Session.
+----------------+
| Point-to-Point |
| Session |
+----------------+
< > < > < >
| | |
+------------------------+ | +------------------------+
| | |
+-------------+ +--------------------+ +-------------+
| Participant |<>----------| Multimedia Session |----------<>| Participant |
+-------------+ +--------------------+ +-------------+
< > < > < >
| | |
| +--------------+ +-------------+ +--------------+ |
| | Media Stream |----<>| RTP Session |<>----| Media Stream | |
| +--------------+ +-------------+ +--------------+ |
| < > < > < > < > |
| | | | | |
+-----------------+ +--------------+ +--------------+ +-----------------+
| Media Transport |-<>| Media Stream | | Media Stream |<>-| Media Transport |
+-----------------+ +--------------+ +--------------+ +-----------------+
Figure 8: Example Point-to-Point Session
In this example, the Full Mesh Session has three Participants, each of which has the same characteristics as the example in the previous section; a single Media Transport per peer Participant, resulting in a single RTP session between each pair of Participants.
+-----------+ +-------------+ +-----------+
| Media |----------------<>| Participant |<>---------------| Media |
| Transport | +-------------+ | Transport |
+-----------+ | +-----------+
| | +------------+ | +------------+ | |
< > < > | Multimedia | | | Multimedia | < > < >
+--------++--------+ | Session | | | Session | +--------++--------+
| Media || Media | +------------+ | +------------+ | Media || Media |
| Stream || Stream | < > | | | < > | Stream || Stream |
+--------++--------+ | | | | | +--------++--------+
| | | | | | | | |
| < > | < > < > < > | < > |
| +---------+ +---------------+ +---------+ |
+-------<>| RTP | | Full Mesh | | RTP |<>------+
+-------<>| Session | | Session | | Session |<>------+
| +---------+ +---------------+ +---------+ |
| < > < > < > < > < > |
| | | | | | |
+--------++--------+ | | | +--------++--------+
| Media || Media | | | | | Media || Media |
| Stream || Stream | | | | | Stream || Stream |
+--------++--------+ | | | +--------++--------+
< > < > | | | < > < >
| | | | | | |
+-----------+ | | | +-----------+
| Media | | | | | Media |
| Transport | | | | | Transport |
+-----------+ +-----------------+ | +-----------------+ +-----------+
| | |
+-------------+ +--------------------+ +-------------+
| Participant |<>-----------| Multimedia Session |----------<>| Participant |
+-------------+ +--------------------+ +-------------+
< > < > < >
| | |
| +--------+ +---------+ +--------+ |
| | Media |----------<>| RTP |<>----------| Media | |
| | Stream | | Session | | Stream | |
| +--------+ +---------+ +--------+ |
| < > < > < > < > |
| | | | | |
+-----------+ +--------+ +--------+ +-----------+
| Media |---------<>| Media | | Media |<>---------| Media |
| Transport | | Stream | | Stream | | Transport |
+-----------+ +--------+ +--------+ +-----------+
Figure 9: Example Full Mesh Session
Text to be provided
TBD
Figure 10: Example Centralized Conference Session
This document simply tries to clarify the confusion prevalent in RTP taxonomy because of inconsistent usage by multiple technologies and protocols making use of the RTP protocol. It does not introduce any new security considerations beyond those already well documented in the RTP protocol [RFC3550] and each of the many respective specifications of the various protocols making use of it.
Hopefully having a well-defined common terminology and understanding of the complexities of the RTP architecture will help lead us to better standards, avoiding security problems.
This document has many concepts borrowed from several documents such as WebRTC [I-D.ietf-rtcweb-overview], CLUE [I-D.ietf-clue-framework], Multiplexing Architecture [I-D.westerlund-avtcore-transport-multiplexing]. The authors would like to thank all the authors of each of those documents.
The authors would also like to acknowledge the insights, guidance and contributions of Magnus Westerlund, Roni Even, Colin Perkins, Keith Drage, and Harald Alvestrand.
Much of the terminology is still a matter of dispute.
It might be useful to distinguish between a single endpoint's view of a source, or RTP session, or multimedia session, versus the full set of sessions and every endpoint that's communicating in them, with the signaling that established them.
(Sure to be many more...)
This document makes no request of IANA.
| [RFC3550] | Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003. |
| [UML] | Object Management Group, "OMG Unified Modeling Language (OMG UML), Superstructure, V2.2", OMG formal/2009-02-02, February 2009. |
NOTE TO RFC EDITOR: Please remove this section prior to publication.