| Network Working Group | E. Ivov |
| Internet-Draft | Jitsi |
| Intended status: Standards Track | E. Marocco |
| Expires: December 19, 2013 | Telecom Italia |
| P. Thatcher | |
| June 17, 2013 |
No Plan: Economical Use of the Offer/Answer Model in WebRTC Sessions with Multiple Media Sources
draft-ivov-rtcweb-noplan-01
This document describes a model for the lightweight use of SDP Offer/Answer in WebRTC. The goal is to minimize reliance on Offer/Answer exchanges in a WebRTC session and provide applications with the tools necessary to implement the signalling that they may need in a way that best fits their custom requirements and topologies. This simplifies signalling of multiple media sources or providing RTP Synchronisation source (SSRC) identification in multi-party sessions. Another important goal of this model is to remove from clients topological constraints such as the requirement to know in advance all SSRC identifiers that they could potentially introduce in a particular session.
The model described here is similar to the one employed by the data channel JavaScript APIs in WebRTC, where methods are supported on PeerConnection without being reflected in SDP.
This document does not question the use of SDP and the Offer/Answer model or the value they have in terms of interoperability with legacy or other non-WebRTC devices.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on December 19, 2013.
Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
In its early stages the RTCWEB working group chose to use the Session Description Protocol (SDP) and the Offer/Answer model [RFC3264] when establishing and negotiating sessions. This choice was also accompanied by the decision not to mandate a specific signalling protocol so that, once interoperability has been achieved, web applications can choose the semantics that best fit their requirements. In some scenarios however, such as those involving the use of multiple media sources, these choices have left open the issue of exactly which operations should be handled by SDP Offer/Answer and which of them should be left to application-specific signalling.
At the time of writing of this document, the RTCWEB working group is considering two approaches to addressing the issue, that are often referred to as Plan A [PlanA] and Plan B [PlanB]. Both of them describe semantics that require Offer/Answer exchanges in a number of situations where this could be avoided, particularly when adding or removing media sources to a session. This requirement applies equally to cases where a client adds the stream of a newly activated web cam, a simulcast flow or upon the arrival or departure of a conference participant.
Plan A handles such notifications with the addition or removal of independent m= lines [PlanA], while Plan B relies on the use of multiplexed m= lines but still depends on the Offer/Answer exchanges for the addition or removal of media stream identifiers [MSID].
By taking the Offer/Answer approach, both Plan A and Plan B take away from the application the opportunity to handle such events in a way that is most fitting for the use case, which, among other things, also goes against the working group's decision to not to define a specific signalling protocol. (It could be argued that it is therefore only natural how proponents of each plan, having different use cases in mind, are remarkably far from reaching consensus).
Reliance on preliminary announcement of SSRC identifiers is another issue. While this could be perceived as relatively straightforward in one-to-one sessions or even conference calls within controlled environments, it can be a problem in the following cases:
By increasing the number of Offer/Answer exchanges Both Plan A and Plan B also increase the risk of encountering glare situations (i.e. cases where both parties attempt to modify a session at the same time). While glare is also possible with basic Offer/Answer and resolution of such situations must be implemented anyway, the need to frequently resort to such code may either negatively impact user experience (e.g. when "back off" resolution is used) or require substantial modifications in the Offer/Answer model and/or further venturing into the land of signalling protocols [ROACH-GLARELESS-ADD].
The goal of this document is to provide directions for use of the SDP Offer/Answer model in a way that satisfies the following requirements:
To achieve the above requirements this specification expects that browsers and WebRTC endpoints in general will only use SDP Offer/Answer to establish transport channels and initialize an RTP stack and codec/processing chains. This also includes any renegotiation that requires the re-initialisation of these chains. For example, adding VP8 to a session that was setup with only H.264, would obviously still require an Offer/Answer exchange.
All other session control and signalling are to be left to applications.
The actual Offer/Answer semantics presented here do not differ fundamentally from those proposed by Plan A and Plan B. The main differentiation point of this approach is the fact that the exact protocol mechanism is left to WebRTC applications. Such applications or lightweight signalling gateways can then implement either Plan A, or Plan B, or an entirely different signalling protocol, depending on what best matches their use cases and topology.
The model presented in this specification relies on use of SDP and Offer/Answer in quite the same way as many of the pre-WebRTC (and most of the legacy) endpoints do: negotiating formats, establishing transport channels and exchanging, in a declarative way, media and transport parameters that are then used for the initialization of the corresponding stacks.
v=0
o=- 0 0 IN IP4 198.51.100.33
s=
t=0 0
a=group:BUNDLE audio video // declaring BUNDLE Support
c=IN IP4 198.51.100.33
a=ice-ufrag:Qq8o/jZwknkmXpIh // initializing ICE
a=ice-pwd:gTMACiJcZv1xdPrjfbTHL5qo
a=ice-options:trickle
a=fingerprint:sha-1 // DTLS-SRTP keying
a4:b1:97:ab:c7:12:9b:02:12:b8:47:45:df:d8:3a:97:54:08:3f:16
m=audio 5000 RTP/SAVPF 96 0 8
a=mid:audio
a=rtcp-mux
a=rtpmap:96 opus/48000/2 // PT mappings
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=extmap:1 urn:ietf:params:rtp-hdrext:csrc-audio-level //5825 header
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level //extensions
[ICE Candidates]
m=video 5002 RTP/SAVPF 97 98
a=mid:video
a=rtcp-mux
a=rtpmap:97 VP8/90000 // PT mappings and resolutions capabilities
a=imageattr:97 \
send [x=[480:16:800],y=[320:16:640],par=[1.2-1.3],q=0.6] \
[x=[176:8:208],y=[144:8:176],par=[1.2-1.3]] \
recv *
a=rtpmap:98 H264/90000
a=imageattr:98 send [x=800,y=640,sar=1.1,q=0.6] [x=480,y=320] \
recv [x=330,y=250]
a=extmap:3 urn:ietf:params:rtp-hdrext:fec-source-ssrc //5825 header
a=extmap:4 urn:ietf:params:rtp-hdrext:rtx-source-ssrc //extensions
a=max-send-ssrc:{*:1} // declaring maximum
a=max-recv-ssrc:{*:4} // number of SSRCs
[ICE Candidates]
The following is an example presenting what this specification views as a typical offer sent by a WebRTC endpoint:
Interoperating with the "widely deployed legacy endpoints" is one of the main reasons for the RTCWEB working group to choose the SDP Offer/Answer model as basis for media negotiation. It is hence important to clarify the compatibility claims that this specification makes.
A "widely deployed legacy endpoint" is considered to have the following characteristics:
While it is relatively simple for RTCWEB to accommodate some of the above, it is obviously impossible to design a model that could simply be labeled as "compatible with legacy". It is reasonable to assume that use cases involving use of such endpoints will be designed for a relatively specific set of devices and applications. The role of the WebRTC framework is to hence provide a least-common-denominator model that can then be extended by applications.
It is just as important not to make choices or assumptions that will render interoperability for some applications or topologies difficult or even impossible.
This is exactly what the use of Offer/Answer discussed here strives to achieve. Audio/Video offers originating from WebRTC endpoints will always have a maximum of one audio and one video m= line. It will be up to applications to determine exactly how many streams they can afford to send once such a session has been established. The exact mechanism to do this is outside the scope of this document (or WebRTC in general).
Note that it is still possible for WebRTC endpoints to indicate support for a maximum number of incoming or outgoing streams for reasons such as processing constraints. Use of the "max-send-ssrc" and "max-recv-ssrc" attributes [MAX-SSRC] could be one way of doing this, although that mechanism would need to be extended to provide ways of distinguishing between independent flows and complementary ones such as layered FEC and RTX. Even with this in mind it is still important, not to rely on the presence of that indication in incoming descriptions as well as to provide applications with a way of retrieving such capabilities from the WebRTC stack (e.g. the browser).
Determining whether a peer has the ability to seamlessly switch from one SSRC to another is also left to application specific signalling. It is worth noting that protocols such as SIP for example, often accompany SSRC replacements with extra signalling (re-INVITEs with a "replaces" header) that can easily be reused by applications or mapped to something that they deem more convenient.
For the sake of interoperability this specification strongly advises against the use of multiple m= lines for a single media type. Not only would such use be meaningless to a large number of legacy endpoints but it is also likely to be mishandled by many of them and to cause unexpected behaviour.
Finally, it is also worth pointing out that there is a significant number of feature rich non-WebRTC applications and devices that have relatively advanced, modern sets of capabilities. Such endpoints hardly fit the "legacy" qualification. Yet, as is often the case with novel and/or proprietary applications, they too have adopted diverse signalling mechanisms and the requirements described in this section fully apply when it comes to interoperating with them.
All of the above semantics are best handled and hence should be left to applications. There are numerous existing or emerging solutions, some of them developed by the IETF, that already cover this. This includes CLUE channels [CLUE], the SIP Event Package For Conference State [RFC4575] and its XMPP variant [COIN] as well as the protocols defined within the Centralised Conferencing IETF working group [XCON] . Additional mechanisms, undoubtedly many based on JSON, are very likely to emerge in the future as WebRTC applications address varying use cases, scenarios and topologies.
The most important part of this specification is hence to prevent certain assumptions or topologies from being imposed on applications. One example of this is the need to know and include in the Offer/Answer exchange, all the SSRCs that can show up in a session. This can be particularly problematic for scenarios that involve non-WebRTC endpoints.
Large scale conference calls, potentially federated through RTP translator-like bridges, would be another problematic scenario. Being able to always pre-announce SSRCs in such situations could of course be made to work but it would come at a price. It would either require a very high number of Offer/Answer updates that propagate the information through the entire topology, or use of tricks such as pre-allocating a range of "fake" SSRCs, announcing them to participants and then overwriting the actual SSRCs with them. Depending on the scenario both options could prove inappropriate or inefficient while some applications may not even need such information. Others could be retrieving it through simplistic means such as access to a centralized resource (e.g. an URL pointing to a JSON description of the conference).
This document assumes use of BUNDLE in WebRTC endpoints. This implies that all RTP streams are likely to end up being received on the same port. A demuxing mechanism is therefore necessary in order for these packets to then be fed into the appropriate processing chain (i.e. matched to an m= line).
This specification uses demuxing based on RTP payload types. When creating offers and answers WebRTC applications MUST therefore allocate RTP payload types only once per bundle group. In cases where rtcp-mux is in use this would mean a maximum of 96 payload types per bundle [RFC5761]. It has been pointed out that some legacy devices may have unpredictable behaviour with payload types that are outside the 96-127 range reserved by [RFC3551] for dynamic use. Some applications or implementations may therefore choose not to use values outside this range. Whatever the reason, offerers that find they need more than the available payload type numbers, will simply need to either use a second bundle group or not use BUNDLE at all (which in the case of a single audio and a single video m= line amounts to roughly the same thing). This would also imply building a dynamic table, mapping SSRCs to PTs and m= lines, in order to then also allow for RTCP demuxing.
While not desirable, the implications of such a decision would be relatively limited. Use of trickle ICE [TRICKLE-ICE] is going to lessen the impact on call establishment latency. Also, the fact that this would only occur in a limited number of cases makes it unlikely to have a significant effect on port consumption.
An additional requirement that has been expressed toward demuxing is the ability to assign incoming packets with the same payload type to different processing chains depending on their SSRCs. A possible example for this is a scenario where two video streams are being rendered on different video screens that each have their own decoding hardware.
While the above may appear as a demuxing and a decoding related problem it is really mostly a rendering policy specific to an application. As such it should be handled by app. specific signalling that could involve custom-formatted, per-SSRC information that accompanies SDP offers and answers.
From a WebRTC perspective, repair flows such as layering, FEC, RTX and to some extent simulcasting, present an interesting challenge, which is why they are considered an open issue by this specification.
On the one hand they are transport utilities that need to be understood, supported and used by browsers in a way that is mostly transparent to applications. On the other, some applications may need to be made aware of them and given the option to control their use. This could be necessary in cases where their use needs to be signalled to non-WebRTC endpoints in an application specific way. Another example is the possibility for an application to choose to disable some or all repair flows because it has been made aware by application-specific signalling that they are temporarily not being used/rendered by the remote end (e.g. because it is only displaying a thumbnail or because a corresponding video tag is not currently visible).
One way of handling such flows would be to advertise them in the way suggested by [RFC5956] and to then control them through application specific signalling. This options has the merit of already existing but it also implies the pre-announcement and propagation of SSRCs and the bloated signalling that this incurs. Also, relying solely on Offer/Answer here would expose an offerer to the typical race condition of repair SSRCs arriving before the answer and the processing ambiguity that this would imply.
Another approach could be a combination of RTCP and RTP header extensions [RFC5285] in a way similar to the one employed by the Rapid Synchronisation of RTP Flows [RFC6051]. While such a mechanism is not currently defined by the IETF, specifying it could be relatively straightforward:
Every packet belonging to a repair flow could carry an RTP header extension [RFC5285] that points to the source stream (or source layer in case of layered mechanisms).
Again, these are just some possibilities. Different mechanisms may and probably will require different extensions or signalling ([SRCNAME] will likely be an option for some). In some cases, where layering information is provided by the codec, an extensions is not going to be necessary at all.
In cases where FEC or simulcast relations are not immediately needed by the recipient, this information could also be delayed until the reception of the first RTCP packet.
One of the main characteristics of this specification is the use of SDP for transport channel setup and media stack initialisation only. In order for applications to be able to cover everything else it is important that WebRTC APIs actually allow for it. Given the initial directions taken by early implementations and specification work, this is currently almost but not entirely possible.
The following is a list of requirements that the WebRTC APIs would need to satisfy in order for this specification to be usable. (Note: some of the items are already possible and are only included for the sake of completeness.)
The following section Section 7.1 provides suggestions for addressing the above requirements.
This document proposes that the following methods and dictionaries be added to the WebRTC API. The changes follow the model of createDataChannel, which has a JS method on PeerConnection that makes it possible to add data channels without going through SDP. Furthermore, just like createDataChannel allows 2 ways to handle neogitation (the "I know what I'm doing; Here's what I want to send; Let me signal everything" mode and the "please take care of it for me; send an OPEN message" mode), this also has 2 ways to handle negotiation (the "I know what I'm doing; Here's what I want to send; Let me signal everything" mode and the "please take care of it for me; send SDP back and forth" mode).
Following the success of createDataChannel, this allows simple applications to Just Work and more advanced applications to easily control what they need to. In particular, it's possible to use this API to implement either Plan A or Plan B.
// The following two method are added to RTCPeerConnection
partial interface RTCPeerConnection {
// Create a stream that is used to send a source stream.
// The MediaSendStream.description can be used for signalling.
// No media is sent until addStream(MediaSendStream) is called.
LocalMediaStream createLocalStream(MediaStream sourceStream);
// Create a stream that is used to receive media from the remote side,
// given the parameters signalled from MedaiSendStream.description.
MediaStream createRemoteStream(MediaStreamDescription description);
}
interface LocalMediaStream implements MediaStream {
// This can be changed at any time, but especially before calling
// PeerConnection.addStream
attribute MediaStreamDescription description;
}
// Represents the parameters used to either send or receive a stream
// over a PeerConnection.
dictionary MediaStreamDescription {
MediaStreamTrackDescription[] tracks;
}
// Represents the parameters used to either send or receive a track over
// a PeerConnection. A track has many "flows", which can be grouped
// together.
dictionary MediaStreamTrackDescription {
// Same as the MediaStreamTrack.id
DOMString id;
// Same as the MediaStreamTrack.kind
DOMString kind;
// A track can have many "flows", such as for Simulcast, FEC, etc.
// And they can be grouped in arbitrary ways.
MediaFlowDescription[] flows;
MediaFlowGroup[] flowGroups;
}
// Represents the parameters used to either send or receive a "flow"
// over a PeerConnection. A "flow" is a media that arrives with a
// single, unique SSRC. One to many flows together make up the media
// for a track. For example, there may be Simulcast, FEC, and RTX
// flows.
dictionay MediaFlowDescription {
// The "flow id" must be unique to the track, but need not be unique
// outside of the track (two tracks could both have a flow with the
// same flow ID).
DOMString id;
// Each flow can go over its own transport. If the JS sets this to a
// transportId that doesn't have a transport setup already, the
// browser will use SDP negotiation to setup a transport to back that
// transportId. If This is set to an MID in the SDP, then that MID's
// transport is used.
DOMString transportId;
// The SSRC used to send the flow.
unsigned int ssrc;
// When used as receive parameters, this indicates the possible list
// of codecs that might come in for this flow. For exmample, a given
// receive flow could be setup to receive any of OPUS, ISAC, or PCMU.
// When used as send parameters, this indicates that the first codec
// should be used, but the browser can use send other codecs if it
// needs to because of either bandwidth or CPU constraints.
MediaCodecDescription[] codecs;
}
dictionary MediaFlowGroup {
DOMString type; // "SIM" for Simulcast, "FEC" for FEC, etc
DOMString[] flowids;
}
dictionary MediaCodecDescription {
unsigned byte payloadType;
DOMString name;
unsigned int? clockRate;
unsigned int? bitRate;
// A grab bag of other fmtp that will need to be further defined.
MediaCodecParam[] params;
}
dictionary MediaCodecParam {
DOMString key;
DOMString value;
}
}
Some additional notes:
API Pros:
API Cons:
Following is an example of how these API additions would be used:
// Imagine I have MyApp, handles creating a PeerConnection,
// signalling, and rendering streams. This is how the new API could be
// used.
var peerConnection = MyApp.createPeerConnection();
// On sender side:
var stream = MyApp.getMediaStream();
var localStream = peerConnection.createSendStream(stream);
sendStream.description = MyApp.modifyStream(localStream.description)
MyApp.signalAddStream(localStream.description, function(response)) {
if (!response.rejected) {
// Media will not be sent.
peerConnection.addStream(localStream);
}
}
// On receiver side:
MyApp.onAddStreamSignalled = function(streamDescription) {
var stream = peerConnection.createReceiveStream(streamDescription);
MyApp.renderStream(stream);
}
// In this exchange, the MediaStreamDescription signalled from the
// sender to the receiver may have looked something like this:
{
tracks: [
{
id: "audio1",
kind: "audio",
flows: [
{
id: "main",
transportId: "transport1",
ssrc: 1111,
codecs: [
{
payloadType: 111,
name: "opus",
// ... more codec details
},
{
payloadType: 112,
name: "pcmu",
// ... more codec details
}]
}]
},
{
id: "video1",
kind: "video",
flows: [
{
id: "sim0",
transportId: "transport2",
ssrc: 2222,
codecs: [
{
payloadType: 122,
name: "vp8"
// ... more codec details
}]
},
{
id: "sim1",
transportId: "transport2",
ssrc: 2223,
codecs: [
{
payloadType: 122,
name: "vp8",
// ... more codec details
}]
},
{
id: "sim2",
transportId: "transport2",
ssrc: 2224,
codecs: [
{
payloadType: 122,
name: "vp8",
// ... more codec details
}]
},
{
id: "sim0fec",
transportId: "transport2",
ssrc: 2225,
codecs: [
{
payloadType: 122,
name: "vp8",
// ...
}]
}],
flowGroups: [
{
semantics: "SIM",
ssrcs: [2222, 2223, 2224]
},
{
semantics: "FEC",
ssrcs: [2222, 2225]
}]
}]
}
None.
Many thanks to Bernard Aboba and Mary Barnes, for reviewing this document and providing numerous comments and substantial input.