Internet DRAFT - draft-groves-clue-multi-content
draft-groves-clue-multi-content
CLUE C. Groves, Ed.
Internet-Draft W. Yang
Intended status: Informational R. Even
Expires: February 13, 2014 Huawei
August 12, 2013
Describing Captures in CLUE and relation to multipoint conferencing
draft-groves-clue-multi-content-00
Abstract
In a multipoint Telepresence conference, there are more than two
sites participating. Additional complexity is required to enable
media streams from each participant to show up on the displays of the
other participants. Common policies to address the multipoint case
include "site-switch" and "segment-switch". The document will
discuss these policies as well as the "composed" policy and how they
work in the multipoint case.
The current CLUE framework document contains the "composed" and
"switched" attributes to describe situations where a capture is mix
or composition of streams or where the capture represents a dynamic
subset of streams. "Composed" and "switched" are capture level
attributes. In addition to these attributes the framework defines an
attribute "Scene-switch-policy" on a capture scene entry (CSE) level
which indicates how the captures are switched.
This draft discusses composition/switching in CLUE and makes a number
of proposals to better define and support these capabilities.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on February 13, 2014.
Groves, et al. Expires February 13, 2014 [Page 1]
Internet-Draft Abbreviated Title August 2013
Copyright Notice
Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1. Role of an MCU in a multipoint conference . . . . . . . . 4
2.2. Relation to scene . . . . . . . . . . . . . . . . . . . . 6
2.3. Description of the contents of a switched/composed
capture . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.4. Attribute interaction . . . . . . . . . . . . . . . . . . 7
2.5. Policy . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.6. Media stream composition and encodings . . . . . . . . . 8
2.7. Relation of switched captures to simultaneous
transmission sets . . . . . . . . . . . . . . . . . . . . 9
2.8. Conveying spatial information for switched/composed
captures . . . . . . . . . . . . . . . . . . . . . . . . 9
2.9. Consumer selection . . . . . . . . . . . . . . . . . . . 10
3. Proposal . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.1. CLUE Syntax Updates . . . . . . . . . . . . . . . . . . . 11
3.1.1. Definitions . . . . . . . . . . . . . . . . . . . . . 12
3.1.2. Multiple Content Capture Details . . . . . . . . . . 12
3.1.3. MCC Attributes . . . . . . . . . . . . . . . . . . . 13
3.1.4. MCC Attributes . . . . . . . . . . . . . . . . . . . 13
3.1.5. Composition policy . . . . . . . . . . . . . . . . . 14
3.1.6. Synchronisation . . . . . . . . . . . . . . . . . . . 14
3.1.7. MCC and encodings . . . . . . . . . . . . . . . . . . 15
3.1.8. MCCs and STSs . . . . . . . . . . . . . . . . . . . . 16
3.1.9. Consumer Behaviour . . . . . . . . . . . . . . . . . 16
3.1.10. MCU Behaviour . . . . . . . . . . . . . . . . . . . . 17
3.1.10.1. Single content captures and multiple contents
capture in the same Advertisement . . . . . . . 17
3.1.10.2. Several multiple content captures in the same
Advertisement . . . . . . . . . . . . . . . . . 18
3.2. Multipoint Conferencing Framework Updates . . . . . . . . 19
Groves, et al. Expires February 13, 2014 [Page 2]
Internet-Draft Abbreviated Title August 2013
3.3. Existing Parameter Updates . . . . . . . . . . . . . . . 20
3.3.1. Composed . . . . . . . . . . . . . . . . . . . . . . 20
3.3.2. Switched . . . . . . . . . . . . . . . . . . . . . . 21
3.3.3. Scene-switch-policy . . . . . . . . . . . . . . . . . 22
3.3.4. MCU behaviour . . . . . . . . . . . . . . . . . . . . 24
4. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 24
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24
6. Security Considerations . . . . . . . . . . . . . . . . . . . 25
7. References . . . . . . . . . . . . . . . . . . . . . . . . . 25
7.1. Normative References . . . . . . . . . . . . . . . . . . 25
7.2. Informative References . . . . . . . . . . . . . . . . . 25
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26
1. Introduction
One major objective for Telepresence is to be able to preserve the
"Being there" user experience. However, in multi-site conferences it
is often (in fact usually) not possible to simultaneously provide
full size video, eye contact, common perception of gestures and gaze
by all participants. Several policies can be used for stream
distribution and display: all provide good results but they all make
different compromises.
The policies are described in [I-D.ietf-clue-telepresence-use-cases].
[RFC6501] has the following requirement:
REQMT-14: The solution MUST support mechanisms to make possible for
either or both site switching or segment switching. [Edt:
This needs rewording. Deferred until layout discussion is
resolved.]
The policies described in the use case draft include the site-switch,
segment-switch and composed policies.
Site switch is described in the CLUE use case "One common policy is
called site switching. Let's say the speaker is at site A and
everyone else is at a "remote" site. When the room at site A shown,
all the camera images from site A are forwarded to the remote sites.
Therefore at each receiving remote site, all the screens display
camera images from site A. This can be used to preserve full size
image display, and also provide full visual context of the displayed
far end, site A. In site switching, there is a fixed relation
between the cameras in each room and the displays in remote rooms.
The room or participants being shown is switched from time to time
based on who is speaking or by manual control, e.g., from site A to
site B."
Groves, et al. Expires February 13, 2014 [Page 3]
Internet-Draft Abbreviated Title August 2013
These policies are mirrored in the framework document through a
number of attributes.
Currently in the CLUE framework document [I-D.ietf-clue-framework]
there are two media capture attributes: Composed and Switched.
Composed is defined as:
A field with a Boolean value which indicates whether or not
the Media Capture is a mix (audio) or composition (video) of
streams.
This attribute is useful for a media consumer to avoid
nesting a composed video capture into another composed
capture or rendering. This attribute is not intended to
describe the layout a media provider uses when composing
video streams.
Switched is defined as:
A field with a Boolean value which indicates whether or not
the Media Capture represents the (dynamic) most appropriate
subset of a 'whole'. What is 'most appropriate' is up to the
provider and could be the active speaker, a lecturer or a
VIP.
There is also a Capture Scene Entry (CSE) attribute "scene switch
policy" defined as:
A media provider uses this scene-switch-policy attribute to
indicate its support for different switching policies.
2. Issues
This section discusses a number of issues in the current framework
around the support of switched/composed captures and media streams
when considering multipoint conferencing. Some issues are more
required functions and some are related to the current description in
the framework document.
2.1. Role of an MCU in a multipoint conference
In a multipoint conference there is a central control point (MCU).
The MCU will have the CLUE advertisements from all the conference
participants and will prepare and send advertisements to all the
conference participants. The MCU will also have more information
about the conference, participants and media which it receives at
conference creation and via call signalling. This data is not stable
Groves, et al. Expires February 13, 2014 [Page 4]
Internet-Draft Abbreviated Title August 2013
since each user who joins or leaves the conference causes a change is
conference state. An MCU supporting SIP may utilise the Conference
event package, XCON and CCMP to maintain and distribute conference
state.
[RFC4575] defines a conference event package. Using the event
framework notifications are sent about changes in the membership of
this conference and optionally about changes in the state of
additional conference components. The conference information is
composed of the conference description, host information, conference
state, users that has endpoints where each endpoint includes the
media description.
[RFC6501] extends the conference event package and tries to be
signalling protocol agnostic. RFC6501 adds new elements but also
provides values for some of the elements defined in RFC4575, for
example it defines roles ( like "administrator", "moderator", "user",
"participant", "observer", and "none").
[RFC6503] Centralized Conferencing Manipulation Protocol (CCMP)
allows authenticated and authorized users to create, manipulate, and
delete conference objects. Operations on conferences include adding
and removing participants, changing their roles, as well as adding
and removing media streams and associated endpoints.
CCMP implements the client-server model within the XCON framework,
with the conferencing client and conference server acting as client
and server, respectively. CCMP uses HTTP as the protocol to transfer
requests and responses, which contain the domain-specific XML-encoded
data objects defined in [RFC6501] "Conference Information Data Model
for Centralized Conferencing (XCON)".
The XCON data model and CCMP provides a generic way to create and
control conferences. CCMP is not SIP specific but SIP endpoint will
subscribe to the conference event package to get information about
changes in the conference state.
Therefore when a MCU implements the above protocols there will be an
interaction between any CLUE states and those within a conferencing
framework. For example: if an endpoint leaves a conference this will
mean that an MCU may need to indicate via CLUE to the other endpoints
that those captures are no longer available and it would also need to
indicate via the Conferencing framework that the endpoint is longer
part of the conference.
The question is how do these concepts relate as the Conferencing
framework does not have the concept of captures or scenes? Other
aspects overlap, for example:
Groves, et al. Expires February 13, 2014 [Page 5]
Internet-Draft Abbreviated Title August 2013
The conference framework has "available media" , CLUE has
encodings to indicate codec.
The conference framework has "users", CLUE has no concept of
users although it has capture attributes that relate to the
users in a capture.
It is noted point to point calls may not implement the conferencing
framework. It is desirable that CLUE procedures be the same whether
an endpoint is communicating with a peer endpoint or an MCU.
2.2. Relation to scene
One of the early justifications for switching / composition was the
ability to switch between sites. When looking at the CLUE framework
there is no concept of "site" in the CLUE hierarchy. The closest
concept is an "endpoint" but this has no identity within the CLUE
syntax. The highest level is the "clueInfo" that includes
captureScenes and an endpoint may have multiple capture scenes.
If the switched and composed attributes are specified at a capture
level it is not clear what the correlation is between the capture and
the endpoint / scenes, particularly when the attributes are described
in the context of sites. A scene may be composed of multiple
captures. Where an MCU is involved in a conference with multiple
endpoints, multiple capture scenes are involved. It becomes
difficult to map all the scenes and capture information from the
source endpoints into one capture scene sent to an endpoint.
Discussion of switching, composition et al. needs to be described in
terms of the CLUE concepts.
When considering the SIP conferencing framework it can be seen that
there are complications with interworking with the scene concept.
There may be multiple media of the same type e.g room view and
presentation but they are easily identified. This also needs to be
considered.
2.3. Description of the contents of a switched/composed capture
Groves, et al. Expires February 13, 2014 [Page 6]
Internet-Draft Abbreviated Title August 2013
When considering switching and composition whilst this may be
represented by one capture and one resulting media stream there may
be multiple original source captures. Each of these source captures
would have had their own set of attributes. A media capture with the
composed attribute allows the description of the capture as whole but
not a description of the constituent parts. In the case of a MCU
taking multiple input media captures and compositing them into one
output capture the CLUE related characteristics of these inputs are
lost in the current solution. Alternate methods such as the RFC6501
layout field etc. may need to be investigated.
Consider the case where MCUs receive CLUE advertisements from various
endpoints. Having a single capture with a switched attribute makes
it difficult to fully express what the content is when it is from
multiple endpoints. It may be possible to specify lists of capture
attribute values when sending an advertisement from the MCU, i.e.
role=speaker,audience but it becomes difficult to relate multiple
attributes, i.e.
(role=speaker,language=English),(role=audience,language=french).
One capture could represent source captures from multiple locations.
A consumer may wish to examine the inputs to a switched capture ,
i.e. choose which of the original endpoints it wants to see/hear. In
order to do this the original capture information would need to be
conveyed in a manner that minimises overhead for the MCU.
By being able to link multiple source captures to one mixed (switched
/composed) capture in a CLUE advertisement allows a fuller
description of the content of the capture.
2.4. Attribute interaction
Today the "composed" and "switched" attributes appear at a media
capture level. If "switched" is specified for multiple captures in a
capture scene it's not clear from the framework what the switching
policy is. For example: If a CSE contains three VCs each with
"switched" does the switch occurs between these captures? Does the
switch occur internal to each capture?
The "scene-switch-policy" CSE attribute has been defined to indicate
switch policy but there doesn't appear to be a description of whether
this only relates to captures marked with "switch" and/or "composed"?
If a CSE marked with "scene-switch-policy" contains non-switched,
non-composed captures what does this mean?
Groves, et al. Expires February 13, 2014 [Page 7]
Internet-Draft Abbreviated Title August 2013
What are the interactions between the two properties? E.g. Are
"switched" and "composed" attributes mutually exclusive or not? Is
switched capture with a scene switch policy of "segment-switched" a
"composed" capture?
These issues need to be clarified in the framework.
2.5. Policy
The "Scene-switch-policy" attribute allows the indication of whether
switched captures are "site" or "segment" switched. However there is
no indication of what the switch or the composition "trigger" policy
is. Content could be provided based on a round robin view, loudest
speaker etc. Where an advertising endpoint supports different
algorithms it would be advantageous for a consumer to know and select
an applicable policy.
2.6. Media stream composition and encodings
Whether single or multiple streams are used for switched captures is
not clear from the capture description. For example:
There are 3 endpoints (A,B,C) each with 3 video
captures.(VCa1,VCa2,VCa3, etc.). A MCU wants to indicate to endpoint
C that it can offer a switched view of endpoints A and B.
It could send an Advertisement with CSE (VCa1,VCa2,VCa3,
VCb1,VCb2,VCb3),scene-switch-policy=site-switch.
Normally such a configuration (without the switch policy) would
relate to 6 different media streams. Switching introduces several
possibilities.
For site switching:
a) There could one media stream with the contents of all the 6
captures. The MCU always send a composed image with the VCs
from the applicable end point.
b) There could be two media streams each containing the VCs from
one endpoint, the MCU chooses which stream to send.
c) There could be 6 media streams. The MCU chooses which 3
streams to send.
For segment switching this is further complicated because the MCU may
choose to send media related to endpoint A or B. There no text
describing any limitation so the MCU may send 1 VC or 5.
Groves, et al. Expires February 13, 2014 [Page 8]
Internet-Draft Abbreviated Title August 2013
Utilising CLUE "encodings" may be a way to describe how the switch is
taking place in terms of media provided but such description is
missing from the framework. One could assume that an individual
encoding be assigned to multiple media captures (i.e. multiple VCs to
indicate they are encoded in the same stream) but again this is
problematic as the framework indicates that "An Individual Encoding
can be assigned to at most one Capture Encoding at any given time."
This could do with further clarification in the framework.
2.7. Relation of switched captures to simultaneous transmission sets
Simultaneous Transmission Set is defined as "a set of Media Captures
that can be transmitted simultaneously from a Media Provider." It's
not clear how this definition would relate to switched or composed
streams. The captures may not be able to be sent at the same time
but may form a timeslot on a particular stream. They may be provided
together but not at precisely the same time.
The current version of the framework in section 6.3 it indicates
that:
"It is a syntax conformance requirement that the simultaneous
transmission sets must allow all the media captures in any
particular Capture Scene Entry to be used simultaneously."
If switching or composition is specified at a capture level only it
is evident that simultaneity constraints do not come into play.
However if multiple captures are used in a single media stream I.e.
associated with the CSE then these may be subject to a simultaneous
transmission set description.
It is also noted that there is a similar issue for encoding group.
See section 8/[Framework]:
"It is a protocol conformance requirement that the Encoding
Groups must allow all the Captures in a particular Capture
Scene Entry to be used simultaneously."
If "switching" is used then there is no need to send the encodings at
the same time.
This needs to be clarified.
2.8. Conveying spatial information for switched/composed captures
CLUE currently allows the ability to signal spatial information
related to a media capture. It is unclear in the current draft how
Groves, et al. Expires February 13, 2014 [Page 9]
Internet-Draft Abbreviated Title August 2013
this would work with switching/composition. In section 6.1 /
[I-D.ietf-clue-telepresence-use-cases] it does say:
"For a switched capture that switches between different
sections within a larger area, the area of capture should use
coordinates for the larger potential area."
This describes a single capture not when there are multiple switched
captures. It appears to focus on segment switching rather than site
switching and does not appear to cover "composed" (if it is related).
An advertiser may or may not want to use common spatial attributes
for captures associated with a switched captures. For example: it
may be beneficial for the Advertiser in a composed image to indicate
that different captures have a different capture area in a virtual
space.
This should be given consideration in the framework.
2.9. Consumer selection
In section 6.2.2 of version 9 [I-D.ietf-clue-framework] it indicates
that an Advertiser may provide multiple values for the "scene-switch-
policy" and that the Consumer may choose and return the value it
requires.
In version 9 of the framework there was no mechanism in CLUE for a
Consumer to choose and return individual values from capture scene,
CSE or media capture attributes.
In version 10 of the framework the text was updated to indicate that
the consumer could choose values from a list. It is not clear that
this capability is needed as the procedure only relates to the
"scene-switch-policy". The switching policy may be better specified
by other means.
3. Proposal
As has been discussed above there are a number of issues with regards
to the support of switched/composed captures/streams in CLUE
particularly when considering MCUs. The authors believe that there
is no single action that can address the above issues. Several
options are discussed below. The options are not mutually exclusive.
1) Introduce syntax to CLUE to better describe source captures
2) Introduce updates to the XCON conferencing framework (e.g.
Conference package, XCON etc.) to introduce CLUE concepts.
Groves, et al. Expires February 13, 2014 [Page 10]
Internet-Draft Abbreviated Title August 2013
3) Update CLUE to better describe the current suite of
attributes with the understanding these provide limited
information with respect to source information.
3.1. CLUE Syntax Updates
The authors believe that there are a number of requirements for this:
- It should be possible to advertise the individual captures
that make up a single switched/composed media stream before
receiving the actual media stream.
- It should be possible to describe the relationship between
captures that make up a single switched/composed media
stream.
- It should be possible to describe this using CLUE semantics
rather than with terms such as "site" or "segment" which need
their own definition.
The authors also believe that whether media is composed, segment
switched, site switched the common element is that the media stream
contains multiple captures from potentially multiple sources.
[I-D.ietf-clue-framework] does have the "Scene-switch-policy"
attribute at a CSE level but as described in section 2 it is not
sufficient for several reasons. E.g. it is not possible assign an
encoding to a CSE, a CSE cannot reference captures from multiple
scenes and there is a relationship with STSs that needs to be
considered.
In order to be able to fully express and support media stream with
multiple captures the authors propose a new type of capture, the
"multiple content capture" (MCC). The MCC is essentially the same as
audio or video captures in that it may have its own attributes the
main difference is that it can also include other captures. It
indicates that the MCC capture is composed of other captures. This
composition may be positional (i.e. segments/tiling) or time
composition (switched) etc. and specified by a policy attribute. The
MCC can be assigned an encoding. For example:
MCC1(VC1,VC2,VC3),[POLICY]
This would indicate that MCC1 is composed of 3 video captures
according to the policy.
One further difference is that a MCC may reference individual
captures from multiple scenes. For example:
Groves, et al. Expires February 13, 2014 [Page 11]
Internet-Draft Abbreviated Title August 2013
CS#1(VC1,VC2)
CS#2(VC3,VC4)
CS#3(MCC1(VC1,VC3))
This would indicate that scene #3 contains a MCC that is composed
from individual encodings VC1 and VC3. This allows the consumer to
associate any capture scene properties from the original scene with
the multiple content capture.
The MCC would be able to be utilised by both normal endpoints and
MCUs. For example: it would allow an endpoint to construct a mixed
video stream that is a virtual scene with a composition of
presentation video and individual captures.
This proposal does not consider any relation to the SIP conferencing
framework.
The sections below provide more detail on the proposal.
3.1.1. Definitions
Multiple content capture: Media capture for audio or video that
indicates the capture contains multiple audio or video captures.
Individual media captures may or may not be present in the resultant
capture encoding depending on time or space. Denoted as MCCn in the
example cases in this document.
3.1.2. Multiple Content Capture Details
The MCC indicates that multiple captures are contained in one media
capture by referencing the applicable individual media captures.
Only one capture type (i.e. audio, video, etc.) is allowed in each
MCC instance. The MCC contains a reference to the media captures as
well attributes associated with the MCC itself. The MCC may
reference individual captures from other capture scenes. If an MCC
is used in a CSE that CSE may also reference captures from other
Capture Scenes.
Note: Different Capture Scenes are not spatially related.
Each instance of the MCC has its own captureID i.e. MCC1. This
allows all the individual captures contained in the MCC to be
referenced by a single ID.
The example below shows the use of a MultipleContent capture:
Groves, et al. Expires February 13, 2014 [Page 12]
Internet-Draft Abbreviated Title August 2013
CaptureScene1 [VC1 {attributes},
VC2 {attributes},
VC3 {attributes},
MCC1(VC1,VC2,VC3){attributes}]
This indicates that MCC1 is a single capture that contains the
captures VC1, VC2 and VC3 according to any MCC1 attributes.
One or more MCCs may also specified in a CSE. This allows an
Advertiser to indicate that several MCC captures are used to
represent a capture scene.
Note: Section 6.1/[I-D.ietf-clue-framework] indicates that "A Media
Capture is associated with exactly one Capture Scene". For MCC this
could be further clarified to indicate that "A Media Capture is
defined in a capture scene and is given an advertisement unique
identity. The identity may be referenced outside the Capture Scene
that defines it through a multiple content capture (MCC).
3.1.3. MCC Attributes
Attributes may be associated with the MCC instance and the individual
captures that the MCC references. A provider should avoid providing
conflicting attribute values between the MCC and individual captures.
Where there is conflict the attributes of the MCC override any that
may be present in the individual captures.
There are two MCC specific attributes "MaxCaptures" and "Policy"
which are used to give more information regarding when the individual
captures appears and what policy is used to determine this.
The spatial related attributes can be further used to determine how
the individual captures "appear" within a stream. For example a
virtual scene could be constructed for the MCC capture with two video
captures with a "MaxCaptures" attribute of 2 and an "area of capture"
attribute provided with an overall area. Each of the individual
captures could then also include an "area of capture" attribute with
a sub-set of the overall area. The consumer would then know the
relative position of the content in the composed stream. For
example: The above capture scene may indicate that VC1 has an x-axis
capture area 1-5, VC2 6-10 and VC3 11-15. The MCC capture may
indicate an x-axis capture area 1-15.
3.1.4. MCC Attributes
MaxCaptures:{integer}
Groves, et al. Expires February 13, 2014 [Page 13]
Internet-Draft Abbreviated Title August 2013
This field is only associated with MCCs and indicates the maximum
number of individual captures that may appear in a capture encoding
at a time. It may be used to derive how the individual captures
within the MCC are composed with regards to space and time.
Individual content in the capture may be switched in time so that
only one of the individual captures/CSEs are shown (MaxCaptures:1).
The individual captures may be composed so that they are all shown in
the MCC (MaxCaptures:n).
For example:
MCC1(VC1,VC2,VC3),MaxCaptures:1
This would indicate that the Advertiser in the capture encoding would
switch (or compose depending on policy) between VC1, VC2 or VC3 as
there may be only a maximum of one capture at a time.
3.1.5. Composition policy
TBD - This attribute is to address what algorithm the endpoint/MCU
uses to determine what appears in the MCC captures. E.g. loudest,
round robin.
3.1.6. Synchronisation
Note: The {scene-switch-policy} attribute has values that indicates
"site-switch" or "segment" switch. The distinction between these is
that "site-switch" indicates that when there is mixed content that
captures related to an endpoint appear together. "segment-switch"
indicates that different endpoints captures could appear together.
An issue is that a Consumer has no concept of "endpoints" only
"capture scenes". Also as highlighted a Consumer has no method to
return parameters for CSEs.
The use of MCCs enables the Advertiser to communicate to the Consumer
that captures originate from different captures scenes. In cases
where multiple MCCs represent a scene (i.e. multiple MCCs in a CSE)
an Advertiser may wish to indicate that captures from one capture
scene are present in the capture encodings of specified MCCs at the
same time. Having an attribute at capture level removes the need for
CSE level attributes which are problematic for consumers.
Synch-id: { integer}
This MCC attribute indicates how the individual captures in multiple
MCC captures are synchronised. To indicate that the capture
encodings associated with MCCs contain captures from the source at
the same time the Advertiser should set the same SynchID on each of
Groves, et al. Expires February 13, 2014 [Page 14]
Internet-Draft Abbreviated Title August 2013
the concerned MCCs. It is the provider that determines what the
source for the captures is. For example when the provider is in an
MCU it may determine that each separate CLUE endpoint is a remote
source of media.
For example:
CaptureScene1[Description=AustralianConfRoom,
VC1(left),VC2(middle),VC3(right),
CSE1(VC1,VC2,VC3)]
CaptureScene2[Description=ChinaConfRoom,
VC4(left),VC5(middle),VC6(right),
CSE2(VC4,VC5,VC6)]
CaptureScene3[MCC1(VC1,VC4){Sync-id:1}{encodinggroup1},
MCC2(VC2,VC5){Sync-id:1}{encodinggroup2},
MCC3(VC3,VC6){encodinggroup3},
CSE3(MCC1,MCC2,MCC3)]
Figure 1: Synchronisation Example
The above advertisement would indicate MCC1,MCC2,MCC3 make up a
capture scene. There would be three capture encodings. Because MCC1
and MCC2 have the same Sync-id, each encoding1 and encoding2 would
together have content from only capture scene 1 or only capture scene
2 at a particular point in time. Encoding3 would not be synchronised
with encoding1 or encoding2.
Without this attribute it is assumed that multiple MCCs may provide
different sources at any particular point in time.
3.1.7. MCC and encodings
MCCs shall be assigned an encoding group and thus become a capture
encoding. The captures referenced by the MCC do not need to be
assigned to an encoding group. This means that all the individual
captures referenced by the MCC will appear in the capture encoding
according to any MCC attributes. This allows an Advertiser to
specify capture attributes associated with the individual captures
without the need to provide an individual capture encoding for each
of the inputs.
If an encoding group is assigned to an individual capture referenced
by the MCC it indicates that this capture may also have an individual
capture encoding.
For example:
Groves, et al. Expires February 13, 2014 [Page 15]
Internet-Draft Abbreviated Title August 2013
CaptureScene1 [VC1 {encoding group1},
VC2 ]
MCC1(VC1,VC2){encoding group3}]
This would indicate that VC1 may be sent as its own capture encoding
from encoding group1 or that it may be sent as part of a capture
encoding from encoding group3 along with VC2.
Note: The section 8/[I-D.ietf-clue-framework] indicates that every
capture is associated with an encoding group. To utilise MCCs this
requirement has to be relaxed.
3.1.8. MCCs and STSs
The MCC can be used in simultaneous sets, therefore providing a means
to indicate whether several multiple content captures can be provided
at the same time. Captures within a MCC can be provided together but
not necessarily at the same time. Therefore by specifying a MCC in
an STS it does not indicate that all the referenced individual
captures may be present at a time. The MaxCaptures attributes
indicates the maximum number of captures that may be present.
An MCC instance of is limited to one media type e.g. video, audio,
text.
Note: This gets around the problem where the framework says that all
captures (even switched ones) within a CSE have to be allowed in a
STS to be sent at the same time.
3.1.9. Consumer Behaviour
On receipt of an advertisement with an MCC the Consumer treats the
MCC as per other individual captures with the following differences:
- The Consumer would understand that the MCC is a capture that
includes the referenced individual captures and that these
individual captures would be delivered as part of the MCC's
capture encoding.
- The Consumer may utilise any of the attributes associated
with the referenced individual captures and any capture scene
attributes from where the individual capture was defined to
choose the captures.
- The Consumer may or may not want to receive all the indicated
captures. Therefore it can choose to receive a sub-set of
captures indicated by the MCC.
Groves, et al. Expires February 13, 2014 [Page 16]
Internet-Draft Abbreviated Title August 2013
For example if the Consumer receives:
MCC1(VC1,VC2,VC3){attributes}
A Consumer should choose all the captures within a MCCs however if
the consumer determines that it doesn't want VC3 it can return
MCC1(VC1,VC2). If it wants all the individual capture then it
returns just a reference to the MCC (i.e. MCC1).
Note: The ability to return a subset of capture is for consistency
with the current framework. It says that a Consumer should choose
all the captures from a CSE but it allows it to select a subset (if
the STS is provided). The intent was to provide equivalent
functionality for a MCC.
3.1.10. MCU Behaviour
The use of MCCs allows the MCU to easily construct outgoing
Advertisements. The following sections provide several examples.
3.1.10.1. Single content captures and multiple contents capture in the
same Advertisement
Four endpoints are involved in a CLUE session. To formulate an
Advertisement to endpoint 4 the following Advertisements received
from endpoint 1 to 3 and used by the MCU. Note: The IDs overlap in
the incoming advertisements. The MCU is responsible for making these
unique in the outgoing advertisement.
Endpoint 1 CaptureScene1[Description=AustralianConfRoom,
VC1(role=audience)]
Endpoint 2 CaptureScene1[Description=ChinaConfRoom,
VC1(role=speaker),VC2(role=audience),
CSE1(VC1,VC2)]
Endpoint 3 CaptureScene1[Description=USAConfRoom,
VC1(role=audience)]
Figure 2: MCU case: Received advertisements
Note: Endpoint 2 above indicates that it sends two streams.
If the MCU wanted to provide a multiple content capture containing
the audience of the 3 endpoints and the speaker it could construct
the following advertisement:
CaptureScene1[Description=AustralianConfRoom,
VC1(role=audience)]
CaptureScene2[Description=ChinaConfRoom,
Groves, et al. Expires February 13, 2014 [Page 17]
Internet-Draft Abbreviated Title August 2013
VC2(role=speaker),VC3(role=audience),
CSE1(VC2,VC3)]
CaptureScene3[Description=USAConfRoom,
VC4(role=audience)]
CaptureScene4[MCC1(VC1,VC2,VC3,VC4){encodinggroup1}]
Figure 3: MCU case: MCC with multiple audience and speaker
Alternatively if the MCU wanted to provide the speaker as one stream
and the audiences as another it could assign an encoding group to VC2
in Capture Scene 2 and provide a CSE in Capture Scene 4:
CaptureScene1[Description=AustralianConfRoom,
VC1(role=audience)]
CaptureScene2[Description=ChinaConfRoom,
VC2(role=speaker){encodinggroup2},
VC3(role=audience),
CSE1(VC2,VC3)]
CaptureScene3[Description=USAConfRoom,
VC4(role=audience)]
CaptureScene4[MCC1(VC1,VC3,VC4){encodinggroup1},
CSE2(MCC1,VC2)]
Figure 4: MCU case: MCC with audience and separate speaker
Therefore a Consumer could choose whether or not to have a separate
"role=speaker" stream and could choose which endpoints to see. If it
wanted the second stream but not the Australian conference room it
could indicate the following captures in the Configure message:
MCC1(VC3,VC4),VC2
Figure 5: MCU case: Consumer Response
3.1.10.2. Several multiple content captures in the same Advertisement
Multiple MCCs can be used where multiple streams are used to carry
media from multiple endpoints. For example:
A conference has three endpoints D,E and F, each end point has three
video captures covering the left, middle and right regions of each
conference room. The MCU receives the following advertisements from
D and E:
Endpoint D CaptureScene1[Description=AustralianConfRoom,
VC1(left){encodinggroup1},
VC2(middle){encodinggroup2},
VC3(right){encodinggroup3},
Groves, et al. Expires February 13, 2014 [Page 18]
Internet-Draft Abbreviated Title August 2013
CSE1(VC1,VC2,VC3)]
Endpoint E CaptureScene1[Description=ChinaConfRoom,
VC1(left){encodinggroup1},
VC2(middle){encodinggroup2},
VC3(right){encodinggroup3},
CSE1(VC1,VC2,VC3)]
Figure 6: MCU case: Multiple captures from multiple endpoints
Note: The Advertisement uses the same identities. There is no co-
ordination between endpoints so it is likely there would be identity
overlap between received advertisements.
The MCU wants to offer Endpoint F three capture encodings. Each
capture encoding would contain a capture from either Endpoint D or
Endpoint E depending on the policy. The MCU would send the
following:
CaptureScene1[Description=AustralianConfRoom,
VC1(left),VC2(middle),VC3(right),
CSE1(VC1,VC2,VC3)]
CaptureScene2[Description=ChinaConfRoom,
VC4(left),VC5(middle),VC6(right),
CSE2(VC4,VC5,VC6)]
CaptureScene3[MCC1(VC1,VC4){encodinggroup1},
MCC2(VC2,VC5){encodinggroup2},
MCC3(VC3,VC6){encodinggroup3},
CSE3(MCC1,MCC2,MCC3)]
Figure 7: MCU case: Multiple MCCs for multiple captures
Note: The identities from Endpoint E have been renumbered so that
they are unique in the outgoing advertisement.
3.2. Multipoint Conferencing Framework Updates
The CLUE protocol extends the EP description defined in the
signalling protocol (SDP for SIP) by providing more information about
the available media. If we look at XCON it uses the information
available from the signalling protocol but instead of using SDP to
distribute the participants information and to control the multipoint
conference. This is done using a data structure defined in XML using
the CCMP protocol over HTML (note that CCMP can be used also over
CLUE channel if required). XCON provide a hierarchy the starts from
conference information that includes users having endpoints that have
media.
Groves, et al. Expires February 13, 2014 [Page 19]
Internet-Draft Abbreviated Title August 2013
The role is part of the user structure while the mixing mode is part
of the conference level information specifying the mixing mode per
each of the media available in the conference.
CLUE on the other end does not have such structure it start from what
is probably, in XCON terms, an end points that has media structured
by Scenes that has media. There is no user or conference level
information though the "role" proposal tries to add the user
information (note that use information is different from the role in
the call or the conference).
The XCON structure looks better when looking at a multipoint
conference. Yet it does not make sense to have such a data model for
the point to point calls. Therefore only going with this option
means that capture attribute information will not be available for
point to point calls.
3.3. Existing Parameter Updates
As discussed in section 2 the existing CLUE attributes surrounding
switching and composition have a number of open issues. This section
proposes changes to the text describing the attributes to better
describe their usage and interaction. It is also assumed that by
using these attributes there is no attempt to describe the any
component source capture information.
3.3.1. Composed
The current CLUE framework describes the "Composed" attribute as:
A boolean field which indicates whether or not the Media
Capture is a mix (audio) or composition (video) of streams.
This attribute is useful for a media consumer to avoid
nesting a composed video capture into another composed
capture or rendering. This attribute is not intended to
describe the layout a media provider uses when composing
video streams.
It is proposed to update the description:
Groves, et al. Expires February 13, 2014 [Page 20]
Internet-Draft Abbreviated Title August 2013
A boolean field which indicates whether or not the Media
Capture has been composed from a mix of audio sources or
several video sources. The sources may be local to the
provider (i.e. video capture device) or remote to the
provider (i.e. a media stream received by the provider from a
remote endpoint). This attribute is useful for a media
consumer to avoid nesting a composed video capture into
another composed capture or rendering.
This attribute does not imply anything with regards to the
attributes of the source audio or video except that the
composed capture will be contained in a capture encoding from
a single source. This attribute is not intended to describe
the layout a media provider uses when composing video
streams.
The "composed" attribute may be used in conjunction with a
"switched" attribute when one or more of the dynamic sources
is a composition.
3.3.2. Switched
The current CLUE framework describes the "Switched" attribute as:
A boolean field which indicates whether or not the Media
Capture represents the (dynamic) most appropriate subset of a
'whole'. What is 'most appropriate' is up to the provider
and could be the active speaker, a lecturer or a VIP.
It is proposed to update the description:
A boolean field which indicates whether the Media Capture
represents a dynamic representation of the capture scene that
contains the capture. It applies to both audio and video
captures.
A dynamic representation is one that provides alternate
capture sub-areas within the overall area of capture
associated with the capture over time in a single capture
encoding from one source. What capture area is contained in
the capture encoding at a particular time is dependent on the
provider policy. For example: a provider may encode the
active speaker or lecturer based on volume level. It is not
possible for consumers to associate attributes with a
particular capture sub-area nor to indicate which sub-capture
area they require.
Groves, et al. Expires February 13, 2014 [Page 21]
Internet-Draft Abbreviated Title August 2013
3.3.3. Scene-switch-policy
The current CLUE framework describes the "Scene Switch Policy"
attribute as:
Scene-switch-policy: {site-switch, segment-switch}
A media provider uses this scene-switch-policy attribute to
indicate its support for different switching policies. In
the provider's Advertisement, this attribute can have
multiple values, which means the provider supports each of
the indicated policies.
The consumer, when it requests media captures from this
Capture Scene Entry, should also include this attribute but
with only the single value (from among the values indicated
by the provider) indicating the Consumer's choice for which
policy it wants the provider to use. The Consumer must
choose the same value for all the Media Captures in the
Capture Scene Entry. If the provider does not support any of
these policies, it should omit this attribute.
The "site-switch" policy means all captures are switched at
the same time to keep captures from the same endpoint site
together. Let's say the speaker is at site A and everyone
else is at a "remote" site.
When the room at site A shown, all the camera images from
site A are forwarded to the remote sites. Therefore at each
receiving remote site, all the screens display camera images
from site A. This can be used to preserve full size image
display, and also provide full visual context of the
displayed far end, site A. In site switching, there is a
fixed relation between the cameras in each room and the
displays in remote rooms. The room or participants being
shown is switched from time to time based on who is speaking
or by manual control.
The "segment-switch" policy means different captures can
switch at different times, and can be coming from different
endpoints. Still using site A as where the speaker is, and
"remote" to refer to all the other sites, in segment
switching, rather than sending all the images from site A,
only the image containing the speaker at site A is shown.
The camera images of the current speaker and previous
speakers (if any) are forwarded to the other sites in the
conference.
Groves, et al. Expires February 13, 2014 [Page 22]
Internet-Draft Abbreviated Title August 2013
Therefore the screens in each site are usually displaying
images from different remote sites - the current speaker at
site A and the previous ones. This strategy can be used to
preserve full size image display, and also capture the non-
verbal communication between the speakers. In segment
switching, the display depends on the activity in the remote
rooms - generally, but not necessarily based on audio /
speech detection.
Firstly it is proposed to rename this attribute to "Capture Source
Synchronisation" in order to remove any confusion with the switch
attribute and also to remove the association with a scene as the any
information regarding source scenes is lost. This is due to that the
CSE represents the current scene. No change in functionality is
intended by the renaming. It is proposed to describe it as follows:
Capture Source Synchronisation: {source-synch,asynch}
By setting this attribute against a CSE it indicates that
each of the media captures specified within the CSE results
in a capture encoding that contains media related to
different remote sources. For example if CSE1 contains
VC1,VC2,VC3 then there will be three capture encodings sent
from the provider each displaying captures from different
remote sources. It is the provider that determines what the
source for the captures is. For example when the provider is
in an MCU it may determine that each separate CLUE endpoint
is a remote source of media. Likewise it is the provider
that determines how many remote sources are involved.
However it is assumed that each capture within the CSE will
contain the same number and set of sources.
"Source-synch" indicates that each capture encoding related
to the captures within the CSE contains media related to one
remote source at the same point in time.
"Asynch" indicates that that each capture encoding may
contain media related to any remote source at any point in
time.
If a provider supports both synchronisation methods it should
send separate CSEs containing separate captures, each CSE
with a separate capture source synchronisation label.
A provider when setting attributes against captures within a
Capture Source Synchronisation marked CSE should consider
that the media related to the remote sources may have its own
separate characteristics. For example: each source may have
Groves, et al. Expires February 13, 2014 [Page 23]
Internet-Draft Abbreviated Title August 2013
its own capture area therefore this needs to be taken into
account in the providers advertisement.
The "Switched" attribute may be used with a capture in a
"Capture Source Synchronisation" marked CSE. This indicates
that one or more of the remote sources associated with the
capture has dynamic media that may change within its own time
frame. i.e. the media from a remote source may change without
an impact on the other captures.
The "Composed" attribute may be used with captures in the
"Capture Source Synchronisation" marked CSE. This indicates
the capture encoding contains a composition or multiple
sources from one remote endpoint at a particular point in
time.
Furthermore it is assumed that if the current set of parameters is
maintained that the indication of the mechanism for the trigger of
switching sources (e.g. loudest source, round robin) is not possible
because the Consumer only chooses captures and not sources. If it's
purely up to the provider then this information would be superfluous.
It is proposed to capture this:
The trigger (or policy) that decides when a source is present
is up to the provider. The ability to provide detailed
information about sources is for further study.
3.3.4. MCU behaviour
When a CLUE endpoint is acting as a MCU it implies the need for an
advertisement aggregation function. That is the endpoint receives
CLUE advertisements from multiple endpoints uses this information,
its media processing capabilities and any policy information to form
advertisements to the other endpoints.
Contributor's note: TBD I think there needs to be a discussion here
about that source information is lost. How individual attributes are
affected. i.e. it may be possible to simply aggregate language
information but not so simple when there's different spatial
information. Also need to consider capture encodings.
4. Acknowledgements
This template was derived from an initial version written by Pekka
Savola and contributed by him to the xml2rfc project.
5. IANA Considerations
Groves, et al. Expires February 13, 2014 [Page 24]
Internet-Draft Abbreviated Title August 2013
It is not expected that the proposed changes present the need for any
IANA registrations.
6. Security Considerations
It is not expected that the proposed changes present any addition
security issues to the current framework.
7. References
7.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
7.2. Informative References
[I-D.groves-clue-capture-attr]
Groves, C., Yang, W., and R. Even, "CLUE media capture
description", draft-groves-clue-capture-attr-01 (work in
progress), February 2013.
[I-D.ietf-clue-framework]
Duckworth, M., Pepperell, A., and S. Wenger, "Framework
for Telepresence Multi-Streams", draft-ietf-clue-
framework-11 (work in progress), July 2013.
[I-D.ietf-clue-telepresence-requirements]
Romanow, A., Botzko, S., and M. Barnes, "Requirements for
Telepresence Multi-Streams", draft-ietf-clue-telepresence-
requirements-04 (work in progress), July 2013.
[I-D.ietf-clue-telepresence-use-cases]
Romanow, A., Botzko, S., Duckworth, M., and R. Even, "Use
Cases for Telepresence Multi-streams", draft-ietf-clue-
telepresence-use-cases-05 (work in progress), April 2013.
[RFC2629] Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629,
June 1999.
[RFC4575] Rosenberg, J., Schulzrinne, H., and O. Levin, "A Session
Initiation Protocol (SIP) Event Package for Conference
State", RFC 4575, August 2006.
[RFC6501] Novo, O., Camarillo, G., Morgan, D., and J. Urpalainen,
"Conference Information Data Model for Centralized
Conferencing (XCON)", RFC 6501, March 2012.
Groves, et al. Expires February 13, 2014 [Page 25]
Internet-Draft Abbreviated Title August 2013
[RFC6503] Barnes, M., Boulton, C., Romano, S., and H. Schulzrinne,
"Centralized Conferencing Manipulation Protocol", RFC
6503, March 2012.
Authors' Addresses
Christian Groves (editor)
Huawei
Melbourne
Australia
Email: Christian.Groves@nteczone.com
Weiwei Yang
Huawei
P.R.China
Email: tommy@huawei.com
Roni Even
Huawei
Tel Aviv
Isreal
Email: roni.even@mail01.huawei.com
Groves, et al. Expires February 13, 2014 [Page 26]
