Internet Engineering Task Force Audio Visual Transport WG Internet-Draft C.Roux,E.Gouleau,D.Curet/ P.Clement Document1 France telecom / Thomcast March, 09 2000 Expires: September, 09 2000 RTP Payload Format for Flexmultiplexed MPEG-4 Streams draft-rgcc-avt-mpeg4flexmux-00.txt STATUS OF THIS MEMO This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as refer- ence material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This document describes an RTP payload format for the transportation of encoded and multiplexed MPEG-4 streams. MPEG-4 is a standard whose aims are to define a generic way to code natural or synthetic scenes. MPEG-4, among other things, supports a normative interface to Intel- lectual Property Management and Protection Systems. MPEG-4 also gives methods to synchronise and multiplex several MPEG-4 encoded streams. RTP, on the other hand, is a protocol that has been especially writ- ten for multimedia stream over the IP network, and to use RTP for the carriage of MPEG-4 data does make sense, enabling such applica- tions as Video on demand or Multicast Teleconferencing. This specification is produced by members of the IST/OPENISE proj- ect. C.Roux,E.Gouleau,D.Curet,P.Clement [Page 1] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 Comments are solicited and should be addressed to the working group's mailing list at openise-platform@telecom.ntua.gr and/or the authors. 1 Introduction MPEG-4 is a recent standard from ISO/IEC for the coding, the syn- chronization and the multiplexing of audiovisual objects that can be described into an audiovisual scene by means of a scene description and that can be linked to compressed natural and synthetic audio- visual data by means of Object Descriptors[1][2][3][4]. The MPEG-4- standard can be seen as a complex toolbox providing compression, synchronization and multiplexing tools. This draft specifies an RTP [5] payload format for transporting multiplexed MPEG-4 encoded data streams. 2 Glossary: 2.1 general: The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [6]. 2.2 MPEG-4 glossary: AU: Access Unit Bifs: Binary format for scene DMIF: Delivery Multimedia Integration Framework, DAI: DMIF Application Interface, ES: Elementary stream, ESI:Elementary stream Interface, FlexMux: Flexible Multiplex. IPMP: Intellectual Property Management and Protection, OCI: Object Content Information, OD: Object descriptor, SL: Synchronisation layer, 3 MPEG-4 presentation: 3.1 MPEG4 layered architecture The MPEG-4 standard (ISO/IEC 14496) can be represented in a layered architecture, where three layers can be identified as follows: C.Roux,E.Gouleau,D.Curet,P.Clement [Page 2] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 +---------------------------------------+ media aware | COMPRESSION LAYER: | and | Elementary Streams (ES) encoding | delivery unaware| MPEG-4 part 2 Visual | layer | MPEG-4 part 3 Audio | | MPEG-4 part 1 Bifs,OD,IPMP,OCI | +---------------------------------------+Elementary Stream ==========================================================Interface (ESI) +---------------------------------------+ media and | SYNC LAYER (SL) | delivery unaware| Elementary streams management | layer | and synchronisation | +---------------------------------------+ DMIF Application ========================================================Interface (DAI) +---------------------------------------+ delivery aware | DELIVERY LAYER (DMIF) | media unaware |provides Flexmultiplexing of SL streams| layer | and transparent access | | to the delivery technology | +---------------------------------------+ Fig.1 the MPEG-4 layered architecture If the Delivery Layer mostly focuses on the control plan, it also encompasses a multiplexing tool, called the Flexmux, that describes a bitstream syntax used to multiplex MPEG-4 SL streams. It has also to be stated that the reconstruction of the correct timing of an MPEG-4 bitstream is supported both by the MPEG-4 SL stream syntax and by the MPEG-4 FlexMux stream syntax. The reconstruction of the correct timing of a MPEG-4 Flexmux stream is possible under some QoS constraints closely related to the reduction of the network jit- ter. MPEG makes the assumption that the carriage of MPEG-4 Flexmux streams over the network should affect any packets with a nearly constant transmission delay. The reconstruction of the correct tim- ing of the MPEG-4 Flexmux streams is based on the fact that this as- sumption can be verified, in accordance with the required accuracy of the reconstructed timing of the MPEG-4 FlexMux stream. Along this document will be specified an RTP payload format to en- able the carriage of Flexmux streams. The reconstruction of the FlexMux stream timing being possible under particular network QoS constraints. 3.2 Overview of MPEG-4 End-System Architecture The above Fig. 1 showed the general layered architecture of MPEG-4 terminals. C.Roux,E.Gouleau,D.Curet,P.Clement [Page 3] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 The Compression Layer processes the traditional individual audio/visual elementary streams (ES) and some associated "systems" elementary streams (ES) such as Bifs, OD, IPMP and OCI elementary streams. The MPEG-4 audio/visual ES syntaxes are defined in [2] and [3]. The "systems" ES syntaxes are described in [1]: the Bifs ES syntax allows a dynamic scene description. The OD ES syntax allows the de- scription of the hierarchical relations, location and properties of different ESs through a dynamic set of Object Descriptors. The "system" ES may require to be carried with a better protection than the traditional audio/visual ESs. The compressed content produced by this layer are organised into Elementary Streams (ESs). The compression layer is unaware of a spe- cific delivery technology, but it can react to the characteristics of a particular delivery layer such as the path-MTU or packet loss or bit error characteristics. The MPEG-4 SL stream syntax is defined in [1]. It provides a unique and homogeneous encapsulation of any of the MPEG-4 ESs. That layer primarily provides the synchronisation between ESs. ESs are organized in Access Units (AU). An AU is the smallest ele- ment to which timestamps can be assigned . Integer or fractional AUs are then encapsulated in SL packets. The MPEG-4 Delivery Layer consists of the Delivery Multimedia Inte- gration Framework defined in [4]. This layer is media unaware but delivery technology aware. It provides transparent access to and de- livery of content irrespective of the technologies used. The DAI interface between the SL layer and the DMIF layer is called the DMIF Application Interface. This interface supports content location in- dependent protocols firstly for establishing the MPEG-4 session and secondly for accessing to transport channels. DMIF monitors trans- port channels on the QoS requirements assigned to the SL streams, and supports the multiplexing of the SL streams, by the means of the MPEG-4 FlexMux tool. MPEG makes the assumption that the carriage of MPEG-4 Flexmux streams over the network should affect packets with an "ideal" con- stant transmission delay; the reconstruction of the correct timing of the MPEG-4 Flexmux streams is based on the that assumption. The specification of this payload format is a part of the MPEG-4 De- livery Layer. It can be presented as an instance of the MPEG-4 De- livery layer. 3.3 MPEG-4 Elementary Stream Data Packetization The ESs are a succession of Access Units (AUs). The AUs produced by the encoders are passed to the SL layer either complete or segmented through the ESI interface. Each segment is signalled with indica- tions of AU boundaries, random access points, and timestamp samples C.Roux,E.Gouleau,D.Curet,P.Clement [Page 4] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 such as the desired AU's decoding time , arrival time and the compo- sition time of the decoded AU. The Sync Layer inserts the complete or segmented AUs s into SL packets. The syntax of the SL packet header can be adapted to the needs of the ES to be carried. The configuration for each individual SL packet header is conveyed in an SLConfigDescriptor. The SL streams can be FlexMultiplexed into FlexMux streams. A Flex- Mux stream is a succession of FlexMux packets. Each FlexMux packet is built from a fixed length FlexMux packet header and a FlexMux packet payload. The FlexMux packet header is composed of a one byte index followed by a one byte length. The index identifies the FlexMux packet in the FlexMux stream. It also indicates whether the FlexMux packet manage- ment follows a MuxCode Mode or a Simple Mode. In the Simple Mode, the index is assigned to only one SL stream, and the FlexMux packet payload is composed of only one complete SL packet. + first + second +============================================| + byte + byte +---- length*bytes-------------------------- | +===============================================================+ | index | length | SL packet | +===============================================================+ +=========================================== + | SL Header | SL Payload | +=========================================== + Fig.2 the MPEG-4 FlexMux packet in Simple mode In the MuxCode Mode, the index can be assigned to several SL streams, and the FlexMux packet payload is assigned to the carriage of several complete fixed length SL packets. In the MuxCode Mode, a version field on four bits is added to the header with four reserved bits set to 1. + first + second + third +===============================| + byte + byte + byte +--------length*bytes-----------| +==========================================================+ | index | length |ver.|res|SL packet1 | ... |SL packetn | +==========================================================+ +===============================| |H1|payload1| ... |Hn|payloadn| +===============================+ Fig.3 the MPEG-4 FlexMux packet in MuxCode mode Ver. (version) (4 bits): the version of the referenced MuxCode table. res (reserved) (4 bits): 4 reserved bits set to 1 C.Roux,E.Gouleau,D.Curet,P.Clement [Page 5] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 The assignment of the index values to the different SL packets is described by the means of a MuxCode Mode table. That MuxCode Mode table should be provided by out-of-band means (e.g. SDP). In the Simple Mode, the "degradationPriority" field existence is the only restriction applied to the SL packet header, if IP packet marking is one mandatory issue. The fixed length FlexMux packet header enables to identify the sensitive parts of the embedded ES, as well as this is possible with SL packets themselves. The SL packet header starts two bytes after the FlexMux packet header. In the MuxCode Mode, the IP packet marking based on the "degradationPriority" field present in each SL packet header will only be coherent with all the SL packets carried in the same FlexMux packet should the same value be assigned to their "degradationPriority" field. Consequently, the value of the first "degradationPriority" field should be identical in all the headers of the following SL packets carried within a same FlexMux packet. A particular index (=238) is reserved to identify a particular Flex- Mux packet. This "238" packet is dedicated to carry a FlexMux Clock Reference time stamp (the FCR, indicating its arrival time) and the bitrate of the FlexMux stream. As far as bitrate is concerned, Flex- Mux streams are piecewise constant bitstreams. FlexMux streams are very similar to what has been already standard- ised with the MPEG-1 system syntax and with the MPEG-2 program syn- tax. Conversion between these three standards is trivial. 3.4 protection supported by the MPEG-4 Elementary Stream Data Packeti- zation By the means of vital ES header repetition, the SL layer can allow the insertion of random access points. This is not trivial as it may violate some MPEG-4 buffer model constraints. By the means of SL packet repetition, the SL layer can allow to du- plicate a particular SL packet. This feature can be used, for in- stance to repeat the SL packet carrying the beginning of an I-VOP. 4 A possible presentation of Applications: Applications can be sorted in several classes, among which three classes can be identified, according to the MPEG-4 layers they are concerned with. Any application may want to take benefit from the functionnalities offered by any of the three layers of the MPEG-4 model. Starting from the compression layer, a first class of application, that can be called the Class 'A' applications, can benefit from the C.Roux,E.Gouleau,D.Curet,P.Clement [Page 6] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 compression layer, and can be characterised by the way they tackle the protection issue. This is separated into two different classes: 1. Class 'A1' applications, with a protection coming from the com- pression layer: the draft proposal draft-ietf-avt-mpeg4-es-00 that describes RTP payload formats for the carriage of MPEG-4 Audio and Visual Elementary streams, and an RTCP format for MPEG-4 upstream messages functionalities. In this draft proposal , MPEG-4 Audio/Visual Elementary streams are directly mapped into RTP pack- ets without using MPEG-4 Systems[1]. Such class 'A1' applications focus on video and audio transport, and rely on a protection mechanism that is provided by the compression layer itself. This draft uses the Simple Visual Profile. 2. Class 'A2' applications, with a protection coming from the deliv- ery layer: A previous draft proposal has to be mentioned: the draft proposal draft-guillemot-genrtp-01. It stemmed from the large variety of MPEG-4 compressed streams and the large variety of error control mechanisms that can be applied to them. Such class 'A2' applications do not focus only on video and audio transport. They rely on a protection mechanism that is not pro- vided by the compression layer but by the RTP payload. A second class 'B' of application can benefit from the two first MPEG-4 layers: the compression layer and the synchronisation layer. They can also be characterised by the way they tackle the protection issue. They can be separated into two different classes: 1. Class 'B1' applications: the draft proposal-ietf-avt-rtp-mpeg4-02 that describes a payload format for transporting MPEG-4 encoded data (Synchronisation Layer streams) using RTP. Such class 'B1' applications do not focus only on video and audio, but also on the other MPEG-4 system ESs. The protection mechanism can be provided both at the compression layer and at the SL layer. That payload format does not support the reconstruction of the correct timing of an MPEG-4 SL stream. 2. Class 'B2' applications: The generic draft draft-guillemot-genrtp- 01.txt, previously mentioned, can still be applied. Such class 'B2' applications rely on a protection mechanism that is not pro- vided by the compression layer but by the RTP payload. A third class 'C' of applications can take advantage of the three MPEG-4 layers, as it is presented in the remaining of the current proposal. There is no draft proposal to give support to that kind of Class 'C' Applications. 5 Advantages and disadvantages that class 'C' applications can inherit from the use of the three MPEG-4 layers: 5.1 Some of the advantages : C.Roux,E.Gouleau,D.Curet,P.Clement [Page 7] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 1. The use of one port per elementary stream can not be much cost ef- fective, both on the server side and on the client side in terms of performance, when the number of elementary streams will in- crease within a scene. 2. The use of one single port for a Flexmultiplexed bitstream enables to send on this port a bunch of ESs that are tightly synchronised together, some of these ESs being themselves Bifs and OD ESs when a scene description is used with Audiovisual ES, some other ESs being OCI ES, and even IPMP ES when such systems are involved. 3. The use of the Flexmux technology enables possible interconnection between internet network and digital television network, where MPEG normatively defines the use the MPEG-4 Flexmux syntax to carry MPEG-4 signals over MPEG-2 transport channels [8]. 4. The reconstruction of the correct timing of the Flexmux stream is possible, if some QoS requirements are supported. 5. IP packet marking can be done in the same efficient way as it is possible to mark them when carrying SL streams. 6. The overall receiver buffer size reduction in terms of MPEG-4 buffers, as MPEG-4 compliant Flexmultiplexed streams, by the use of the MPEG-4 timestamps, respect the MPEG-4 system decoder model. 7. The overall bandwidth management is easier. The Flexmux syntax al- lows to have piecewise constant Flexmux bitstream, with an inband signalling mechanism. 8. Protection can be enhanced by means of repetitions of vital SL packets. When the fourth (4.) listed advantage has to be satisfied, one con- straint for the application is that the object time base of each MEG-4 Elementary Stream Flexmultiplexed together shall be locked. As an example, to be carried over MPEG2 transport channels [ 8], the object time base of each MPEG-4 Elementary Stream multiplexed in the same FlexMux stream shall be locked to the MPEG2 System Time Clock (27 000 000 Hz). The fifth (5.) listed advantage can be reached as the "degradationPriority" field present in each SL packet header is only two bytes further in a FlexMux packet header than it is in a SL packet header. Some constraint also apply to the Flexmux mode, see the º3.2 on "MPEG-4 Elementary Stream Data Packetisation" for clari- fication. The "degradationPriority" field enables, at the SL level and at the FlexMux level to identify the sensitive parts of the em- bedded ES, on a packet per packet granularity. The eighth (8.) listed advantage is a feature that is supported by the SL layer, see º3.4 about the "protection supported by the MPEG-4 Elementary Stream Data Packetization". The RFC 2343 (RTP Payload Format for Bundled MPEG) has to be men- tioned, even if it was designed to carry MPEG-2 (and not MPEG-4) video and audio ESs. That RFC 2343 payload format does not aim at timing reconstruction. C.Roux,E.Gouleau,D.Curet,P.Clement [Page 8] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 5.2 Some of the disadvantages: The first disadvantage with the packetisation of the MPEG-4 Flexmul- tiplexed streams is the added packet header overhead. This is the fact because MPEG-4 does not support a reduction mechanism of the carried MPEG-4 Flexmultiplexed streams packet headers. This issue needs certainly be resolved using a mechanism similar to what was once proposed with GeRM [7] in November 98. During their transport, FlexMux packets need not be compliant to the MPEG-4 standard, it is only when they are delivered to the Flexdemultiplexer that the com- pliance point is defined. The second disadvantage comes from the protection point of view. It appears that protection can only rely on the compression layer in some cases (for Visual there are only few profiles supporting pro- tection). When an application is not only interested in the tradi- tional AudioVisual ESs, but also interested in the "system" ESs, it is obvious that MPEG-4 system does not provide any means to protect the "scene" Ess (Bifs and OD), unless SL packet and FlexMux packet duplication which is somehow an incomplete protection mechanism. 6 The benefits of using RTP for MPEG-4 FlexMux stream transport in- clude: a. Ability to synchronise MPEG-4 FlexMux streams with other RTP payloads. MPEG-4 FlexMux streams and other real-time data streams received from multiple end-systems can be combined into a set of consolidated streams through RTP mixers. b. Monitoring MPEG-4 delivery performance through RTCP. c. the mandatory delivery, on the receiving side, of packets in their sending order. d. the use of RTP error resilience tools (likeFEC), supported by some already defined RTP payloads, that will be needed for MPEG-4 ESs ( "system", and some AudioVisual). 7 MPEG-4 FlexMux streams over RTP: This section specifies the RTP packetization rule for MPEG-4 FlexMux streams. 7.1 The RTP packet header : An MPEG-4 FlexMux stream is mapped directly onto the RTP payload without any addition of extra header fields or removal of any FlexMux packet header syntaxic elements. Each RTP packet will contain a timestamp derived from the sender's clock reference. This clock is synchronised to the FlexMux Clock C.Roux,E.Gouleau,D.Curet,P.Clement [Page 9] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 Reference (FCR) and represents the target transmission time of the first byte of the RTP packet payload. On the receiving side, the RTP packet timestamp will not be passed to the MPEG-4 Flexdemultiplexor. This use of the timestamp is slightly different from the normal use in RTP, in that it is not considered to be the media display time- stamp. The first purpose of this RTP timestamp will then be to re- duce (after estimation) the network jitter, and the relative time drift between the transmitter and the receiver. There are packetization restrictions due to the fact that no syn- chronisation pattern is part of the FlexMux packet header: An RTP packet payload should start with a FlexMux packet. An RTP packet will contain an integer number of FlexMux packets. The mapping between The ES identifiers and the FlexMux channels should be provided by out-of-band means (e.g. SDP). The IP packet marking facility may be needed. If this is the case, as it is based on the "degradationPriority" field present in each FlexMux packet, all the FlexMux packets grouped in the same RTP packet should have the "degradationPriority" field filled with the same value. Access Units are sent in their decoding order. SL packetisation and FlexMux packetisation is done accordingly.The size of the FlexMux packets should be adjusted such that the resulting RTP packet (em- bedding one or several FlexMux packets) is not larger than the path- MTU. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC |M| PT | sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | synchronization source (SSRC) identifier | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | contributing source (CSRC) identifiers | | .... | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | RTP payload | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 4 - An RTP packet for MPEG-4 FlexMux stream 7.2 RTP header fields usage for MPEG-4 FlexMux streams: Payload Type (PT): The assignment of a particular RTP payload type to this new packet format is outside the scope of this document, and is not specified here. If the dynamic payload type assignment is used, it can be specified by some out-of-band means (e.g. SDP) that the MPEG-4 FlexMux payload format is used for the corresponding RTP packets. C.Roux,E.Gouleau,D.Curet,P.Clement [Page 10] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 Marker (M) bit: set to zero. Extension (X) bit: Defined by the RTP profile used. Sequence Number: Increment by one for each RTP data packet sent. It starts with a random initial value for security reasons. Timestamp: it represents the target transmission time for the first byte of the RTP packet. Unless specified by an out-of-band means (e.g. SDP), the resolution of the timestamp is set to its default (90KHz). SSRC, CC and CSRC fields are used as described in RFC 1889 [5]. 7.3 the RTP Payload: The RTP payload format is the way to improve the transport of Flex- Mux streams by using protection mechanisms. There are several mechanisms which have been presented. Among them, a first one is described in the RFC 2733: "An RTP Payload Format for Generic For- ward Error Correction", J. Rosenberg and H. Schulzrinne, December 1999 and a second one was the recently proposed draft-guillemot- genrtp-01. With FlexMux protection, what is needed, for FlexMux packet protec- tion is to have a mechanism that can work on a sequence of packets, but also on a packet per packet basis. Two consecutive FlexMux pack- ets may require different protections as they may carry info from different ESs: even no protection for the first one , a high protec- tion for the second one, or vice versa: 1. The RFC 2733 is rather adapted to a large sequence of packets hav- ing the same protection requirements, and does not seem able to make distinction between the media carried within each packet. That RFC 2733 could certainly be adapted. 2. The genrtp-01 draft did not take into account FlexMux packets, and is now presented in a new version [9] adapted to FlexMux streams. What is here suggested is to define objects in the RTP payload, as it is proposed in [9]. The RTP payload being a succession of ob- jects. Each object is byte aligned. Each object starting with its length. A payload object is either a protection data object, or a complete FlexMux packet. Each payload object follows an identification byte, its object type byte. The RTP payload starts with one object type byte. C.Roux,E.Gouleau,D.Curet,P.Clement [Page 11] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 7.3.1 The object type byte syntax: 0 1 2 3 4 5 6 7 +-+-+-+-+-+-+-+ |G|E| XT | +-+-+-+-+-+-+-+ Figure 5 -the object type byte syntax G (Group) (1 bit): If this field is 1, it indicates that the payload object associated to the object type byte is followed by another payload object. If this field is 0, it indicates that the payload object associated to the current header is the last payload object of the payload. E (Extension) (1 bit): If its value is 1 then the next payload object contains protection data. If its value is 0, then the next payload object contains a FlexMux packet. XT (Extension type) (6 bits): This field is only meaningfull if E is set to 1 [9]. It then specifies the type of protection data. Examples of types will be FEC data with the specification of the FEC coding scheme (parity codes, block codes such as Reed Solomon codes,...), redundant data with the specification of the redundant data encoding scheme, duplicated high priority - e.g. headers - data,...etc. 7.3.2 Two RTP payload examples with one FlexMux packet: without protection data: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 |0 |xxxxxx | | +-+-+-+-+-+-+-+ | | | | FlexMux packet | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 6: An RTP payload for one FlexMux packet without protection data C.Roux,E.Gouleau,D.Curet,P.Clement [Page 12] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 with protection data: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1|1|XT=FEC | | +-+-+-+-+-+-+ | | Protection Data | | -+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+_ | |0|0| xxxxxx | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | | FlexMux Packet | +-+-++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+ Figure 7: An RTP payload for one FlexMux packet with protection data 8 Other technical issues: 8.1 IP packet marking: The Type of Service field (DS byte) of the IP packet can be initial- ised, on a FlexMux packet per packet basis, taking into account the degradation-priority field to be found in the SL packet header which starts two bytes after the beginning of the FlexMux packet header. 8.2 IP packet overhead reduction: This issue has to be investigated, but the problem can find reason- nable solutions, as the SL packet have, them selves, relatively stable configurations. 9 Security Considerations: RTP packets transporting information with the proposed payload for- mat are subject to the security considerations discussed in the RTP specification [1]. This implies that confidentiality of the media streams is achieved by encryption. If the entire stream (extension data and FlexMuxdata) is to be se- cured and all the participants are expected to have the keys to de- code the entire stream, then the encryption is performed in the usual manner, and there is no conflict between the two operations (encapsulation and encryption). C.Roux,E.Gouleau,D.Curet,P.Clement [Page 13] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 The need for a portion of stream (e.g. extension data) to be en- crypted with a different key, or not to be encrypted, would require application level signaling protocols to be aware of the usage of the XT field, in the object type byte, and to exchange keys and ne- gotiate their usage on the media and extension data separately. 10 Acknowledgments: This work is based on discussions that took place in the common IETF/MPEG Ad Hoc group. We would like to thank Steve Casner and Carsten Herpel, the two chairmen for the cooperative work they conduct. Thanks also to Christine Guillemot for the discussions that we had during the COMIQS project. 11 References: [1] ISO/IEC 14496-1 MPEG-4 Systems December 1999 [2] ISO/IEC 14496-2 MPEG-4 Visual December 1999 [3] ISO/IEC 14496-3 MPEG-4 Audio December 1999 [4] ISO/IEC 14496-6 Delivery Multimedia Integration Framework, De- cember 1999. [5] H.Schulzrinne, S.Casner, R.Frederick, V.Jacobson "RTP: A Transport Protocol for Real Time Applications" RFC 1889, Internet Engineering Task Force, January 1996. [6] S. Bradner, Key words for use in RFCs to Indicate Requirement Levels, RFC 2119, March 1997. [7] M. Handley, "GeRM: Generic RTP Multiplexing," work in prog- ress, draft-ietf-avt-germ-00.txt, November 1998. [8] ISO/IEC Amendment 7: Transport of ISO/IEC 14496 data over ISO/IEC 13818-1: N3050 document, January 2000 [9] C. Guillemot, P. Christ, S. Wesner, A. Klemets, "RTP payload format for MPEG-4 with scaleable and flexible error resiliency", draft-guillemot-avt-genrtp-02.txt, March 2000. 12 Authors' Addresses Catherine Roux, Dominique Curet, Emmanuel Gouleau, France Telecom R&D catherine.roux@cnet.francetelecom.fr dominique.curet@cnet.francetelecom.fr emmanuel.gouleau@cnet.francetelecom.fr C.Roux,E.Gouleau,D.Curet,P.Clement [Page 14] Internet-Draft RTP Payload Flexmultiplexed MPEG-4 Streams 10/03/00 Pierre Cl‰ment Thomcast Pierre.clement@thomcast.thomson-csf.com C.Roux,E.Gouleau,D.Curet,P.Clement [Page 15]