Audio/Video Transport P. Yang Internet-Draft F. Xia Intended status: Standards Track X. Wu Expires: April 25, 2011 Huawei Technologies Co., Ltd. October 22, 2010 Preamble Acquisition of MPEG2-TS Multicast Sessions draft-xia-avt-mpeg2ts-preamble-03.txt Abstract The ITU-T Rec. H.222.0 | ISO/IEC 13818-1 Transport Stream (MPEG2-TS) addresses the combining of one or more elementary streams of video and audio, as well as other data, into single or multiple streams which are suitable for storage or transmission. The necessary and sufficient information contained in the Program Specific Information(PSI) tables to demultiplex and present programs must be acquired before a RTP receiver can process any data received in MPEG2-TS. In this document, a Retransmission Server is specified to deliver MPEG2-TS preamble prior to unicast burst RTP packets. The Retransmission Server caches raw RTP packets with MPEG2-TS preamble information, and sends them to the RTP receiver which initiates rapid acquisition of MPEG2-TS multicast sessions. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on April 25, 2011. Copyright Notice Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Yang, et al. Expires April 25, 2011 [Page 1] Internet-Draft Preamble Acquisition October 2010 Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Abbreviation . . . . . . . . . . . . . . . . . . . . . . . . . 4 4. Overview of MPEG-2 Transport Streams . . . . . . . . . . . . . 5 5. Preamble Acquisition of MPEG2-TS Multicast Sessions . . . . . 8 5.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . 8 5.2. MPEG2-TS Preamble Structure . . . . . . . . . . . . . . . 11 5.3. The Implementation of MPEG2-TS Preamble . . . . . . . . . 12 5.3.1. PCR information . . . . . . . . . . . . . . . . . . . 13 5.3.2. PES header information . . . . . . . . . . . . . . . . 13 5.3.3. Discontinuity indicator information . . . . . . . . . 14 5.3.4. Elementary Video Encoding Information . . . . . . . . 14 5.3.5. Other extended PSIs Information . . . . . . . . . . . 15 5.4. RTP Header of reconstructing new RTP packets . . . . . . . 15 5.5. Message Flow . . . . . . . . . . . . . . . . . . . . . . . 16 6. The timing of the MPEG2-TS Preamble . . . . . . . . . . . . . 19 7. Security Considerations . . . . . . . . . . . . . . . . . . . 20 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 20 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 20 9.1. Normative References . . . . . . . . . . . . . . . . . . . 20 9.2. Informational References . . . . . . . . . . . . . . . . . 22 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 22 Yang, et al. Expires April 25, 2011 [Page 2] Internet-Draft Preamble Acquisition October 2010 1. Introduction Real-time multimedia multicast flows usually carry streams of inter- related data. Certain information must first be acquired by the receivers to start processing multimedia data sent in the multicast session. [I-D.ietf-avt-rapid-acquisition-for-rtp] refers to this information as Reference Information. Part of the Reference Information is conventionally sent periodically in the multicast session and usually consists of items such as a description of the schema for the rest of the data, references to which data to process, encryption information including keys, as well as any other information required to process the data in the primary multicast stream. The ITU-T Rec. H.222.0 | ISO/IEC 13818-1 Transport Stream[MPEG2-TS] is a stream definition which is tailored for communicating or storing one or more programs of coded data according to ITU-T Rec. H.262 | ISO/IEC 13818-2 and ISO/IEC 13818-3 and other data in environments. The MPEG2-TS is already widespread in the digital broadcasting and IPTV services over both terrestrial and satellite networks, not only in Europe but also in Asia and North America. When the primary multicast stream is using an encapsulation method of MPEG2-TS over RTP that multiplexes video and audio content, together with ancillary metadata, and produces a synchronized multiplexed stream, the receiver must first acquire the necessary and sufficient information before demultiplexing and decoding an incoming MPEG2-TS. However, these necessary and sufficient information of MPEG2 Transport Stream does not reside in MPEG2-TS contiguously and is usually dispersed over a large period of time. When the receivers starts receiving the Reference Information from the random access point as detailed in [I-D.ietf-avt-rapid-acquisition-for-rtp], the Reference Information of MPEG2-TS cannot be completely received particularly the MPEG2-TS Program Specific Information (PSI) tables so that the receivers cannot demultiplex and decode correctly. In order to demultiplex and decode correctly for the receivers, parts of the Reference Information of MPEG2-TS, extracted from MPEG2 transport stream packets prior to the random access point(RAP) over a long period( even several previous Reference Information away), must be acquired before demultiplexing and decoding. In this document, we refers to this information as MPEG2-TS preamble. This document describes how to convey this preamble information to the receiver before the unicast burst stream defined in RAMS [I-D.ietf-avt-rapid-acquisition-for-rtp]. Yang, et al. Expires April 25, 2011 [Page 3] Internet-Draft Preamble Acquisition October 2010 2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 3. Abbreviation The following abbreviations are used in this document: Yang, et al. Expires April 25, 2011 [Page 4] Internet-Draft Preamble Acquisition October 2010 CAT: Conditional Access Table ECM: Entitlement Control Message EMM: Entitlement Management Message GoP: Group of Picture IDR: Instantaneous Decoding Refresh IPMP: Intelligent Property Management and Protection MP2T: MPEG2 Transport Stream MPEG2-TS: MPEG2 Transport Stream NAL: Network Abstract Layer NIT: Network Information Table PAT: Program Association Table PCR: Program Clock Reference PES: Packetized Elementary Stream PMT: Program Map Table PPS: Picture Parameter Set PS: Private Section PSI: Program Specific Information RAP: Random Access Point SEI: Supplemental Enhancement Information SPS: Sequence Parameter Set TSDT: Transport Stream Description Table 4. Overview of MPEG-2 Transport Streams ITU-T Rec. H.222.0 | ISO/IEC 13818-1 Transport Stream addresses the combining of one or more elementary streams of video and audio, as well as other data, into single or multiple streams which are Yang, et al. Expires April 25, 2011 [Page 5] Internet-Draft Preamble Acquisition October 2010 suitable for storage or transmission. Systems coding follows the syntactical and semantic rules imposed by this Specification and provides information to enable synchronized decoding of decoder buffers over a wide range of retrieval or receipt conditions. The basic multiplexing approach for single video and audio elementary streams is illustrated in Figure 1. The video and audio data is encoded as described in ITU-T Rec. H.262 | ISO/IEC 13818-2 and ISO/ IEC 13818-3. The resulting compressed elementary streams are packetized to produce Packetized Elementary Streams(PES) packets. video +-------------+ +----------+ Video PES +---------+ input--->|Video Encoder|--->|Packetizer|---------->| | +-------------+ +----------+ | | | | Audio +-------------+ +----------+ Audio PES |MPEG-2 | input--->|Audio Encoder|--->|Packetizer|---------->| | Network +-------------+ +----------+ |Transport|----> | | Optional Application Data(e.g subtitle)---> |Stream | | | Program Specific Information---> |Mux | | | ... ...---> | | +---------+ Figure 1: The overview of MPEG-2 Transport Streams The MPEG2 Transport Stream coding layer allows one or more programs to be combined into a single stream. Data from each elementary stream are multiplexed together with information that allows synchronized presentation of the elementary streams within a program. A Transport Stream consists of one or more programs. Audio and video elementary streams consist of access units(A coded representation of a presentation unit). Elementary Stream data is carried in PES packets. A PES packet consists of a PES packet header followed by packet data. PES packets are inserted into Transport Stream packets. The first byte of each PES packet header is located at the first available payload location of a Transport Stream packet. Transport Stream packets begin with a 4-byte prefix, which contains a 13-bit Packet ID (PID). The PID identifies, via the Program Specific Information (PSI) tables, the contents of the data contained in the Transport Stream packet. Transport Stream packets of one PID value Yang, et al. Expires April 25, 2011 [Page 6] Internet-Draft Preamble Acquisition October 2010 carry data of one and only one elementary stream. The PSI tables are carried in the Transport Stream. There are Six PSI tables shown in table 2.28 of H.222.0(2006) (There were only the first four PSI tables in the ITU-T Rec. H.222.0(2000 E) | ISO/IEC 13818-1:2000 E): o Program Association Table; o Program Map Table; o Conditional Access Table; o Network Information Table; o Transport Stream Description Table; o IPMP Control Information Table. These tables contain the necessary and sufficient information to demultiplex and present programs. Program Association Table contains the PIDs of NIT and PMT, what's more, the private section in Transport Stream packets with a PID value is designated as a Program Map Table PID in the Program Association Table. The Program Map Table provides among other information, which PIDs, and therefore which elementary streams are associated to form each program. This table also indicates the PID of the Transport Stream packets which carry the PCR for each program. The Conditional Access Table shall be present if scrambling is employed. The Network Information Table is optional and its contents are not specified by the ITU-T Rec. H.222.0 | ISO/IEC 13818-1. Transport Stream Description Table(TSDT) has 256(an 8-bit field) transport stream description elements shown in Table 2-45 - Program and program element descriptors of H.222.0(2006). The IPMP Control Information Table shall be present if IPMP as described in ISO/IEC 13818-11 is used by any of the components in the ITU-T Rec. H.222.0 | ISO/IEC 13818-1 stream. Except for these PSI tables, ETSI DVB EN 300 468 and ATSC Document A/69 also define some extended PSIs and PSIP information. Other standard organizations and operators may define some extended PSI information. We need to consider which PSI information is necessary to demultiplex and decode. In this document, terminologies PSI and MPEG2-TS preamble are interchangeable. Media including video, audio and text in an MPEG2-TS is self describing, and the receiver must parse certain control information in the PAT, CAT and PMT tables (i.e., PSI) contained in the transport stream in order to know how to parse the rest of the stream (i.e., to find the audio and video elementary streams, private data and the encryption information for a given program). This document specifies a mechanism to acquire PSI rapidly when a receiver joins in a MPEG2-TS multicast session. Yang, et al. Expires April 25, 2011 [Page 7] Internet-Draft Preamble Acquisition October 2010 5. Preamble Acquisition of MPEG2-TS Multicast Sessions 5.1. Overview In video coding, a group of picture (GOP) specifies the order in which intra-frame and inter-frames are arranged. The GOP is a group of successive pictures within a coded video stream. Each coded video stream consists of successive GOPs. A GOP always begins with a Random Access Point (RAP)- an intra-frame(e.g. IDR-frame). Afterwards several inter-frames(e.g. P-frame) follow. The Intra- frame is a reference frame which contains the full image which represents a fixed image and which is independent of other picture types and do not require any additional information to reconstruct it. The inter-frame is a prediction frame which contains motion- compensated difference information from the preceding Intra-frame or other inter-frames in a video compression stream which is expressed in terms of one or more neighboring frames. When a receiver joins a primary multicast session, it does not have control over what point in the flow is currently being transmitted, it needs to wait until the next RAP and other necessary and sufficient information for demultiplexing and decoding (which we refer to as the Reference Information in the [I-D.ietf-avt-rapid- acquisition-for-rtp]) shows up in the multicast stream before it can start decoding. In order to reduce the acquisition delay when the RTP receiver joins a multicast session at a random point in time, RAMS in the [I-D.ietf-avt-rapid-acquisition-for-rtp] introduces the method as Unicast- based Rapid Acquisition of Multicast RTP Sessions illustrated in Figure 2 where an auxiliary unicast RTP session carrying the Reference Information to the receiver precedes/ accompanies the primary multicast stream. This unicast RTP flow is transmitted at a faster than natural rate to further accelerate the acquisition. In MPEG2 transport stream case, the Reference Information is often not contiguous in the flow but dispersed over a large period, it even spans several GOPs. It is often distributed in different RTP packets with other MPEG2-TS packets which are not part of the reference information. Figure 2 illustrates an RTP stream where PAT, PMT and ECM information distribute in the three different RTP packets. Yang, et al. Expires April 25, 2011 [Page 8] Internet-Draft Preamble Acquisition October 2010 ... RTP SN(28255) PAT(TS1) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... RTP SN(28266) PMT(TS6) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... RTP SN(28291) ECM(TS4) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... RTP SN(28301)(Random Access point: IDR-frame(TS6) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 2 Original RTP sequence only with PAT, PMT and ECM In Figure 2 the TS1 packet of RTP packet with sequence number 28255 is a PAT packet. The TS6 packet of RTP packet with sequence number 28266 is a PMT packet. The TS4 packet of RTP packet with sequence number 28291 is an ECM packet. TS6 of RTP packet with sequence number 28301 is a beginning of a Random Access point of IDR-frame. The Reference Information for this RTP sequence starting at RTP sequence number 28301 needs the PAT information in the RTP SN 28255, the PMT information of in the RTP SN 28266 and the ECM information of in the RTP SN 28291. The process is now to extracts the PSI information and to build a new RTP packet illustrated in Figure 3 where PAT is in TS1, PMT in TS2 and ECM in TS3. RTP SN(xxx) PAT(TS1),PMT(TS2),ECM(TS3) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ RTP(RFC4588) SN(28301)(Random Access point: IDR-frame(TS6) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... Figure 3 Preamble MPEG2-TS RTP sequence In MPEG2-TS PSI information, PAT, PMT, and ECM(ECM shall be present Yang, et al. Expires April 25, 2011 [Page 9] Internet-Draft Preamble Acquisition October 2010 if scrambling is employed) are always present. Other reference information enabling automatic configuration of the receiver to demultiplex and decode the various streams of programs within the multiplex are optionally provisioned depending on different policies and demand of demultiplexing and decoding. For example, the Conditional Access Table shall be present if scrambling is employed. The IPMP Control Information Table shall be present if IPMP as described in ISO/IEC 13818-11 is used by any of the components in the ITU-T Rec. H.222.0 | ISO/IEC 13818-1 stream. The Reference information (Time and Date Table (TDT), Service Description Table (SDT),etc.) in the ETSI DVB EN 300 468 shall be optionally present to provide identification of services and events for the user. The following RTP sequence illustrated in Figure 4 includes more PSI table elements. ... RTP SN(10739) CAT(TS2) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... RTP SN(10758) PAT(TS3) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... RTP SN(10796) PMT(TS3) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... RTP SN(10829) ECM(TS2) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... RTP SN(10834) PAT(TS4) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... RTP SN(10851) NIT(TS5) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... RTP SN(10872) PMT(TS3) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Yang, et al. Expires April 25, 2011 [Page 10] Internet-Draft Preamble Acquisition October 2010 |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... RTP SN(10910) PAT(TS2) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... RTP SN(10912) SDT(TS2),TDT(TS7) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ RTP SN(10913)(Random Access point: IDR-frame(TS1),ECM(TS2) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 4 Original RTP sequence with PAT, PMT, ECM, CAT, SDT... Figure 5 shows an example of MPEG2-TS preamble providing the RTP sequence with PAT, PMT ECM, NIT, CAT, SDT and TDT. SN(xxx) PAT(TS1),PMT(TS2),ECM(TS3),NIT(TS4),CAT(TS5),SDT(TS6),TDT(TS7) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ RTP(RFC4588) SN(10913)(Random Access point: IDR-frame(TS1),ECM(TS2) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP(H)|TS1(188)|TS2(188)|TS3(188)|TS4(188)|TS5(188)|TS6(188)|TS7(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 5 Preamble MPEG2-TS RTP sequence 5.2. MPEG2-TS Preamble Structure The order of PSI information need to be considered when rebuilding RTP packets. Except for some fixed PIDs(0x0000 for PAT, 0x0001 for CAT, 0x0002 for Transport Stream Description Table, 0x0003 for IPMP Control Information Table), other PIDs are inherited from those fixed PIDs. For example the PID of PMT is inherited from PAT. The PID of ECM is inherited from CAT, or PMT, or Transport Stream Description Table. The PIDs of Private_Section are designated as a Program Map Table PID in the Program Association Table. Figure 6 shows the inheritance of PAT and CAT. Yang, et al. Expires April 25, 2011 [Page 11] Internet-Draft Preamble Acquisition October 2010 +-+-+ +-+-+ |PAT| PID(0x0000) |CAT| PID(0x0001) +-+-+ +-+-+ | | -------------------- ------------ | | | | | +-+-+ +-+-+ +-+-+ +-+-+ +-+-+ |NIT| |PMT| | PS| Private Section |ECM| |EMM| +-+-+ +-+-+ +-+-+ +-+-+ +-+-+ | --------- | | +-+-+ +-+-+ |ECM| |EMM| +-+-+ +-+-+ Figure 6 The inheritance of PSI PIDs 5.3. The Implementation of MPEG2-TS Preamble The MPEG2-TS Preamble Information of the Reference Information is mainly composed of essential PSIs which affect the demultiplexing and decoding, and also include PES header and elementary coded video information if the Random Access Point-Intra-frame hasn't included this information. There are dozens of PSI tables of MPEG2-TS which include PAT, PMT, PCR, CAT, ECM/EMM, Transport Stream Description Elements of the 8-bit field, IPMP Control Information Table, Private Section. Except for these PSI tables defined in the ITU-T Rec. H.222.0 | ISO/IEC 13818-1, both ETSI DVB EN 300 468 and ATSC Document A/69 define some extended PSIs and PSIP information. Other standard organizations and operators may define some extended PSI information. The implementation need to consider which PSI information affect on the start-up of RAMS. The MPEG2-TS PSI information are not present simultaneously, and may be distributed in different RTP packets over a large period of time. They can be part of the preamble information or the RTP_Rx can receive them later from the unicast burst and the multicast stream. The important elements are the PAT, PMT, PCR and ECM/EMM and the implementation can decide which elements to include in the Preamble stream. Nevertheless, this document doesn't limit the implementation and it cab decide which are the important elements of PSIs information. In order to efficiently transmit the MPEG2-TS preamble information and not affect the start-up of RAMS, the implementation reconstructs new RTP packets by using the raw MPEG2-TS preamble information extracted from the MPEG2-TS packets sequence of the primary multicast Yang, et al. Expires April 25, 2011 [Page 12] Internet-Draft Preamble Acquisition October 2010 stream. Figure 7 illustrates the detailed process of reconstructing a RTP packet of MPEG2-TS preamble. Nevertheless, in the process of reconstruction, it is necessary to keep the MPEG2-TS Preamble Order described in the section 5.2 MPEG2-TS Preamble Structure in order to guarantee the demultiplexing and decoding correctly. +-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+ ...|RTP(H)|...TS2(ECM)...| ...|RTP(H)|...TS5(PAT)...|... +-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-| | | +------ + | | +-----------------------------+ | | +-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+ | |...|RTP(H)|...TS3(PMT)...| ...|RTP(H)|...TS5(PCR)...|... | | +-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+ | | | | | | +---+ +----+ +----|------------|------------+ | | | | | SN(xxx) v v v v +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |RTP (H)|TS1-PAT(188)|TS2-PMT(188)|TS3-ECM(188)|TS4-PCR(188)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 7 RTP Reconstruction Process of MPEG2-TS Preamble 5.3.1. PCR information There are two ways to transmit PCR information in MPEG2-TS stream. One is an independent PCR PID where the discontinuity indicator value is set to '0'. The other is by having the PCR information as the adaptation field inserted in TS packets of video and share the same PID as the video stream. Whether the PCR information is an independent PCR PID or shares a TS packet with video, the PCR information shall be transmitted as an MPEG2-TS preamble. 5.3.2. PES header information Normally the first TS packet of a Random Access Point (RAP)-intra- frame contains a PES Header following TS header, and then by video coded frame data. As is shown in Figure 8. +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |TS Header| PES Header | video coded frame data | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 8 The first TS packet of RAP with PES Header Yang, et al. Expires April 25, 2011 [Page 13] Internet-Draft Preamble Acquisition October 2010 However, in some cases of MPEG2-TS, several video frames share one PES header. It can happen that the first TS packet of the reference information of Random Access Point doesn't include PES header. As is shown in Figure 9. +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |TS Header| video coded frame data | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 9 The first TS packet of RAP without PES Header The PES Header is very important for the demultiplexing, RTP Receivers cannot demultiplex and decode correctly the MPEG2-TS stream before PES header. In this case, the PES Header must be an MPEG2-TS preamble information transmitted before RAMS. 5.3.3. Discontinuity indicator information In some cases, the field of discontinuity indicator of PSI information is set to '1' which indicates that the discontinuity state is true. When the discontinuity state is true in PSI packet of a PID not designated as a PCR_PID, the continuity counter in that packet may be discontinuous with respect to the previous Transport Stream packet of the same PID. After the occurrence of a discontinuity indicator set to '1' in the Transport Stream packets of the same PID which contains PSI information, these discontinuous Transport Stream packets all shall be transmitted as the MPEG2-TS preamble, otherwise it shall not be constructed in such a way that discarding such a packet will cause the loss of PES packet payload data or PSI data. When reconstructing these Transport Stream packets of PSI information, it is necessary to keep their original order. 5.3.4. Elementary Video Encoding Information Elementary Video Encoding Information (e.g. SPS(Sequence Parameter Set)/PPS(Picture Parameter Set) /SEI(Supplemental Enhanced Information in MPEG4/H.264 codec, Video Sequence information in MPEG2 codec, Sequence and Entry-Point information in VC-1 codec, etc.) is very important for the video decoding. Like the PES header, it may happen that several video frames share the elementary video coded information in MPEG2 transport stream so that an intra-frame may not contain elementary video encoding information. At this time, elementary video coded information shall be transmitted as part of the MPEG2-TS preamble. The Sequence Parameter Set is a kind of NAL unit and contains syntax elements that apply to zero or more entire coded video sequences as determined by the content of a seq_parameter_set_id syntax element Yang, et al. Expires April 25, 2011 [Page 14] Internet-Draft Preamble Acquisition October 2010 found in the picture parameter set referred to by the pic_parameter_set_id syntax element found in each slice header. The Picture Parameter Set is also a kind of NAL unit and contains syntax elements that apply to zero or more entire coded pictures as determined by the pic_parameter_set_id syntax element found in each slice header. The Video Sequence information (e.g. Sequence header, Extension and user data, Group of pictures header, etc.) is the highest syntactic structure of coded video bit streams and contains with a sequence header which may optionally be followed by a group of pictures header and then by one or more coded frames. The Sequence and Entry-Point information (e.g. sequence-level header, entry-point header ) are required for the decoder and shall be sent view the MPEG2-TS premable. 5.3.5. Other extended PSIs Information Except for essential PSIs information which affect on the demultiplexing and decoding, other PSIs are recommended to be transmitted if they don't exist after the reference information of random access point for a long time. 5.4. RTP Header of reconstructing new RTP packets The format of the RTP header of reconstructing new RTP packets is specified in RFC 3550 and reprinted in Figure 10 for convenience. This RTP payload format for MPEG2-TS preamble uses raw MPEG2-TS packets with the Reference Information from the original MPEG2-TS packets sequence in a manner consistent with that specification. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC |M| PT | sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | synchronization source (SSRC) identifier | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | contributing source (CSRC) identifiers | | .... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 10 RTP header according to RFC 3550 The RTP header information to be set according to this RTP payload format is set as follows: Marker bit (M): 1 bit Yang, et al. Expires April 25, 2011 [Page 15] Internet-Draft Preamble Acquisition October 2010 The Maker bit is set to one if there is only one RTP packet of MPEG2-TS preamble. The last RTP packet of MPEG2-TS preamble is set to one, others are set to zero if there are more than one RTP packet of MPEG2-TS preamble. Sequence number (SN): 16 bits The SN is set and used in accordance with RFC 3550. In order to make it easier for the RAMS-I message to convey the sequence number of the first message in the unicast channel which is a preamble packet the sequence numbers of the MPEG2-TS Preamble shall be equal to minus one by the first original sequence number of the unicast retransmission RTP packet if there is only one RTP packet of MPEG2-TS preamble. If there are n RTP packets, their sequence numbers are equal to minus one, two,..., n by the first original sequence number of the unicast retransmission RTP packet respectively. Note that the SSRCs of the preamble stream and the unicast burst are different. Timestamp: 32 bits The RTP timestamp is set to the sampling timestamp of the content. The timestamp of the MPEG2-TS Preamble shares the same timestamp space with primary multicast RTP packets, and is set to the previous RTP timestamp of the random access point(RAP) which is the first original sequence number of the primary multicast RTP packet. Synchronization Source (SSRC): 32 bits The SSRC identifier SHOULD be chosen randomly with collision detection, is not likely to choose the same number with primary multicast session. 5.5. Message Flow Figure 10 depicts message flows for MPEG2TS Preamble acquisition. A RTP receiver sends a rapid acquisition request for a new multicast RTP session to the feedback target address of that session. This RTCP feedback message is defined as the RAMS-Request (RAMS-R) message in [I-D.ietf-avt-rapid-acquisition-for-rtp] and MAY also contain parameters, such as the bandwidth limit, buffering capacity available at the RTP receiver, and so on. A retransmission server receives the RAMS-R message and sends an RAMS-Information (RAMS-I) message to the RTP receiver. The first RAMS-I message MAY precede the unicast burst or it MAY be sent during the burst. The retransmission server then starts sending unicast Yang, et al. Expires April 25, 2011 [Page 16] Internet-Draft Preamble Acquisition October 2010 MPEG2TS Preamble RTP packets prior to delivering Unicast RTP Burst. In this document, a unicast "MPEG2TS Preamble RTP" delivering process is added for transporting necessary MPEG2-TS preamble information. The unicast MPEG2TS Preamble RTP stream shall be transmitted before the Unicast RTP burst, as shown in Figure 10. The information is carried as RFC 2250 [RFC2250] RTP packets. ------------------------- | Retransmission Server | ----------- | ------ ------------ | -------- ------------ | Multicast | | | FT | | Burst/Ret. | | | | | RTP | | Source | | | | | Source | | | Router | | Receiver | | | | ------ ------------ | | | | (RTP_Rx) | ----------- | | | | -------- ------------ | ------------------------- | | | | | | | |-- RTP Multicast ---------->--------------->| | |-. RTCP Multicast -.-.-.-.->-.-.-.-.-.-.-.->| | | | | | | | | |********************************| | | |* PORT MAPPING SETUP *| | | |********************************| | | | | | | |<~~~~~~~~~~~~~~~~~~~~~~~~~ RTCP RAMS-R ~~~| | | | | | | | |********************************| | | |* UNICAST SESSION ESTABLISHED *| | | |********************************| | | | | | | | |~~~ RTCP RAMS-I ~~~~~~~~~~~~~~~>| | | | | | | | |... Unicast RTP Burst .........>| | | | | | | |<~~~~~~~~~~~~~~~~~~~~~~~~ (RTCP RAMS-R) ~~| | | | | | | | |~~ (RTCP RAMS-I) ~~~~~~~~~~~~~~>| | | | | | | | | | | | | | |<= SFGMP Join ==| | | | | | |-- RTP Multicast ------------------------------------------->| |-. RTCP Multicast -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.>| | | | | | | | | | | | | |<~~~~~~~~~~~~~~~ RTCP RAMS-T ~~~| | | | | | : : : : : Yang, et al. Expires April 25, 2011 [Page 17] Internet-Draft Preamble Acquisition October 2010 | | |<.=.= Unicast RTCP Reports .=.=>| : : : (for the unicast session) : | | | | | :::::::> MPEG2TS Preamble RTP -------> Multicast RTP Flow .-.-.-.> Multicast RTCP Flow .=.=.=.> Unicast RTCP Reports ~~~~~~~> Unicast RTCP Feedback Messages =======> SFGMP Messages .......> Unicast RTP Flow Figure 10: Message flows for MPEG2TS Preamble Acquisition The SDP[RFC4566] description of the MPEG2-TS Preamble information is conveyed in an RAMS session description as defined in [I-D.ietf-avt- rapid-acquisition- for-rtp]. Figure 11 shows an SDP example which multiplexes the MPEG2-TS Preamble information and an RAMS unicast retransmission session based on Section 8.2 of [I-D.ietf-avt-rapid- acquisition-for-rtp]. Yang, et al. Expires April 25, 2011 [Page 18] Internet-Draft Preamble Acquisition October 2010 v=0 o=S1 1122334455 1122334466 IN IP4 rams.example.com s=Rapid Acquisition Example with MPEG2-TS Preamble Data t=0 0 a=group:FID 3 4 a=rtcp-unicast:rsi m=video 41000 RTP/AVPF 98 i=Primary Multicast Stream c=IN IP4 233.252.0.2/255 a=source-filter: incl IN IP4 233.252.0.2 192.0.2.2 a=rtpmap:98 MP2T/90000 a=rtcp:41001 IN IP4 192.0.2.1 a=rtcp-fb:98 nack a=rtcp-fb:98 nack ssli a=ssrc:123321 cname:iptv-ch32@rams.example.com a=mid:3 m=video 41002 RTP/AVPF 99 100 i=Unicast Retransmission Stream + Preamble Data c=IN IP4 192.0.2.1 a=rtpmap:99 rtx/90000 a=rtcp:41003 a=fmtp:99 apt=98; rtx-time=5000 a=preamble 100 a=rtpmap:100 MPEG2TS-Preamble/90000 a=mid:4 Figure 11: The SDP description of the MPEG2-TS Preamble information conveying in an RAMS session 6. The timing of the MPEG2-TS Preamble The RTP preamble packets are received by the RTP receiver. Once received, there is a need to immediately feed the preamble information to the Transport Stream (TS) processor to enable the demultiplexing and decoding. Due to the timing of the TS processor, it is necessary to have a mechanism to ensure correctly processing of each TS packet of the preamble information. There are several mechanisms to do it. Here, we propose three mechanisms. 1. The RTP stack removes RTP header of the preamble information and passes the whole preamble information to the decoder, and the decoder is responsible for processing the preamble information correctly. 2. The RTP stack controls the timing of the preamble TS packets sent to the decoder. This timing control may include inserting dummy TS packets, repeating the TS preamble packets, or other means of pacing. Yang, et al. Expires April 25, 2011 [Page 19] Internet-Draft Preamble Acquisition October 2010 3. The RTP stack sends the preamble TS packets to the TS decoder, without inserting any dummy packet, at a rate that will allow the TS decoder to process them correctly. The choice of mechanism is implementation specific and depends on the TS processor requirements and available buffers. 7. Security Considerations Comparing to [I-D.ietf-avt-rapid-acquisition-for-rtp], this document specifies delivering PSI information to a RTP receiver prior to unicast RTP burst packets. PSI is key information for a decoder to parse MPEG2-TS streaming packets, and any tampered PSI results in denial of service of the RTP receiver. Security considerations in [I-D.ietf-avt-rapid-acquisition-for-rtp] also applies. 8. Acknowledgements The authors thank the following individuals for their contributions, comments and suggestions to this document: Roni Even, Sandy (Alexander) MacInnis, VAN CAENEGEM Tom, Franceschini Guido, Eric Friedrich, Ali C. Begen (abegen), Stephan Wenger, Colin Perkins, Ingemar Johansson S, Hui Deng, Martin Ellis, David R Oran. 9. References 9.1. Normative References [H.222.0] ITU-T H.222.0(2000 E), "Information technology - Generic coding of moving pictures and associated audio information - Part 1: Systems". [H.222.0(2006)] ITU-T H.222.0(2006), "Information technology - Generic coding of moving pictures and associated audio information - Part 1: Systems", May 2006. [H.262] ITU-T H.262(2000), "Information technology - Generic coding of moving pictures and associated audio information - Part 2: Video". [I-D.ietf-avt-rapid-acquisition-for-rtp] Steeg, B., Begen, A., Caenegem, T., and Z. Vax, "Unicast- Based Rapid Acquisition of Multicast RTP Sessions", draft-ietf-avt-rapid-acquisition-for-rtp-16 (work in Yang, et al. Expires April 25, 2011 [Page 20] Internet-Draft Preamble Acquisition October 2010 progress), October 2010. [ISO13818-1] ISO/IEC 13818-1:2000(E), "Information technology - Generic coding of moving pictures and associated audio information: Systems". [ISO13818-11] ISO/IEC 13818-11:2004, "Information technology - Generic coding of moving pictures and associated audio information - Part 11: IPMP on MPEG-2 systems". [ISO13818-1:2007] ISO/IEC 13818-1:2007, "Information technology - Generic coding of moving pictures and associated audio information: Systems". [ISO13818-2] ISO/IEC 13818-2:2000, "Information technology - Generic coding of moving pictures and associated audio information - Part 2: Video". [ISO13818-3] ISO/IEC 13818-3:1998, "Information technology - Generic coding of moving pictures and associated audio information - Part 3: Audio". [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2250] Hoffman, D., Fernando, G., Goyal, V., and M. Civanlar, "RTP Payload Format for MPEG1/MPEG2 Video", RFC 2250, January 1998. [RFC3388] Camarillo, G., Eriksson, G., Holler, J., and H. Schulzrinne, "Grouping of Media Lines in the Session Description Protocol (SDP)", RFC 3388, December 2002. [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003. [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, "Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, Yang, et al. Expires April 25, 2011 [Page 21] Internet-Draft Preamble Acquisition October 2010 July 2006. [RFC4588] Rey, J., Leon, D., Miyazaki, A., Varsa, V., and R. Hakenberg, "RTP Retransmission Payload Format", RFC 4588, July 2006. 9.2. Informational References [ATSC Doc. A/65] ATSC Document A/65, "ATSC Standard: Program and System Information Protocol(PSIP) for Terrestrial Broadcast and Cable". [ATSC Doc. A/69] ATSC Document A/69, "ATSC Recommended Practice: Program and System Information Protocol Implementation Guidelines for Broadcasters". [ETSI EN 300 468] ETSI EN 300 468, "Digital Video Broadcasting (DVB); Specification for Service Information (SI) in DVB systems". [ETSI TS 101 197] ETSI TS 101 197, "Digital Video Broadcasting (DVB); DVB Simulcrypt - Head-end architecture and synchronization". [ETSI TS 103 197] ETSI TS 103 197, "Digital Video Broadcasting (DVB); Head- end implementation of DVB SimulCrypt". [I-D.ietf-avt-rtcp-guidelines] Ott, J. and C. Perkins, "Guidelines for Extending the RTP Control Protocol (RTCP)", draft-ietf-avt-rtcp-guidelines-04 (work in progress), May 2010. Yang, et al. Expires April 25, 2011 [Page 22] Internet-Draft Preamble Acquisition October 2010 Authors' Addresses Peilin Yang Huawei Technologies Co., Ltd. 101 Software Avenue Nanjing 210012 P.R.China Phone: +86 25 56622638 Email: yangpeilin@huawei.com Frank Xia Huawei Technologies Co., Ltd. 1700 Alma Dr. Suite 500 Plano, TX 75075 Phone: +1 972-509-5599 Email: xiayangsong@huawei.com Xingfen Wu Huawei Technologies Co., Ltd. 101 Software Avenue, Yuhua District Nanjing 210001 P.R.China Phone: +86 25 84565870 Email: wuxingfen@huawei.com Yang, et al. Expires April 25, 2011 [Page 23]