Audio Video Transport WG Q. Xie Internet-Draft J. Schumacher Expires: January 14, 2006 Motorola July 13, 2005 RTP Payloads for Anti-shadow Redundancy draft-xie-avt-rtp-anti-shadow-01.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on January 14, 2006. Copyright Notice Copyright (C) The Internet Society (2005). Abstract This document defines a special form of RTP redundant packatization format for multimedia broadcast and multicast services to combat service interruptions caused by losses of large numbers of consecutive media frames. Along with the payload format, this document also defines the procedures the RTP sender and receiver will need to use to conceal the large frames losses. Xie & Schumacher Expires January 14, 2006 [Page 1] Internet-Draft RTP Anti-shadow Redundancy Payloads July 2005 Table of Contents 1. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.1 Anti-shadowing algorithm for multicast/broadcast . . . . . 3 2.1.1 Send media for anti-shadowing operation . . . . . . . 3 2.1.2 Receive media for anti-shadowing operation . . . . . . 4 3. Anti-shadowing RTP payload format . . . . . . . . . . . . . . 6 3.1 Anti-shadowing header extension format . . . . . . . . . . 7 3.2 RTP header usage . . . . . . . . . . . . . . . . . . . . . 7 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 4.1 Mapping MIME Parameters into SDP . . . . . . . . . . . . . 8 4.2 Usage in Offer/Answer . . . . . . . . . . . . . . . . . . 8 5. Security Considerations . . . . . . . . . . . . . . . . . . . 8 6. Normative References . . . . . . . . . . . . . . . . . . . . . 8 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 9 Intellectual Property and Copyright Statements . . . . . . . . 10 Xie & Schumacher Expires January 14, 2006 [Page 2] Internet-Draft RTP Anti-shadow Redundancy Payloads July 2005 1. Conventions The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL, when they appear in this document, are to be interpreted as described in RFC 2119 [2]. 2. Introduction A major technical problem of the increasingly important wireless IP multimedia broadcast/multicast services is that the signal can be temporarily blocked (or "shadowed") for moving receivers by tall buildings in a city, or by tunnels through mountains or under rivers. In the worst scenarios, the receiver may be in a "shadow" for up to several minutes (though far less an issue but wireline IP multimedia services can still suffer similar but minor "shadowing" problem due to, e.g., transit congestion conditions in a local network. Existing loss management techniques are not effective in combat against such loss of a large number of consecutive frames. For example, forward error correction only works if no more than a few consecutive frames are lost. A large play-out buffer will only delay the interruption but won't be able to conceal the loss of data to the end. Re-transmission will be extremely complex and expensive, if not entirely impossible, in a multicast/broadcast session, not to mention that any experienced retransmission delay will accumulate. 2.1 Anti-shadowing algorithm for multicast/broadcast Anti-shadowing algorithm can be readily applied to the broadcast or multicast of stored or pre-recorded media. Because of the need of generating the forward-shifted anti-shadowing stream, however, to apply this algorithm to live media requires the insertion of a delay equal to or greater than the forward-shift at the broadcast/multicast headend. 2.1.1 Send media for anti-shadowing operation In addition to the primary media stream, the anti-shadowing operation requires the RTP sender to transmit an anti-shadowing (AS) media stream at the same time. This AS media stream is forward-shifted in time relative to the primary media stream. The amount of forward- shift MUST remain constant for the duration of the session. In order to save bandwidth, the AS stream may be sent at a lower quality or resolution (either temporally or spatially or both) than the primary media stream. For example, for an audio session the AS stream may use a lower quality encoding mode and/or consists of half Xie & Schumacher Expires January 14, 2006 [Page 3] Internet-Draft RTP Anti-shadow Redundancy Payloads July 2005 the number of frames of the primary stream (i.e., 2-to-1 decimated). The following diagram illustrates the anti-shadowing transmission of a media data session, in which the AS stream is created by decimating the primary stream by half and then forward-shifted the decimated stream by 155 frames at the transmission (aussuming each frame represents 20 ms original media, this represents a 3.1 second forward shift). Primary stream AS stream (forward-shifted by 155 frames) | | v v [ 111 ] [ 266 ] | | v | [ 110 ] | | | v v ^ [ 109 ] [ 264 ] | | | | v | [ 108 ] | T | | I v v M [ 107 ] [ 262 ] E | | v | [ 106 ] | | | v v [ 105 ] [ 260 ] | | v | [ 104 ] | | | v v [ 103 ] [ 258 ] | | v v Trasnmit first 2.1.2 Receive media for anti-shadowing operation Under normal considtions, the receiver receives both the primary and AS streams. The received media frames from the primary stream is passed directly to the appropriate media decoder for play-out. Xie & Schumacher Expires January 14, 2006 [Page 4] Internet-Draft RTP Anti-shadow Redundancy Payloads July 2005 However, the receiver stores the received media frames from the AS stream in an internal anti-shadowing buffer. The depth of this anti- shadowing buffer will need to be enough to hold the forward-shift amount of AS media. For example, if the AS stream is forward-shifted by 3.1 seconds, then the anti-shadowing buffer will have a size that is enough to hold at least 3.1 second worth of the AS stream. (Maybe a figure to show the receiver diagram) The anti-shadowing buffer will continuously be checked for staleness of the media frames it holds (a frame becomes stale when it becomes older than the last frame passed to the media decoder). Once a frame is found stale, it is purged from the anti-shadowing buffer. At the begining of a new broadcast/multicast session, the anti- shadowing buffer will be empty. When the first media frame arrives from the primary stream, the play-out will start immediately. And at the same time, the arrrving AS frames will gradually fill up the anti-shadowing buffer. After the forward-shift amount of time (in our example, 3.1 sec) from the start, the anti-shadowing buffer will be full, holding the forward-shift amount of AS frames. Because of the continuous purge of stale frames and the arrival of new AS frames, under normal condition the anti-shadowing buffer will always hold the next forward-shift amount of AS frames. Let's see an example. For illustrative purpose, we assume that the session starts at t=0, AS forward-shift=3.1 sec, AS stream is the same as the primary stream, and the timestamp of the first media frame is 0.0 sec and each frame represents 0.1 sec of media: Xie & Schumacher Expires January 14, 2006 [Page 5] Internet-Draft RTP Anti-shadow Redundancy Payloads July 2005 Time Timestamp of Timestamp range of frame being played media in AS buffer t < 0 - (buffer empty) t=0 0.0 (1st frame) 3.1 (1 AS frame) t=0.1 0.1 3.1 - 3.2 (2 AS frames) ... t=2.9 2.9 3.1 - 6.0 t=3.0 3.0 3.1 - 6.1 (AS buffer is full) t=3.1 3.1 3.2 - 6.2 (a stale frame purged) ... ... ... t=37 37.0 37.1 - 40.1 (always hold the next ... 3.1 sec worth of AS media) When the receiver enters a shadow, coverage gap, or any other conditions that prevent the receiver from getting new media data, the receiver switches to anti-shadowing mode, in which it starts to play out stored AS media frames from its anti-shadowing buffer, and avoid a service interruption due to the shadow. It is obvious that, in the anti-shadowing mode, the receiver will be able to continue the play-out for up to the forward-shift amount of time. If the shadow condition ends within the forward-shift amount of time (meaning new media data starts to arrive again), the receiver will immediatly switch back to its normal operation to play out from newly arrived primary frames. And at the same time, the arrival of new AS frames will "re-fill" the anti-shadowing buffer. However, if the duration of the shadow is longer than the forward- shift amount of time, the receiver will run out of media frames from its AS buffer. At that point, service interruption will occur. Therefore, the amount of forward-shift of the AS stream determines the max duration of shadow that can be completely cancealed. 3. Anti-shadowing RTP payload format The basic design principle is to use RTP header extension to carry AS frames. The advantage of this approach is that for receivers that do not support anti-shadowing operation, they can simply ignore the RTP header extension and only process frames from the primary stream normally. This is the key for backward compatibility. The following diagram shows the layout of an RTP packet with AS header extension. Xie & Schumacher Expires January 14, 2006 [Page 6] Internet-Draft RTP Anti-shadow Redundancy Payloads July 2005 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header (primary) | +...............................................................+ | Anti-shadowing Header Extension | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Payload (primary) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Here, the RTP header and payload carries the primary stream media frame(s) and will follow the RFC that defines the RTP payload format for the particular media encoding used by the primary stream (e.g., RFC3267 is followed if AMR-WB is used for the primary stream). 3.1 Anti-shadowing header extension format The AS header extension definition follows the rules given in RFC3550, as shown in the following diagram. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | AS ext magic # = 0x83AB | length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | AS RTP packet | | .... | As shown above, the first 16 bit of the extension will carry a magic number (0x83AB) indicating that this is an anti-shadowing header extension. As defined in RFC3550, the length field indicates the number of 32-bit words in the extension, excluding the four-octet extension header. Following the length field is a standard RTP packet that carries one or more AS frame. The AS RTP packet will be padded to the next 32-word boundary if needed. Note, the timestamp in the AS RTP packet MUST use the same base and resolution as that of the primary RTP header. The amount of forward- shift of the AS stream is indicated by the timestamp of the AS RTP packet. Moreover, the AS stream frame(s) carried in the AS RTP packet may be differently encoded, use different mode, and/or be decimated from the primary stream. 3.2 RTP header usage The format of the RTP header that contains AS header extension follows what is specified in RFC 3550 [5], including the use of X bit to indicate the presence of the AS extension. Xie & Schumacher Expires January 14, 2006 [Page 7] Internet-Draft RTP Anti-shadow Redundancy Payloads July 2005 4. IANA Considerations Editor's note: should the extension magic number be registered with IANA? Any MIME param needed for this? Encoding considerations: These types are defined for transfer via RTP [5] as described in Section ?? of RFC XXXX. Security considerations: See Section ?? of RFC XXXX. Person & email address to contact for further information: Qiaobing.Xie@motorola.com Intended usage: COMMON. It is expected that many VoIP applications (as well as mobile applications) will use this type. Author/Change controller: * Qiaobing.Xie@motorola.com * IETF Audio/Video transport working group 4.1 Mapping MIME Parameters into SDP TBD 4.2 Usage in Offer/Answer TBD 5. Security Considerations TBD 6. Normative References [1] Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. [2] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [3] Handley, M. and V. Jacobson, "SDP: Session Description Protocol", RFC 2327, April 1998. [4] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with the Session Description Protocol (SDP)", RFC 3264, June 2002. Xie & Schumacher Expires January 14, 2006 [Page 8] Internet-Draft RTP Anti-shadow Redundancy Payloads July 2005 [5] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", RFC 3550, July 2003. [6] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and Video Conferences with Minimal Control", RFC 3551, July 2003. Authors' Addresses Qiaobing Xie Motorola, Inc. 1501 W. Shure Drive, 2-F9 Arlington Heights, IL 60004 US Phone: +1-847-632-3028 Email: qxie1@email.mot.com Joe Schumacher Motorola, Inc. 1501 W. Shure Drive, 2-B11 Arlington Heights, IL 60004 US Phone: +1-847-632-5978 Email: j.schumacher@motorola.com Xie & Schumacher Expires January 14, 2006 [Page 9] Internet-Draft RTP Anti-shadow Redundancy Payloads July 2005 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society (2005). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Xie & Schumacher Expires January 14, 2006 [Page 10]