Audio/Video Transport Working Group A.Miyazaki Internet Draft H.Fukushima Document: draft-ietf-avt-rtp-selret-01.txt K.Hata February 06, 2001 T.Wiebke Expires: August 2001 R.Hakenberg C.Burmeister Matsushita N.Takatori S.Okumura T.Ohno Mitsubishi RTP Retransmission Payload Format 1 Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. 2 Abstract This document describes new RTP payload formats to enable multiple and optional selective retransmissions in RTP. They are especially applicable to environments where low delay feedback is available, such as described in [LDF,LDF2]. These payload formats can be used to separate the media stream according to prioritization of packets or according to the status of the transmission (i.e. transmission or retransmission). This serves as a retransmission framework, where we give examples how this can be used for different retransmission schemes. Expires August 2001 1 RTP Retransmission Payload Format March 2001 Table of Contents 1 Status of this Memo 1 2 Abstract 1 3 Conventions used in this document 2 4 Introduction to the problem 2 5 Outline of the solution 4 6 Retransmission Payload Format 5 7 Low Priority Payload Format 6 8 Feedback 7 9 Congestion Control 8 10 A Multiple Retransmission Algorithm 9 11 A Selective Retransmission Algorithm 9 12 Security Considerations 10 13 Acknowledgements 11 14 Intellectual property considerations 11 15 References 11 16 AuthorĘs Addresses 11 3 Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [1]. 4 Introduction to the problem The Real-Time Transport Protocol (RTP)[3] was designed as an Internet protocol to transmit real-time (or near real-time) data over multicast or unicast. Even though the multicast capability is a strong feature of RTP, a common use of it is non-interactive unicast transmission of media streams. This application, further referred to as media streaming, has lower delay requirements. A constant offset delay, in which already received packets are stored at the client, is tolerable. Typically RTP is run over the User Datagram Protocol (UDP)[4]. While the unreliable UDP performs good for reliable channels, the quality of the received media stream is bad, when transmitted over error- prone links. Especially compressed streams, such as MPEG video-data, are highly susceptible to packet loss, which will lead to a bad video quality at the client. E.g. in a wireless environment with a unstable transmission quality (bit error rate (BER) about 10E-3 .. 10E-6) a sufficient quality of video and voice transmitted by RTP cannot be achieved. The scenario for such a media streaming over an error-prone link might be the one in Figure 1. Media files are stored on a media server which is connected via an internet link to a mobile network. Expires August 2001 2 RTP Retransmission Payload Format March 2001 The wireless link between the mobile network and the mobile terminal, which works as the client in this scenario, is generally unreliable and subject to bit errors. Media Mobile Mobile Server Network Terminal (Client) *** | +----+ ** ** +--+ | | * * | | | | -------------- * * ~ ~ ~ ~ ~ ~ ~ | | +----+ * * +--+ ** ** Internet *** Wireless Link Link Figure 1 : Scenario for media streaming To achieve a better quality of the received media stream, especially when transmitted over error-prone links, retransmissions of lost packets of the stream seems to be a possible solution. However the quite low delay requirements of media streaming applications and the bandwidth limitations of the wireless link have to be considered. In wireless systems bandwidth is a scarce resource and efficient use of this resource is of vital importance. Hence reliability for the complete media stream might not be achievable. Furthermore in compressed media streams, e.g. MPEG video stream, not every frame is equally important. In this example some frames contain data that other data depends upon, which makes these frames more important. Current protocols applying retransmission for reliable transmission (e.g. TCP) make no use of this characteristic of compressed media streams. Therefore a retransmission scheme that retransmits selected packets, but if necessary these packets several times seems to be the best solution and can substantially enhance the quality of the received media stream. In the next section a framework is outlined which is based on the payload formats defined in section ??? and ???. This framework enables us to do multiple selective retransmissions as we describe in section ???. Expires August 2001 3 RTP Retransmission Payload Format March 2001 5 Outline of the solution Much work has been done recently to make RTP more reliable. As one result a new feedback timing behavior for unicast sessions or small multicast groups is proposed [9] and a framework of how to use this feedback [10]. These documents include the ability to indicate the detection of packet loss from the receiver to the sender. With this functionality it is possible for the server to do retransmissions for lost packets. However this scheme would be very general and simple and lacks the possibility to do selective or multiple retransmissions effectively. The payload formats presented in this document are meant as a general solution to enable retransmissions in RTP. Based on this scheme we present two examples how the payload formats could be used to realize first a retransmissions scheme which concentrates on bandwidth efficiency within the media stream and second a somehow more complex retransmission scheme for special applications that indicates the priority of the transmitted media in-band. It is described in detail in section ???. The payload formats are not meant to replace a media RTP payload format, but are used additionally. This means that e.g. an RTP retransmission packet could be composed as follows: +----------------------------------------------------+ | | | RTP header with PT=RTX/LP | | | +----------------------------------------------------+ | | |RTX/LP Payload Type specific header with PT=media_pt| | | +----------------------------------------------------+ : : : Media Payload Type specific header : : : +----------------------------------------------------+ | | | Media Payload | : : The sender of RTP packets which wants to use the retransmission capabilities has different possibilities at transmitting RTP packets. For packets that are transmitted for the first time, the sender MAY use the Low Priority (LP) payload format. If so, it has the possibility to signal the packetĘs priority in band and if it is a low priority packet the sequence number of the last high priority packet. This can be used at the receiver of the packets to detect if the lost packet was high or low priority. Low priority packets are Expires August 2001 4 RTP Retransmission Payload Format March 2001 usually not subject to retransmissions and thus no retransmission request has to be sent for low priority packets. Retransmissions should be transmitted with the Retransmission (RTX) payload format, because the RTX payload format specific header in combination with the usual RTP header contains all information that is needed at the receiver to detect lost retransmissions and reconstruct the original packet (e.g. original sequence number or timestamp). 6 Retransmission Payload Format Retransmissions in RTP can be sent with the RTX payload format. This enables multiple retransmissions. The syntax of the header is as follows: 0 1 2 3 4 5 6 7 +---+---+---+---+---+---+---+---+ | X | PT of original packet | +---+---+---+---+---+---+---+---+ |(LSB) Delta RTP Sequence Number| +---+---+---+---+---+---+---+---+ : : + MSB Delta RTP Sequence Number + if X=1 : : +...............................+ Extended Delta Sequence Number (X): 1 bit This bit indicates the used length of the Delta RTP Sequence Number. For X=0 the least significant 8 bits of the Delta RTP Sequence Number are transmitted, for X=1 24 bits are transmitted. Payload Type of original packet: 7 bit This field contains the original payload type information, i.e. the payload type of the payload that is transported in this packet. (LSB) Delta RTP Sequence Number: 8 bit This field contains the difference of this packetĘs RTP sequence number to the RTP sequence number of the original transmission of this packetĘs payload. If the number of bits that are needed to encode the difference exceeds 8, X=1 MUST be set and the 16 most significant bits of the 24 bit Delta RTP Sequence Number are transmitted in the next field. MSB Delta RTP Sequence Number: 16 bit For X=1 this field carries the 16 most significant bits of the Delta RTP Sequence Number (see also "(LSB) Delta RTP Sequence Number"). Expires August 2001 5 RTP Retransmission Payload Format March 2001 7 Low Priority Payload Format In certain scenarios, e.g. environments with a very low bit rate feedback channel, it is necessary to limit the feedback that is sent. This can be done by the means of this low priority (LP) payload type. If these kind of packets are lost, the receiver does not need to send feedback. Still these packets are needed to detect the loss of high priority packets, and therefore they contain the sequence number of the last important packet that was sent. If the receiver gets a low priority packet, it has only to verify if it has received the indicated last important packet. Only for these packets retransmission requests should be sent. The syntax of the packet is as follows: 0 1 2 3 4 5 6 7 +---+---+---+---+---+---+---+---+ |DTI| PT of original packet | +---+---+---+---+---+---+---+---+ | Delta SN | +---+---+---+---+---+---+---+---+ : : + Delta Time + : : +...............................+ DTI (Delta Time Indicator): 1 bit DTI=1 indicates that the delta time field is present (i.e. the payload format header has a size of 4 bytes. DTI=0 means that the Delta Time field is not used and thus the header has a size of 2 bytes. Payload Type of original packet: 7 bit This field contains the original payload type information, i.e. the payload type of the payload that is transported in this packet. Delta SN (Delta Sequence Number): 8 bit The offset in sequence number space to the previous important packet. It is calculated as follows: DSN = SN_this - SN_important. Delta Time: 15 bit This field indicates the difference between the time stamp of the last important RTP packet (that should be retransmitted if lost) and that of the current RTP packet. It can be used by the receiver to compute the timestamp of a lost important packet. The default unit of this field is [ms] and by default the following table is used to encode the Delta Time values. Expires August 2001 6 RTP Retransmission Payload Format March 2001 Delta Time [ms] | bits -----------------+-------------------- more than 32767 | 0111 1111 1111 1111 32766 | 0111 1111 1111 1110 32765 | 0111 1111 1111 1101 : | : 2 | 0000 0000 0000 0010 1 | 0000 0000 0000 0001 0 | 0000 0000 0000 0000 -1 | 1111 1111 1111 1111 -2 | 1111 1111 1111 1110 : | : -32766 | 1000 0000 0000 0010 -32767 | 1000 0000 0000 0001 less than -32768 | 1000 0000 0000 0000 It is possible to define other units or coding tables, however, how to specify and negotiate this is outside the scope of this document. 8 Feedback Feedback is an important part of any retransmission scheme. To be efficient, also under near real-time conditions, it might be needed to receive this feedback with the lowest delay that is possible. The basic RTP set constraints about feedback timing rules, which might not be suitable for a retransmission scheme under real-time or near real-time conditions. To enable low delay feedback a new RTP profile was defined, which redefines these timing rules according to the group size of the multicast session. In unicast sessions it is possible to send feedback virtually immediately. Hence this profile provides the means to do enable low delay feedback which is needed for the retransmission schemes. In the Low Delay Feedback profile RTCP feedback messages are defined, which are suited for retransmission requests or acknowledgements. Generally a simple feedback message, called general NACK is suitable to be used as a retransmission request. It is defined in [LDF] as follows: Expires August 2001 7 RTP Retransmission Payload Format March 2001 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|E| FMT=1 | PT=RTPFB | length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of packet sender | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of media source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PID | BLP | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The PID is used to indicate a lost packet for which the retransmission is requested and the BLP might be used to indicate several lost packets in one NACK message. A detailed description of this and other low delay feedback messages can be found in [LDF2]. 9 Congestion Control In general retransmission schemes contradict congestion control. If packets are lost, which is generally accepted to be an indication of congestion, the sender increases the network load by sending additional retransmissions. While this behavior cannot be accepted in best effort environments such as the Internet, it is reasonable in quality of service (QoS) controlled networks, e.g by reserving a certain bit rate for the RTP session. However the used bit rate MUST be monitored to control that the maximum rate is by no means exceeded. Even though retransmission algorithms normally contradict congestion control they could enhance the quality of service, in environments where losses occur not only due to congestion but also (or mainly) due to bit errors on the link. Taking this information into account the following rule MUST be followed, if the retransmissions are used in an best effort environment (e.g. the Internet). At reception of a retransmission request or detecting packet loss by other means, the sender SHOULD NOT increase the sending bit rate. Retransmissions MAY be sent within the current bit rate, e.g. by skipping or delaying the transmission of other scheduled packets. The rules for congestion control of [RTP],[AV-Profile],[LDF] MUST be followed. Expires August 2001 8 RTP Retransmission Payload Format March 2001 10 A Multiple Retransmission Algorithm This section gives an examples how the RTX payload format is used to enable multiple retransmissions in RTP. The sender transmits the media data in normal RTP packets using an RTP payload format for that media. If packets get lost the receiver will detect that by the gap in the sequence number. To signal lost packets the NACK message, described in section 8 is used. The PID field is used to specify a lost packet. The default method, which is to use the RTP sequence number to indicate the lost packet, is used. The BLP field can be used to indicate several lost packets in one NACK message. The detailed semantic is described in [LDF2]. At receiving a retransmission request the sender decides, according to the priority of the packet, the network conditions (see section 9) and maybe other factors, if the packet is to be retransmitted. In case of a positive decision, it uses the RTX payload format (see section 6) to retransmit the packet. The sequence number of the RTP packet is increased by one over the previous packet sent. This enables the receiver to detect also a lost retransmission. Example of a retransmission packet: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC=0 | PT=RTX | RTP sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP timestamp = timestamp of original transmission | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of media source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| PT=??? | DSN=12 | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + : : : Media Payload : : : 11 A Selective Retransmission Algorithm A special scenario, where the previously described multiple retransmission algorithm does not work well, is if the feedback channel has a very limited capacity. Possible scenarios are small multicast groups, where it is according to the defined timing rules for RTCP feedback seldom possible to send immediate feedback, or the feedback channel is mainly used for other purposes (e.g. wireless control channel) and has a very low bit rate available for RTCP feedback. In this scenarios it is very important to limit the Expires August 2001 9 RTP Retransmission Payload Format March 2001 feedback to the very minimum, which would mean that feedback is only sent in form of retransmission requests for really important data. To solve the above described requirements on the retransmission scheme, it is necessary that the receiver knows the priority of the lost packets, to decide if a retransmission request should be sent or not. To do so at least two solutions apply: First the important and non-important packets are sent in different streams, using a different sequence number space. Second the low priority packet format is used. There are scenarios where the first solution is applicable and thus should be used, because it uses less overhead. However there are drawbacks with this solution. Given that only very few high priority packets are sent, e.g. once every 2 seconds, the time between loosing a packet and detection of this packet loss is at least this inter-packet time. Hence in scenarios as described above, where feedback can be sent only very infrequently, and thus only really important packets are retransmitted, a solution which uses the low priority payload format might be the only possible solution at all. The mechanism works as described in the following. The sender sends normal media data in RTP packets using only the media payload type for high priority packets and the low priority payload format plus additionally the media payload type for low priority packets. In the low priority packets, the Delta SN field is used at the receiver to identify the last high priority packet. If this packet was not received, a retransmission request is sent. The receiver can also reconstruct the timestamp of the lost packet if the sender used the Delta Time field. From this time information it is possible to calculate the time after which the packet cannot be used any more (display time). In this case the client should maintain a round trip time (RTT) measurement and compare the calculated display time with the estimated time when the retransmission could arrive. If the retransmission would arrive already outdated, a retransmission request should not be sent. At receiving a retransmission request from the receiver, the server has to make the same decision as described in the previous section, i.e. it has to judge according to the packets priority (which is most probably quite high, because a retransmission request for it was sent), network status etc. if the packet should be retransmitted. If so it uses the retransmission packet type, to enable multiple retransmissions as described in the previous section. 12 Security Considerations T.B.D. Expires August 2001 10 RTP Retransmission Payload Format March 2001 13 Acknowledgements For the work on this retransmission framework, we received a lot of help and advice from Colin Perkins. Especially the mechanism of sending retransmissions with a new sequence number in the same sequence number space, using a different payload type was his idea. 14 Intellectual property considerations Matsushita has filed patent applications that might possibly have technical relation to this contribution. If part(s) of the contribution by Matsushita employee is (are) included in a standard and Matsushita has patents and/or patent application(s) that are essential to implementation of such included part(s) in said standard, Matsushita is prepared to grant - on the basis of reciprocity (grantback) - a license on such included part(s) on reasonable, non-discriminatory terms and conditions (in according with paragraph 10.3.3 of the RFC 2026). 15 References 1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. 2 Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 3 Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson, "RTP: A Transport Protocol for real-time applications", RFC 1889, January 1996. 4 Wenger, S., Ott, J., "RTCP-based Feedback for Predictive Video Coding", IETF Internet Draft, work in progress, July 2000. 5 Fukunaga, S. et al.,"Low Delay RTCP Feedback Format", IETF Internet Draft, work in progress, November 2000. 16 AuthorĘs Addresses Akihiro Miyazaki Matsushita Electric Industrial Co., Ltd 1006, Kadoma, Kadoma City, Osaka, Japan Tel. +81-6-6900-9632 Fax. +81-6-6900-9169 Mail akihiro@isl.mei.co.jp Expires August 2001 11 RTP Retransmission Payload Format March 2001 Hideaki Fukushima Matsushita Electric Industrial Co., Ltd 1006, Kadoma, Kadoma City, Osaka, Japan Tel. +81-6-6900-9192 Fax. +81-6-6900-9193 Mail fukusima@isl.mei.co.jp Koichi Hata Matsushita Electric Industrial Co., Ltd 1006, Kadoma, Kadoma City, Osaka, Japan Tel. +81-6-6900-9192 Fax. +81-6-6900-9193 Mail hata@isl.mei.co.jp Thomas Wiebke Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Tel. +49-(0)6103-766-161 Fax. +49-(0)6103-766-166 Mail wiebke@panasonic.de Rolf Hakenberg Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Tel. +49-(0)6103-766-162 Fax. +49-(0)6103-766-166 Mail hakenberg@panasonic.de Carsten Burmeister Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Tel. +49-(0)6103-766-263 Fax. +49-(0)6103-766-166 Mail burmeister@panasonic.de Norihito Takatori Mitsubishi Electric Corporation Information Technology R&D Center 5-1-1 Ofuna, Kamakura, Kanagawa, 247-8501, Japan Tel. +81-467-41-2103 Fax. +81-467-41-2137 Mail takatori@isl.melco.co.jp Seiji Okumura Mitsubishi Electric Corporation Information Technology R&D Center 5-1-1 Ofuna, Kamakura, Kanagawa, 247-8501, Japan Tel. +81-467-41-2103 Fax. +81-467-41-2137 Mail okumura@isl.melco.co.jp Expires August 2001 12 RTP Retransmission Payload Format March 2001 Tsugihiko Ohno Mitsubishi Electric Corporation Information Technology R&D Center 5-1-1 Ofuna, Kamakura, Kanagawa, 247-8501, Japan Tel. +81-467-41-2103 Fax. +81-467-41-2137 Mail tohno@isl.melco.co.jp Expires August 2001 13