Audio/Video Transport Working Group A.Miyazaki Internet Draft H.Fukushima Document: draft-ietf-avt-rtp-selret-02.txt K.Hata July 18, 2001 T.Wiebke Expires: January 2001 R.Hakenberg C.Burmeister Matsushita N.Takatori S.Okumura T.Ohno Mitsubishi RTP Retransmission Payload Format 1 Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. 2 Abstract This document describes new RTP payload formats to enable multiple and optional selective retransmissions in RTP. They are especially applicable to environments where low delay RTCP feedback is available, such as described in [4]. These payload formats can be used to separate the media stream according to prioritization of packets or according to the status of the transmission (i.e. transmission or retransmission). This serves as a retransmission framework, where we give examples how this can be used for different retransmission schemes. RTP Retransmission Payload Format July 2001 Table of Contents 1 Status of this Memo 1 2 Abstract 1 3 Conventions used in this document 2 4 Introduction to the problem 2 5 Outline of the solution 4 6 Retransmission Payload Format 5 7 Low Priority Payload Format 6 8 Feedback 7 9 Congestion Control 8 10 A Multiple Retransmission Algorithm 8 11 A Selective Retransmission Algorithm 9 12 Security Considerations 11 13 Acknowledgements 11 14 Intellectual property considerations 11 15 References 11 16 Author's Addresses 11 3 Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [1]. 4 Introduction to the problem The Real-Time Transport Protocol (RTP)[3] was designed as an Internet protocol to transmit real-time (or near real-time) data over multicast or unicast. Even though the multicast capability is a strong feature of RTP, a common use of it is non-interactive unicast transmission of media streams. This application, further referred to as media streaming, has lower delay requirements. A constant offset delay, in which already received packets are stored at the client, is tolerable. Typically RTP is run over the User Datagram Protocol (UDP). While the unreliable UDP performs good for reliable channels, the quality of the received media stream is bad, when transmitted over error- prone links. Especially compressed streams, such as MPEG video-data, are highly susceptible to packet loss, which will lead to a bad video quality at the client. E.g. in a wireless environment with a unstable transmission quality (bit error rate (BER) about 10E-3 .. 10E-6) a sufficient quality of video and voice transmitted by RTP cannot be achieved. The scenario for such a media streaming over an error-prone link might be the one in Figure 1. Media files are stored on a media server which is connected via an internet link to a mobile network. RTP Retransmission Payload Format July 2001 The wireless link between the mobile network and the mobile terminal, which works as the client in this scenario, is generally unreliable and subject to bit errors. Media Mobile Mobile Server Network Terminal (Client) *** | +----+ ** ** +--+ | | * * | | | | -------------- * * ~ ~ ~ ~ ~ ~ ~ | | +----+ * * +--+ ** ** Internet *** Wireless Link Link Figure 1 : Scenario for media streaming To achieve a better quality of the received media stream, especially when transmitted over error-prone links, retransmissions of lost packets of the stream seems to be a possible solution. However the quite low delay requirements of media streaming applications and the bandwidth limitations of the wireless link have to be considered. In wireless systems bandwidth is a scarce resource and efficient use of this resource is of vital importance. Hence reliability for the complete media stream might not be achievable. Furthermore in compressed media streams, e.g. MPEG video stream, not every frame is equally important. In this example some frames contain data that other data depends upon, which makes these frames more important. Current protocols applying retransmission for reliable transmission (e.g. TCP) make no use of this characteristic of compressed media streams. Therefore a retransmission scheme that retransmits selected packets, but if necessary these packets several times seems to be the best solution and can substantially enhance the quality of the received media stream. In the next section a framework is outlined which is based on the payload formats defined in sections 6 and 7. This framework enables us to do multiple selective retransmissions as we describe in sections 10 and 11. RTP Retransmission Payload Format July 2001 5 Outline of the solution Much work has been done recently to make RTP more reliable. As one result a new feedback timing behavior for unicast sessions or small multicast groups is proposed [4], which includes a framework of how to use this feedback. This document include the ability to indicate the detection of packet loss from the receiver to the sender. With this functionality it is possible for the server to do retransmissions for lost packets. However this scheme would be very general and simple and lacks the possibility to do selective or multiple retransmissions effectively. The payload formats presented in this document are meant as a general solution to enable retransmissions in RTP. Based on this scheme we present two examples how the payload formats could be used to realize first a retransmissions scheme which concentrates on bandwidth efficiency within the media stream and second a somehow more complex retransmission scheme for special applications that indicates the priority of the transmitted media in-band. It is described in detail in section 11. The payload formats are not meant to replace a media RTP payload format, but are used additionally. This means that e.g. an RTP retransmission packet could be composed as follows: +-----------------------------------------------------+ | | | RTP header with PT=RTX/LP | | | +-----------------------------------------------------+ | | |RTX/SEL Payload Type specific header with PT=media_pt| | | +-----------------------------------------------------+ : : : Media Payload Type specific header : : : +-----------------------------------------------------+ | | | Media Payload | : : The sender of RTP packets which wants to use the retransmission capabilities has different possibilities at transmitting RTP packets. For packets that are transmitted for the first time, the sender MAY use the Selective (SEL) payload format. If so, it has the possibility to signal the packet's priority and the sequence number of the last high priority packet in band. This can be used at the receiver of the packets to detect if the lost packet was high or low priority. Low priority packets are usually not subject to RTP Retransmission Payload Format July 2001 retransmissions and thus no retransmission request has to be sent for low priority packets. Retransmissions SHOULD be transmitted with the Retransmission (RTX) payload format, because the RTX payload format specific header in combination with the usual RTP header contains all information that is needed at the receiver to detect lost retransmissions and reconstruct the original packet (e.g. original sequence number or timestamp). 6 Retransmission Payload Format Retransmissions in RTP SHOULD be sent with the RTX payload format. This enables multiple retransmissions. The syntax of the header is as follows: 0 1 2 3 4 5 6 7 +---+---+---+---+---+---+---+---+ | X | PT of original packet | +---+---+---+---+---+---+---+---+ |(LSB) Delta RTP Sequence Number| +---+---+---+---+---+---+---+---+ : : + MSB Delta RTP Sequence Number + if X=1 : : +...............................+ Extended Delta Sequence Number (X): 1 bit This bit indicates the used length of the Delta RTP Sequence Number. For X=0 the least significant 8 bits of the Delta RTP Sequence Number are transmitted, for X=1 24 bits are transmitted. Payload Type of original packet: 7 bit This field contains the original payload type information, i.e. the payload type of the payload that is transported in this packet. (LSB) Delta RTP Sequence Number: 8 bit This field contains the difference of this packet's RTP sequence number to the RTP sequence number of the original transmission of this packet's payload, calculated modulo 2^32, i.e. Delta SN = (SN(this) _ SN(original)) mod 2^32. If the number of bits that are needed to encode the difference exceeds 8, X=1 MUST be set and the 16 most significant bits of the 24 bit Delta RTP Sequence Number are transmitted in the next field. MSB Delta RTP Sequence Number: 16 bit For X=1 this field carries the 16 most significant bits of the Delta RTP Sequence Number (see also "(LSB) Delta RTP Sequence Number"). RTP Retransmission Payload Format July 2001 7 Selective Payload Format In certain scenarios, e.g. environments with a very low bit rate feedback channel, it is necessary to limit the feedback that is sent. This can be done by the means of this selective (SEL) payload type. If packets of a low priority are lost, the receiver does not need to send feedback. Still these packets are needed to detect the loss of high priority packets, and therefore they contain the sequence number of the last important packet that was sent. If the receiver gets a low priority packet, it has only to verify if it has received the indicated last important packet. Only for these packets retransmission requests should be sent. The syntax of the packet is as follows: 0 1 2 3 4 5 6 7 +---+---+---+---+---+---+---+---+ |DTI| PT of original packet | +---+---+---+---+---+---+---+---+ |PRI| Delta SN | +---+---+---+---+---+---+---+---+ : : + Delta Time + : : +...............................+ DTI (Delta Time Indicator): 1 bit DTI=1 indicates that the delta time field is present (i.e. the payload format header has a size of 4 bytes. DTI=0 means that the Delta Time field is not used and thus the header has a size of 2 bytes. Payload Type of original packet: 7 bit This field contains the original payload type information, i.e. the payload type of the payload that is transported in this packet. PRI (Priority): 1 bit This field indicates the priority of the packet. PRI=1 means a high priority packet, where PRI=0 is sent for low priority packets. Delta SN (Delta Sequence Number): 7 bit The offset in sequence number space to the previous important packet. It is calculated as follows: DSN = SN_this - SN_important. Delta Time: 15 bit This field indicates the difference between the time stamp of the last important RTP packet (that should be retransmitted if lost) and that of the current RTP packet. It can be used by the receiver to compute the timestamp of a lost important packet. RTP Retransmission Payload Format July 2001 The default unit of this field is [ms] and by default the following table is used to encode the Delta Time values. Delta Time [ms] | bits -----------------+-------------------- more than 32766 | 0111 1111 1111 1111 32766 | 0111 1111 1111 1110 32765 | 0111 1111 1111 1101 : | : 2 | 0000 0000 0000 0010 1 | 0000 0000 0000 0001 0 | 0000 0000 0000 0000 -1 | 1111 1111 1111 1111 -2 | 1111 1111 1111 1110 : | : -32766 | 1000 0000 0000 0010 -32767 | 1000 0000 0000 0001 less than -32767 | 1000 0000 0000 0000 It is possible to define other units or coding tables, however, how to specify and negotiate this is outside the scope of this document. 8 Feedback Feedback is an important part of any retransmission scheme. To be efficient, also under near real-time conditions, it might be needed to receive this feedback with the lowest delay that is possible. The basic RTP set constraints about feedback timing rules, which might not be suitable for a retransmission scheme under real-time or near real-time conditions. To enable low delay feedback a new extended RTP profile was defined, which redefines these timing rules according to the group size of the multicast session. In unicast sessions it is possible to send feedback virtually immediately. Hence this profile provides the means to do enable low delay feedback which is needed for the retransmission schemes. In the Low Delay Feedback profile RTCP feedback messages are defined, which are suited for retransmission requests or acknowledgements. Generally a simple feedback message, called general NACK is suitable to be used as a retransmission request. It is defined in [4] as follows: RTP Retransmission Payload Format July 2001 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|E| FMT=1 | PT=RTPFB | length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of packet sender | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of media source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PID | BLP | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The PID is used to indicate a lost packet for which the retransmission is requested and the BLP might be used to indicate several lost packets in one NACK message. A detailed description of this and other low delay feedback messages can be found in [4]. 9 Congestion Control In general retransmission schemes contradict congestion control. If packets are lost, which is generally accepted to be an indication of congestion, the sender increases the network load by sending additional retransmissions. While this behavior cannot be accepted in best effort environments such as the Internet, it is reasonable in quality of service (QoS) controlled networks, e.g by reserving a certain bit rate for the RTP session. However the used bit rate MUST be monitored to control that the maximum rate is by no means exceeded. Even though retransmission algorithms normally contradict congestion control they could enhance the quality of service, in environments where losses occur not only due to congestion but also (or mainly) due to bit errors on the link. Taking this information into account the following rule MUST be followed, if the retransmissions are used in an best effort environment (e.g. the Internet). At reception of a retransmission request or detecting packet loss by other means, the sender MUST NOT increase the sending bit rate on mid term timescales. Retransmissions MAY be sent within the current bit rate, e.g. by skipping or delaying the transmission of other scheduled packets. The rules for congestion control of [3] and [4] MUST be followed. 10 A Multiple Retransmission Algorithm This section gives an examples how the RTX payload format is used to enable multiple retransmissions in RTP. RTP Retransmission Payload Format July 2001 The sender transmits the media data in normal RTP packets using an RTP payload format for that media. If packets get lost the receiver will detect that by the gap in the sequence number. To signal lost packets the NACK message, described in section 8 is used. The PID field is used to specify a lost packet. The default method, which is to use the RTP sequence number to indicate the lost packet, is used. The BLP field can be used to indicate several lost packets in one NACK message. The detailed semantic is described in [4]. At receiving a retransmission request the sender decides, according to the priority of the packet, the network conditions (see section 9) and maybe other factors, if the packet is to be retransmitted. In case of a positive decision, it uses the RTX payload format (see section 6) to retransmit the packet. The sequence number of the RTP packet is increased by one over the previous packet sent. This enables the receiver to detect also a lost retransmission. Example of a retransmission packet: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC=0 | PT=RTX | RTP sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP timestamp = timestamp of original transmission | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of media source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| PT=??? | DSN=12 | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + : : : Media Payload : : : 11 A Selective Retransmission Algorithm A special scenario, where the previously described multiple retransmission algorithm does not work well, is if the feedback channel has a very limited capacity. Possible scenarios are small multicast groups, where it is according to the defined timing rules for RTCP feedback seldom possible to send immediate feedback, or the feedback channel is mainly used for other purposes (e.g. wireless control channel) and has a very low bit rate available for RTCP feedback. In this scenarios it is very important to limit the feedback to the very minimum, which would mean that feedback is only sent in form of retransmission requests for really important data. RTP Retransmission Payload Format July 2001 To solve the above described requirements on the retransmission scheme, it is necessary that the receiver knows the priority of the lost packets, to decide if a retransmission request should be sent or not. To do so at least two solutions apply: First the important and non-important packets are sent in different streams, using a different sequence number space. Second the selective packet format is used. There are scenarios where the first solution is applicable and thus should be used, because it uses less overhead. However there are drawbacks with this solution. Given that only very few high priority packets are sent, e.g. once every 2 seconds, the time between loosing a packet and detection of this packet loss is at least this inter-packet time. Hence in scenarios as described above, where feedback can be sent only very infrequently, and thus only really important packets are retransmitted, a solution which uses the selective payload format might be the only possible solution at all. The mechanism works as described in the following. The sender sends all media data in RTP packets using the SEL payload type plus additional the media payload type. The priority field is set according to the packet's priority. In all packets the Delta SN field is used to signal the delta to the sequence number of the previous high priority packet. The receiver can use this information to check if the last high priority packet was received correctly or not. If this packet was not received or he cannot reliable detect the priority of lost packets (if many packets were lost in a row), a retransmission request is sent. The receiver can also reconstruct the timestamp of the lost packet if the sender used the Delta Time field. From this time information it is possible to calculate the time after which the packet cannot be used any more (display time). In this case the client should maintain a round trip time (RTT) measurement and compare the calculated display time with the estimated time when the retransmission could arrive. If the retransmission would arrive already outdated, a retransmission request should not be sent. At receiving a retransmission request from the receiver, the server has to make the same decision as described in the previous section, i.e. it has to judge according to the packets priority (which is most probably quite high, because a retransmission request for it was sent), network status etc. if the packet should be retransmitted. If so it uses the retransmission packet type, to enable multiple retransmissions as described in the previous section. However the SEL payload type is additionally used to enbale the priority detection at the client. RTP Retransmission Payload Format July 2001 12 Security Considerations T.B.D. 13 Acknowledgements For the work on this retransmission framework, we received a lot of help and advice from Colin Perkins. Especially the mechanism of sending retransmissions with a new sequence number in the same sequence number space, using a different payload type was his idea. 14 Intellectual property considerations Matsushita has filed patent applications that might possibly have technical relation to this contribution. If part(s) of the contribution by Matsushita employee is (are) included in a standard and Matsushita has patents and/or patent application(s) that are essential to implementation of such included part(s) in said standard, Matsushita is prepared to grant - on the basis of reciprocity (grantback) - a license on such included part(s) on reasonable, non-discriminatory terms and conditions (in according with paragraph 10.3.3 of the RFC 2026). 15 References 1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. 2 Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 3 Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson, "RTP: A Transport Protocol for real-time applications", RFC 1889, January 1996. 4 Ott,J. et al., "Extended RTP Profile for RTCP-based Feedback", IETF Internet Draft, draft-ietf-avt-rtcp-feedback-00, work in progress, July 2001. 16 Author's Addresses Akihiro Miyazaki Matsushita Electric Industrial Co., Ltd RTP Retransmission Payload Format July 2001 1006, Kadoma, Kadoma City, Osaka, Japan Tel. +81-6-6900-9632 Fax. +81-6-6900-9169 Mail akihiro@isl.mei.co.jp Hideaki Fukushima Matsushita Electric Industrial Co., Ltd 1006, Kadoma, Kadoma City, Osaka, Japan Tel. +81-6-6900-9192 Fax. +81-6-6900-9193 Mail fukusima@isl.mei.co.jp Koichi Hata Matsushita Electric Industrial Co., Ltd 1006, Kadoma, Kadoma City, Osaka, Japan Tel. +81-6-6900-9192 Fax. +81-6-6900-9193 Mail hata@isl.mei.co.jp Thomas Wiebke Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Tel. +49-(0)6103-766-161 Fax. +49-(0)6103-766-166 Mail wiebke@panasonic.de Rolf Hakenberg Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Tel. +49-(0)6103-766-162 Fax. +49-(0)6103-766-166 Mail hakenberg@panasonic.de Carsten Burmeister Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Tel. +49-(0)6103-766-263 Fax. +49-(0)6103-766-166 Mail burmeister@panasonic.de Norihito Takatori Mitsubishi Electric Corporation Information Technology R&D Center 5-1-1 Ofuna, Kamakura, Kanagawa, 247-8501, Japan Tel. +81-467-41-2103 Fax. +81-467-41-2137 Mail takatori@isl.melco.co.jp Seiji Okumura Mitsubishi Electric Corporation Information Technology R&D Center RTP Retransmission Payload Format July 2001 5-1-1 Ofuna, Kamakura, Kanagawa, 247-8501, Japan Tel. +81-467-41-2103 Fax. +81-467-41-2137 Mail okumura@isl.melco.co.jp Tsugihiko Ohno Mitsubishi Electric Corporation Information Technology R&D Center 5-1-1 Ofuna, Kamakura, Kanagawa, 247-8501, Japan Tel. +81-467-41-2103 Fax. +81-467-41-2137 Mail tohno@isl.melco.co.jp