Network Working Group R. Huang INTERNET-DRAFT R. Even Intended Status: Informational Huawei Expires: August 18, 2014 V. Singh Aalto University D. Romascanu Avaya L. Deng China Mobile February 14, 2014 Considerations for Selecting RTCP Extended Report (XR) Metrics for the RTCWEB Statistics API draft-huang-xrblock-rtcweb-rtcp-xr-metrics-03 Abstract This document describes monitoring features related to RTCWEB. It provides a list of RTCP XR metrics that are useful and may need to be supported in some RTCWEB implementations. Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Copyright and License Notice Copyright (c) 2014 IETF Trust and the persons identified as the Expires August 18, 2014 [Page 1] INTERNET DRAFT February 14, 2014 document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 RTP Statistics in WebRTC Implementations . . . . . . . . . . . 3 4 Considerations for Impact of Measurement Interval . . . . . . . 4 5 Candidate Metrics . . . . . . . . . . . . . . . . . . . . . . . 4 5.1 Network Impact Metrics . . . . . . . . . . . . . . . . . . 5 5.1.1 Loss and Discard Packet Count Metric . . . . . . . . . 5 5.1.2 Burst/Gap Pattern Metrics for Loss and Discard . . . . 6 5.1.3 Run Length Encoded Metrics for Loss, Discard . . . . . 6 5.1.3 ECN related Metrics . . . . . . . . . . . . . . . . . . 7 5.2 Application Impact Metrics . . . . . . . . . . . . . . . . 7 5.2.1 Discard Octets Metric . . . . . . . . . . . . . . . . . 7 5.2.2 Frame Impairment Summary Metrics . . . . . . . . . . . 8 5.2.3 Jitter Buffer Metrics . . . . . . . . . . . . . . . . . 8 5.3 Recovery metrics . . . . . . . . . . . . . . . . . . . . . 9 5.3.1 Post-repair Packet Count Metrics . . . . . . . . . . . 9 5.3.2 Run Length Encoded Metric for Post-repair . . . . . . . 9 6 Security Considerations . . . . . . . . . . . . . . . . . . . . 9 7 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 10 8 Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . 10 9 References . . . . . . . . . . . . . . . . . . . . . . . . . . 10 9.1 Normative References . . . . . . . . . . . . . . . . . . . 10 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12 Expires August 18, 2014 [Page 2] INTERNET DRAFT February 14, 2014 1 Introduction Web based real-time communication (WebRTC) is becoming prevalent. To help measure the quality of the WebRTC services better, applications need to be able to estimate the service quality and to inform the application about network problems. If sufficient information (metrics or statistics) is provided to the applications, it can function better in providing better media quality. [RTCWEB-REQ] specifies a requirement for statistics, which is listed below for convenient reading. "F38 The browser MUST be able to collect statistics, related to the transport of audio and video between peers, needed to estimate quality of service." [RTCWEB-STAT] describes a registration procedure for choosing metrics reported by the JavaScript API. It also identifies basic metrics reported in the RTCP Sender and Receiver Report (SR/RR) to fulfill this requirement. These basic metrics from RTCP SR/RR may not be sufficient for precise quality monitoring or troubleshooting. They are better to be complemented with correspondent metrics defined in RTCP XR. Thus, indicating a minimum set of additional statistic metrics would be helpful. In this document, we provide some guidelines on what kind of metrics should be chosen to complement the metrics from basic RTCP SR/RR specified in [RTCWEB-STAT]. 2 Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 3 RTP Statistics in WebRTC Implementations Statistics are generated in browsers in WebRTC implementations. They could be divided into 3 categories: stream level, session level and other aspects. RTP statistics are usually stream level as they are always assigned with corresponding SSRCs, and they could be generated locally or remotely. For those remote RTP statistics, local browsers need standard way to get the information from the other side. [RTCWEB-STAT] defines some RTP statistics generated locally or obtained from the remote side using standard RTCP SR/RR periodically. They are SentPacketCount, SentOctetCount, packetsLost, Jitter, ReceivedPacketCount, ReceivedOctetCount. However, these only provides partial and limited information, which may not be sufficient for diagnosing problems or quality monitoring, example is in Section 5.1.1. RTP Control Protocol Extended Reports [RFC3611] and other extensions discussed in XRBLOCK working group have defined more Expires August 18, 2014 [Page 3] INTERNET DRAFT February 14, 2014 statistics complementing those in RTCP SR/RR. Some of these extended metrics should be considered to use in WebRTC implementations. Section 5 presents some detail discussions on the use of metrics. However, RTCP XR are not mandated in RTCWEB because some of the information conveyed by RTCP XR is highly application dependent. at the current stage, we only consider those extended statistics measured at a local endpoint and useful for a range of WebRTC implementations. RTCP XR may be supported after a successful SDP negotiation between browsers, thereafter the application has access to both local and remote statistics. Statistics can be extracted by the application from the local browser by JavaScript applications using an API. Once the JavaScript application gets this information, they can report it to application servers or 3rd party monitoring systems, which provide quality estimations or diagnosis services for application developers. The endpoint may use some standard protocol e.g., RTCP XR, or a private protocol to send the metrics to a measurement server. But this part is outside the scope of this memo, also outside the scope of RTCWEB, and will not be discussed in more detail. 4 Considerations for Impact of Measurement Interval RTCP extensions like RTCP XR usually share the same timing interval with RTCP SR/RR, i.e., they are sent as compound packets, together with the RTCP SR/RR. Alternatively, the RTCP XR can use a different measurement interval and all XRs using the same measurement interval are compounded together and the measurement interval is indicated in a specific measurement information block [RFC6776]. When using WebRTC Statistics APIs (see section 7 of [WebRTCAPI]), JavaScript applications can query this information at arbitrary intervals. Some applications may choose 1 second or another interval. Statistics generated remotely, e.g. those conveyed by RTCP SR/RR, won't change until the next RTCP interval, even if JavaScript applications query it at a very small interval. But those generated locally, e.g. statistics suggested in this memo, have no such limitation. 5 Candidate Metrics Since following metrics are all defined in RTCP XR which is not mandated in WebRTC, all of them are local. However, if RTCP XR is supported by negotiation between two browsers, following metrics can also be generated remotely and be sent to local by RTCP XR packets. Following metrics are classified into 3 categories: network impact metrics, application impact metrics and recovery metrics. Network Expires August 18, 2014 [Page 4] INTERNET DRAFT February 14, 2014 impact metrics are the statistics recording the information only for network transmission. They are useful for network problem diagnosis. Application impact metrics mainly collect the information in the viewpoint of application, e.g., bitrate, frames rate or jitter buffers. Recovery metrics reflect how well the repair mechanisms, e.g. loss concealment, retransmission or FEC, perform. All of the 3 types of metrics are useful for quality estimations of services in WebRTC implementations. JavaScript application developers could use these metrics to better calculate MoS values or Media Delivery Index (MDI) for their services. 5.1 Network Impact Metrics 5.1.1 Loss and Discard Packet Count Metric In multimedia transport, packets that are received abnormally are classified into 3 types: lost, discarded and duplicate packets. Packet loss may be caused by network devices breakdown, bit-error corruption or serious congestions (packets dropped by an intermediate router queue). Duplicated data packets may be due to network delays which causes the sender to retransmit the original packets. Discarded packets are packets that have been delayed long enough and are considered useless by the receiver. Lost and discarded packets cause problems for multimedia services, as missing data and long delay can cause degradation in service quality, e.g., missing large blocks of contiguous packets (lost or discarded) may cause choppy audio, and long network transmission delay time may cause audio or video buffering. RTCP SR/RR defines a metric for counting the total number of RTP data packets that have been lost since the beginning of reception. But this statistic doesn't distinguish lost packets from discarded and duplicate packets. Packets that arrive late and are discarded are not treated as lost, and duplicate packets will be regarded as a normally received packet. This metric is misleading if many duplicate packets are received or packets discarded, which causes the quality of media transport to look okay from the statistic while actually users are experiencing bad service quality, because packets are still missing. So in such cases, it's better to use more accurate metrics in addition to those defined in RTCP SR/RR. The lost packets and duplicated packets metrics defined in Statistics Summary Report Block of [RFC3611] extend the information of loss carried in standard RTCP SR/RR. They explicitly give an account of lost and duplicated packets. Lost packets counts are useful for network problem diagnosis. It's better to use the loss packets metrics of [RFC3611] to indicated the packet lost counting instead of the cumulative number of packets lost metric of [RFC3550]. Duplicated packets are usually rare and have little effect on QoS evaluation. So it is not suitable to be used in WebRTC. Expires August 18, 2014 [Page 5] INTERNET DRAFT February 14, 2014 Using loss metrics without considering discard metrics may result in inaccurate quality evaluation, as packet discard due to jitter is often more prevalent than packet loss in modern IP networks. The discarded metric specified in [RFC7002] counts the number of packets discarded due to the jitter. It augments the loss statistics metrics specified in standard RTCP SR/RR. For those RTCWEB services with jitter buffer requiring precise quality evaluation and accurate troubleshooting, this metric is useful as a complement to the metrics of RTCP SR/RR. 5.1.2 Burst/Gap Pattern Metrics for Loss and Discard RTCP SR/RR defines coarse metrics regarding loss statistics, the metrics are all about per call statistics and not detailed enough to capture some transitory nature of the impairments like bursty packet loss. Even if the average packet loss rate is low, the lost packets may occur during short dense periods, resulting in short periods of degraded quality. Distributed burst provides a higher subjective quality than a non-burst distribution for low packet loss rates whereas for high packet loss rates the converse is true. So capturing burst gap information is very helpful for quality evaluation and locating impairments. If RTCWEB services have the requirement to evaluate the services quality, burst gap metrics provides more accurate information than RTCP SR/RR. [RFC3611] introduces burst gap metrics in VoIP report block. These metrics record the density and duration of burst and gap periods, which are helpful in isolating network problems since bursts correspond to periods of time during which the packet loss/discard rate is high enough to produce noticeable degradation in audio or video quality. Burst gap related metrics are also introduced in [RFC7003] and [RFC6958] which define two new report blocks for usage in a range of RTP applications beyond those described in RFC3611. These metrics distinguish discarded packets from loss packets that occur in the bursts period and provides more information for diagnosing network problems. Besides that, the metric number of bursts counts the burst events which could provide useful information to evaluate the frequency of burst occurrences. So if WebRTC services have the requirement to do quality evaluation and observe when and why quality degrades, these metrics should be considered. 5.1.3 Run Length Encoded Metrics for Loss, Discard Run-length encoding uses a bit vector to encode information about the packet. Each bit in the vector represents a packet and depending on the signaled metric it defines if the packet was lost, duplicated, discarded, or repaired. An endpoint typically uses the run length encoding to accurately communicate the status of each packet in the Expires August 18, 2014 [Page 6] INTERNET DRAFT February 14, 2014 interval to the other endpoint. [RFC3611], [XRBLOCK-DISCARDRLE] define run-length encoding for lost and duplicate packets, discarded packets. For WebRTC, the application could benefit from the additional information. If losses occur after discards, an endpoint may be able to correlate the two run length vectors to identify congestion- related losses, i.e., a router queue became overloaded causing delays and then overflowed. If the losses are independent, it may indicate bit-error corruption. In the case of RTCP XR supported, this type of metrics are not suggested to use due to their enormous amount of data. Whereas, for local case, it's fine to have them. 5.1.3 ECN related Metrics ECN can be used to minimize the impact of congestion on real-time multimedia traffic. The use of ECN provides a way for the network to send congestion control signals to the media transport without having to impair the media. Unlike packet loss, ECN signals unambiguously indicate congestion to the transport as quickly as feedback delays allow and without confusing congestion with losses that might have occurred for other reasons such as transmission errors. ECN related metrics, such as ECN-CE Counter found in [RFC6679], could be used to show the cumulative number of RTP packets received from this SSRC since the receiver joined the RTP session that were ECN-CE marked, including ECN-CE marks in any duplicate packets. It is useful for detecting network congestion status before the actual packet loss occurs. Media senders can control how they reduce their transmission rate and hence media quality, rather than responding to and trying to conceal the effects of unpredictable packet loss. It is hence recommended that ECN related metrics (either ECN Feedback Report and ECN Summary Report) be considered for RTCWEB applications in ECN-enabled networks. The definition of these metrics could be found in [RFC6679]. 5.2 Application Impact Metrics 5.2.1 Discard Octets Metric The metric reports the cumulative size of the packets discarded in the interval, it is complementary to number of discarded packets. An application measures sent octets and received octets to calculate sending rate and receiving rate, respectively. The application can calculate the actual bitrate in a particular interval by subtracting the discarded octets from the received octets. Expires August 18, 2014 [Page 7] INTERNET DRAFT February 14, 2014 For WebRTC, discarded octets supplements the sent and received octets and provides an accurate method for calculating the actual bitrate which is an important parameter to reflect the quality of the media. The discarded bytes metric is defined in [XRBLOCK-DISCARDBYTES]. 5.2.2 Frame Impairment Summary Metrics RTP has different framing mechanisms for different payload types. For audio streams, a single RTP packet may contain one or multiple audio frames, each of which has a fixed length. On the other hand, in video streams, a single video frame may be transmitted in multiple RTP packets. The size of each packet is limited by the Maximum Transmission Unit (MTU) of the underlying network. However, statistics from standard SR/RR only collect information from transport layer, which may not fully reflect the quality observed by the application. Video is typically encoded using two frame types i.e., key frames and derived frames. Key frames are normally just spatially compressed, i.e., without prediction from other pictures. The derived frames are temporally compressed, i.e., depend on the key frame for decoding. Hence, Key frames are much larger in size than derived frames. The loss of these key frames results in a substantial reduction in video quality. Thus it is meaningful to consider this application layer information in WebRTC implementations, which influence sender strategies to mitigate the problem or require the accurate assessment of users' quality of experience. The following metrics can also be considered for WebRTC's Statistics API: number of discarded key frames, number of lost key frames, number of discarded derived frames, number of lost derived frames. These metrics could be used to calculate Media Loss Rate (MLR) of MDI. Details of the definition of these metrics are in [RFC7003]. Besides these, the metric, number of frames, should also be considered since frame rate, an important parameter for quality estimation, could be calculated from it. 5.2.3 Jitter Buffer Metrics The size of the jitter buffer affects the end-to-end delay on the network and also the packet discard rate. When the buffer size is too small, slower packets are not played out and dropped, while when the buffer size is too large, packets are held longer than necessary and consequently reduce conversational quality. Measurement of jitter buffer should not be ignored in the evaluation of end user perception of conversational quality. Jitter buffer related metrics, such as maximum and nominal jitter buffer, could be used to show how the jitter buffer behaves at the receiving end of RTP stream. They are useful for providing better end-user quality of experience (QoE) when jitter buffer factors are used as inputs to calculate MoS values. Expires August 18, 2014 [Page 8] INTERNET DRAFT February 14, 2014 Thus for those cases, jitter buffer metrics could be considered. The definition of these metrics could be found in [RFC7005]. 5.3 Recovery metrics 5.3.1 Post-repair Packet Count Metrics Error-resilience mechanisms, like RTP retransmission or FEC, are optional in RTCWEB because the overhead of the repair bits adding to the original streams. But they do help to greatly reduce the impact of packet loss and enhance the quality of transmission. Web applications could support certain repair mechanism after negotiation between both sides of browsers when needed. For these web applications using repair mechanisms, providing some statistic information for the performance of their repair mechanisms could help to have a more accurate quality evaluation. The un-repaired packets count and repaired loss count defined in [XRBLOCK-PRCOUNT] provide the recovery information of the error- resilience mechanisms to the monitoring application or the sending endpoint. The endpoint can use these metrics to ascertain the ratio of repaired packets to lost packets. Including this kind of metrics helps the application evaluate the effectiveness of the applied repair mechanisms. 5.3.2 Run Length Encoded Metric for Post-repair [RFC5725] defines run-length encoding for post-repair packets. When using error-resilience mechanisms, the endpoint can correlate the loss run length with this metric to ascertain where the losses and repairs occurred in the interval. This provides more accurate information for recovery mechanisms evaluation than those in Section 5.3.1. However, it is not suggested to use due to their enormous amount of data when RTCP XR are supported. For WebRTC, the application may benefit from the additional information. If losses occur after discards, an endpoint may be able to correlate the two run length vectors to identify congestion- related losses, i.e., a router queue became overloaded causing delays and then overflowed. If the losses are independent, it may indicate bit-error corruption. Lastly, when using error-resilience mechanisms, the endpoint can correlate the loss and post-repair run lengths to ascertain where the losses and repairs occurred in the interval. For example, consecutive losses are likely not to be repaired by a simple FEC scheme. 6 Security Considerations Expires August 18, 2014 [Page 9] INTERNET DRAFT February 14, 2014 The monitoring activities are implemented between two browsers or browser-to-server. Also encryption procedures, such as those being suggested for a Secure RTCP (SRTCP), can be used. It is believed that monitoring in RTCWEB introduces no new security considerations beyond those described in [RTCWEB-RTPUSAGE] and [RTCWEB-SECURITY]. 7 IANA Considerations There is no IANA action in this document. 8 Acknowledgement The authors would like to thank Colin Perkins, Al Morton, and Shida Schubert for their valuable comments and suggestions on this document. 9 References 9.1 Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003. [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed., "RTP Control Protocol Extended Reports (RTCP XR)", RFC 3611, November 2003. [RTCWEB-REQ] Holmberg, C., Hakansson, S., and G. Eriksson, "Web Real- Time Communication Use-cases and Requirements", I-D.ietf- rtcweb-use-cases-and-requirements, December 2012. [RTCWEB-STAT] Alvestrand, H., "A Registry for WebRTC statistics identifiers", I-D.alvestrand-rtcweb-stats-registry, September 24, 2012. [RTCWEB-RTPUSAGE] Perkins, C., Westerlund, M., and J. Ott, "Web Real- Time Communication (WebRTC): Media Transport and Use of RTP", I-D.ietf-rtcweb-rtp-usage, February 2013. [RTCWEB-SECURITY] Rescorla, E., "Security Considerations for RTC- Web", I-D.ietf-rtcweb-security, January 2013. [RFC6679] Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P., and K. Carlberg, "Explicit Congestion Notification (ECN) Expires August 18, 2014 [Page 10] INTERNET DRAFT February 14, 2014 for RTP over UDP", RFC 6679, August 2012. [RFC6792] Wu, Q., Ed., Hunt, G., and P. Arden, "Monitoring Architecture for RTP", RFC 6792, November 2012. [RFC7002] Hunt, G., Clark, A., Zorn, G., and Q. Wu, "RTCP XR Report Block for Discard Metric Reporting", RFC 7002, September 2013. [XRBLOCK-DISCARDBYTES] Singh, V., Ott, J., Curcio, I.D.D., "RTP Control Protocol (RTCP) Extended Reports (XR) for Bytes Discarded Metric", I-D.ietf-xrblock-rtcp-xr-bytes- discarded-metric, October 2013 [XRBLOCK-PRCOUNT] Huang, R., Singh, V., "RTP Control Protocol (RTCP) Extended Report (XR) for Post-Repair Non-Run Length Encoding (RLE) Loss Count Metrics", I-D.huang-xrblock- post-repair-loss-count, September 2013. [RFC7003] Hunt, G., Clark, A., and Q. Wu, Ed., "RTCP XR Report Block for Burst/Gap Discard Metric Reporting", RFC 7003, September 2013. [RFC6958] Hunt, G., Clark, A., Zhang, S., Ed., "RTCP XR Report Block for Burst/Gap Loss Metric Reporting", RFC 6958, April 2013. [XRBLOCK-DISCARDRLE] Singh, V., Ott, J., Curcio, I.D.D., "RTP Control Protocol (RTCP) Extended Reports (XR) for Run Length Encoding (RLE) of Discarded Packets", I-D.ietf-xrblock- discard-rle-metrics, October 2013. [RFC5725] Begen, A., Hsu, D., Lague, M., "Post-Repair Loss RLE Report Block Type for RTP Control Protocol (RTCP) Extended Reports (XRs)", February 2010 [RFC7005] Clark, A., Singh, V., and Q. Wu, "RTCP XR Report Block for Jitter Buffer Metric Reporting", RFC 7005, September 2013 [RFC6798] Clark, A., Wu, Q., Ed., "RTCP Control Protocol (RTCP) Extended Report (XR) Block for Packet Delay Variation Metric Reporting", November 2012. [RFC7003] Zorn, G., Schott, R., Wu, Q., Huang, R., "RTP Control Protocol (RTCP) Extended Report (XR) Blocks for Summary Statistics Metrics Reporting", September 2013 [WebRTCAPI] Bergkvist, A., Burnett, D., Jennings, C., Ed., Expires August 18, 2014 [Page 11] INTERNET DRAFT February 14, 2014 http://dev.w3.org/2011/webrtc/editor/webrtc.html, June 2013 Authors' Addresses Rachel Huang Huawei 101 Software Avenue, Yuhua District Nanjing 210012 China EMail: rachel.huang@huawei.com Roni Even Huawei 14 David Hamelech Tel Aviv 64953 IsraelOctober 11, 2013October 11, 2013 EMail: roni.even@mail01.huawei.com Varun Singh Aalto University School of Electrical Engineering Otakaari 5 A Espoo, FIN 02150 Finland Email: varun@comnet.tkk.fi URI: http://www.netlab.tkk.fi/~varun/ Dan Romascanu Avaya Email: dromasca@avaya.com Lingli Deng China Mobile Expires August 18, 2014 [Page 12] INTERNET DRAFT February 14, 2014 Email: denglingli@chinamobile.com Expires August 18, 2014 [Page 13]