XRBLOCK Working Group Alan Clark Internet Draft Telchemy Intended status: Standards Track Martin Kastner Expires: May 17, 2012 Telchemy Geoff Hunt Unaffiliated November 14, 2011 RTCP XR Report Block for QoE Metrics Reporting draft-clark-xrblock-rtcp-xr-qoe-00 Abstract This document defines an RTCP XR Report Block that allows the reporting of QoE metrics for use in voice, audio and video services. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on May 17, 2012. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Clark & Kastner [Page 1] RTCP XR QoE Metrics November 2011 1. Introduction 1.1. QoE Metrics Report Block This draft defines a new block types to augment those defined in RFC3611 for use in reporting QoE metrics. QoE metrics consider the impact of a range of transmission and payload (content) related impairments on the quality of a service from the user viewpoint. 1.2. RTCP and RTCP XR Reports The use of RTCP for reporting is defined in RFC3550 [2]. RFC3611 [3] defined an extensible structure for reporting using an RTCP Extended Report (XR). This draft defines a new Extended Report block that MUST be used as defined in RFC3550 and RFC3611. 1.3 Performance Metrics Framework The Performance Metrics Framework [9] provides guidance on the definition and specification of performance metrics. Metrics described in this draft either reference external definitions or define metrics generally in accordance with the guidelines in [9]. 1.4 Applicability This memo applies to any application of RTP for which QoE measurement algorithms are defined. 2. Definitions 2.1 QoE Metrics A QoE ("Quality of Experience") metric is intended to provide a measure that is indicative of the user's view of a service. This is commonly expressed as a MOS ("Mean Opinion Score") which usually (but not always) is a 1.0-5.0 numerical scale in which a 1.0 represents "Unacceptable" and 5.0 represents "Excellent". True MOS scores are obtained using subjective testing, and tend vary from test to test. Subjective testing is also not suitable for measuring the quality of operational services and hence it is common practice to use objective algorithms to estimate subjective quality. During the development of such QoE algorithms, there is extensive comparison against both subjective test data and data from other "trusted" objective test tools. Clark & Kastner [Page 2] RTCP XR QoE Metrics November 2011 ITU-T Recommendation P.564 defines a methodology for verifying the performance of QoE estimation algorithms for Voice over IP services. There is standardization work underway related to QoE metrics for video and audio. The continuous progression of work in this area means that new algorithms may be defined in the future, hence this memo does make provision for new algorithms. Implementors are advised that IPR disclosures have been made in respect of most known QoE estimation algorithms and they should check the IPR disclosure databases and policies of the relevant standards organizations (for example ITU and ETSI). ITU-T Recommendation P.800.1 describes terminology that should be use for MOS scores used to describe Speech quality. This uses the abbreviations LQ and CQ for Listening and Conversational Quality respectively, and extends these using O for Objective, E for Estimated and S for Subjective. Hence an objectively measured listening quality MOS score would be denoted MOS-LQO. MOS scores typically use a common scale of 1 to 5 and are scaled for comparison with subjectively measured MOS. MOS scores for narrowband speech and wideband speech, or for low resolution video and high resolution video are typically placed into the same range. This occurs because a subjective test is usually a comparitive test amongst similar codecs or devices. Hence a high quality AMR-WB or G.722 wideband voice call may have a lower MOS score than a narrowband G.729 call, even though the quality is higher. Similary, a video subjective test typically uses devices with similar resolution and hence a high definition system may have the same MOS score as a standard definition system. ITU-T P.800.1 addressed this issue of MOS scaling through the use of an additional N or W qualifier to denote Narrowband or Wideband. So a MOS-LQON score is an objectively measured listening quality MOS for narrowband (8kHz sample rate) conditions. Some codecs are able to switch dynamically between narrowband and wideband, which is addressed by the the "M" or mixed qualifier. The issue for audio video MOS is very similar to that of speech. This is addressed by recent work in ITU-T [11] which introduced the idea of Absolute and Relative MOS. Absolute MOS "does" include the effects of image resolution whereas Relative MOS does "not". This draft presumes that ITU-T will adopt similar terminology to P.800.1 for video MOS. [Editors note, will need updating as ITU update relevant standards] Two cases of MOS-CQ have been treated separately in this draft. The first of these is MOS-CQEN, which is an Estimated (not measured) MOS based on ITU-T G.107. The MOS value is calculated by first calculating an R (or RCQ) value and then converting Clark & Kastner [Page 3] RTCP XR QoE Metrics November 2011 this to a MOS. This conversion leads to a MOS score that is typically higher than current subjective test data (4.45 vs 4.2), which can lead to difficulty interpreting the values. The second case is MOS-CQEN-TTC which is related to a Japanese national standard - TTC JJ201.01. JJ201.01 is based on G.107 however Japanese MOS scores are typically much lower than those in other countries and a MOS score for G.711 would be 3.8 in Japan versus 4.2 for a typical subjective test and 4.45 for G.107. It is extremely important that the correct MOS is referenced. For example a MOS of 3.6 would represent a small degree of degration (0.2) using the Japanese JJ201.01 scaling but a very large degradation ( 0.85) using G.107 scaling. 2.2 Channel Certain types of encoder (for example stereo audio codecs) incorporate multiple audio or video channels into a single encoded stream which is then packetized and carried in RTP or MPEG Transport. Within the scope of this memo, the term "channel" applies to this definition only - if multiple audio or video streams are carried either in separate RTP sessions (identified by an SSRC) or MPEG Transport program streams (identified by a PID) then the Measurement Identifier block MUST be used to identify the stream to which metrics apply. 3. QoE Metrics Block 3.1 Report Block Structure 0 1 2 3 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | BT=N | I | Tag | block length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Chan | Type | Calc alg | QoE Metric | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ .......... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Chan | Type | Calc alg | QoE Metric | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3.2 Definition of Fields in QoE Metric Report Block block type (BT): 8 bits A QoE Report Block is identified by the constant QOEX. [Note to RFC Editor: please replace QOEX with the IANA provided RTCP XR block type for this block.] Clark & Kastner [Page 4] RTCP XR QoE Metrics November 2011 Measurement Type Indication (I): 2 bits This field is used to indicate whether the QoE Metrics are Sampled, Interval or Cumulative metrics, that is, whether the reported values applies to the most recent measurement interval duration between successive metrics reports (I=10) (the Interval Duration), to the accumulation period characteristic of cumulative measurements (I=11) (the Cumulative Duration) or is a sampled instantaneous value (I=01). Numerical values for interval or duration are provided in the Measurement Identifier block referenced by the tag field below. Measurement Identifier association (tag): 6 bits This field is used to identify the Measurement Identifier block which describes this measurement. The relevant Measurement Identifier block has the same tag value as the QoE block Note that there may be more than one Measurement Identifier block per RTCP packet. Block length: 16 bits The length of this report block in 32-bit words minus one. Channel The channel number of the audio or video stream to which this metric applies Type 0000000 - 0011111 Speech QoE Scores 0100000 - 0111111 Audio QoE Scores 1000000 - 1011111 Video QoE Scores 1100000 - 1111111 Other application QoE Scores Clark & Kastner [Page 5] RTCP XR QoE Metrics November 2011 Speech QoE Scores (see ITU-T P.800.1 [10] for definitions) 0000000 MOS-LQON - Listening Quality MOS (Narrowband) 0000001 MOS-LQOW - Listening Quality MOS (Wideband) 0000010 MOS-LQOU - Listening Quality MOS (Ultra wideband) 0000011 MOS-LQOM - Listening Quality MOS (Mixed) 0000100-0000111 - Reserved 0001000 MOS-CQON - Conversational Quality MOS (Narrowband) 0001001 MOS-CQOW - Conversational Quality MOS (Wideband) 0001010 MOS-CQOU - Conversational Quality MOS (Ultra wideband) 0001011 MOS-CQOM - Conversational Quality MOS (Mixed) 0001100 MOS-CQEN - Conversational Quality MOS (Narrowband) Scaled per ITU-T G.107 0001101 MOS-CQEN-TTC - Conversational Quality MOS (Narrowband) Scaled per TTC JJ201.01 [8] (Japan) 0001110-0001111 - Reserved 0010000 MOS-TQON - Talking Quality MOS (Narrowband) 0010001 MOS-TQOW - Talking Quality MOS (Wideband) 0010010 MOS-TQOU - Talking Quality MOS (Ultra wideband) 0010011 MOS-TQOM - Talking Quality MOS (Mixed) 0010100 - 0010111 - Reserved 0011000 R-LQ - R Factor - Listening Quality 0011001 R-CQ - R Factor - Conversational Quality [6] 0011010 - 0011111 - Reserved Audio QoE Scores (see ITU-T P.??? and [11]) 0100000 Absolute MOS-AQOA - Audio Quality MOS, absolute scaling 0100001 Relative MOS-AQOR - Audio Quality MOS, relative scaling Video and Multimedia QoE Scores (see ITU-T P.??? and [11]) 1000000 Absolute MOS-VQOA - Video Quality MOS, absolute scaling 1000001 Relative MOS-VQOR - Video Quality MOS, relative scaling 1000100 Absolute MOS-AQOA - Audio-Video Quality MOS, absolute 1000101 Relative MOS-AQOR - Audio-Video Quality MOS, relative Other application QoE Scores 1100000 - 1111111 Reserved for other interactive applications that use RTP for communication Clark & Kastner [Page 6] RTCP XR QoE Metrics November 2011 Calculation Algorithm 0 - ITU-T P.564 Compliant Algorithm [5] (Voice) 1 - G.107 [6] (Voice) 2 - G.107 / ETSI TS 101 329-5 Annex E [6,7] (Voice) 3 - TTC JJ201.01 [8] (Japan) 4 - Reserved for ITU-T P.NAMS 5 - Reserved for ITU-T P.NBAMS 255 - Indicated via SDP QoE Metric A 8:8 integer scaled representation of the QoE metric value. This allows values in the range 0.0 to 255.996 to be represented. 4. SDP Signaling RFC3611 [3] defines the use of SDP (Session Description Protocol) [4] for signaling the use of XR blocks. XR blocks MAY be used without prior signaling. This section augments the SDP [4] attribute "rtcp-xr" defined in RFC3611[3] by providing a "xr-format" to signal the use of the report block defined in this document. rtcp-xr-attrib = "a=" "rtcp-xr" ":" [xr-format *(SP xr-format)] CRLF (defined in RFC3611) xr-format = xr-format / qoe-metrics qoe-metrics = "qoe-metrics" [EQUAL word] DIGIT = %x30-39 format-ext = non-ws-string non-ws-string = 1*(%x21-FF) CRLF = %d13.10 5. IANA Considerations This document creates a new block type within the IANA "RTCP XR Block Type Registry" called the QoE Metrics, and a new [new-xrblock] parameter within the "RTCP XR SDP Parameters Registry". Clark & Kastner [Page 7] RTCP XR QoE Metrics November 2011 6. Security Considerations RTCP reports can contain sensitive information since they can provide information about the nature and duration of a session established between two or more endpoints. 7. Contributors 8. References Normative [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [2] Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003. [3] Friedman, T., Caceres, R., and A. Clark, "RTP Control Protocol Extended Reports (RTCP XR)", RFC 3611, November 2003. [4] Handley, M. and V. Jacobson, "SDP: Session Description Protocol", RFC 4566, July 2006. [5] ITU-T Recommendation P.564, Conformance testing for narrowband Coice over IP transmission quality assessment models [6] ITU-T Recommendation G.107, "The E Model, a computational model for use in transmission planning" [7] ETSI TS 101 329-5, QoS Measurement for Voice over IP [8] TTC 201.01 (Japan) A method for speech quality assessment for Coice over IP [9] Clark A., Claise B. "Guidelines for Considering New Performance Metrics Development", RFC6390, October 2011 [10] ITU-T P.800.1 "Mean Opinion Score (MOS) terminology" Informative [11] ITU-T TD483 "Interpretation of MOS in different contexts", January 2011 Clark & Kastner [Page 8] RTCP XR QoE Metrics November 2011 Author's Addresses Alan Clark Telchemy Incorporated 2905 Premiere Parkway, Suite 280 Duluth, GA 30097 USA Email: alan.d.clark@telchemy.com Martin Kastner Telchemy Incorporated 2905 Premiere Parkway, Suite 280 Duluth, GA 30097 USA Email: martin.kastner@telchemy.com Geoff Hunt Unaffiliated Clark & Kastner [Page 9]