Network Working Group D. Malas Internet Draft Level 3 Communications Expires: November 2006 May 22, 2006 SIP Performance and Reliability Metrics draft-malas-performance-metrics-00.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on November 22, 2006. Abstract This document defines the use of industry recommended reliability metrics for use with the SIP. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [1]. Table of Contents 1. Introduction...................................................2 malas Expires November 22, 2006 [Page 1] Internet-Draft SIP Reliability and Performance Metrics May 2006 2. Metrics........................................................3 2.1. Answer Seizure Ratio (ASR)................................3 2.2. Post Invite Delay (PID)...................................3 2.3. Call Hold Time (CHT)......................................4 2.4. Ineffective Attempts (IA).................................5 2.5. Cutoff Calls (CC).........................................5 2.6. Call Success Rate (CSR)...................................5 2.7. Defects Per Million (DPM).................................6 3. Security Considerations........................................6 4. IANA Considerations............................................6 5. Conclusions....................................................6 6. Acknowledgments................................................6 7. References.....................................................6 7.1. Normative References......................................6 7.2. Informative References....................................7 Author's Addresses................................................7 Intellectual Property Statement...................................7 Disclaimer of Validity............................................8 Copyright Statement...............................................8 Acknowledgment....................................................8 1. Introduction SIP has become a standard among many service providers, vendors, and end users. Although there are many different standards for measuring the reliability and performance of signaling protocols, none of these have been adapted for use with SIP. This document is intended for providing a guideline for the above listed entities in providing a standard approach for measuring and reporting SIP reliability and performance metrics. This will allow a common approach and understanding of expectations between service providers, vendors, and the users of those services. Not all metrics for reliability and performance map to all applications of the SIP. This document provides an overview of many different metrics, which may be used as an individual or set of metrics necessary based on the use of SIP. There are many metrics available for determining both reliability and performance. Although this document contains a number of them, it is not intended to be exhaustive. Instead, it is designed to provide a common sub-set with a common agreed upon definition. malas Expires November 22, 2006 [Page 2] Internet-Draft SIP Reliability and Performance Metrics May 2006 2. Metrics 2.1. Answer Seizure Ratio (ASR) ASR [4] is defined as the number of INVITE requests resulting in a 200 OK response, to the total number of attempted INVITE requests. # of INVITE Requests w/ associated 200OK ASR = ---------------------------------------- Total # of INVITE Requests UA1 UA2 | | |INVITE | |---------------------> | | /\ 100| | <---------||----------| | || | | Seizure | | || 180| | <---------||----------| | \/ 200| | <---------------------| | | | | 2.2. Post Invite Delay (PID) In a successful request attempt, PID [4] is defined as the time interval from the moment the INVITE message containing the necessary information is passed by the originating agent or user to the intended mediation or destination agent until a response is received indicating an audible or visual status of the INVITE request. When SIP is utilized for traditional telephony-like dialing, this metric is synonymous with the Post Dial Delay (PDD). In SIP, the message indicating status would be a 180 and 183 message received in response to an INVITE request. In some cases, a 180 or 183 message is not received, but rather a 200 message is received as the first status message instead. In these situations, the 200 message would be used to calculate the interval. In a failed request attempt, the interval is defined from the INVITE request and a failure indication status response. A failure response is described as a 4XX, 5XX, or possible 6XX message. malas Expires November 22, 2006 [Page 3] Internet-Draft SIP Reliability and Performance Metrics May 2006 PID = Time of Status Indicative Response – Time of INVITE SUM (Time of Status Indicative Response – Time of INVITE) APID = --------------------------------------------------------- SUM # of INVITE Requests UA1 UA2 | | |INVITE | |---------------------> | | /\ 100| | <---------||----------| | PID | | || | | \/ 180| | <---------------------| | | 2.3. Call Hold Time (CHT) CHT [4] is usually calculated as an average and is defined as the duration of a call from receipt of a 200 OK and an associated BYE message indicating call completion. CHT = Time of BYE – Time of 200 OK SUM (Time of BYE – Time of 200 OK) ACHT = ----------------------------------------- SUM # of INVITE w/ 200OK & BYE UA1 UA2 | | |INVITE | |---------------------> | | 100| | <---------------------| | 180| | <---------------------| | 200| | <---------------------| | /\ | | || | | CHT | | || | |BYE \/ | |---------------------> | malas Expires November 22, 2006 [Page 4] Internet-Draft SIP Reliability and Performance Metrics May 2006 2.4. Ineffective Attempts (IA) IA [3] utilizing SIP is defined as a proxy or agent, which internally releases a call with a failed or congested condition. The following failure responses may provide a guideline for this criterion: 1. 408 Request Timeout 2. 500 Server Internal Error 3. 503 Service Unavailable 4. 504 Server Timeout # of IA Attempts IA % = ----------------------- # of INVITE Requests UA1 UA2 | | |INVITE | |---------------------> | | 503| | <---------------------| |ACK | |---------------------> | | | 2.5. Cutoff Calls (CC) CC [3] utilizing SIP is defined as an active session, which is terminated due to a condition other than a normal disconnected such as a BYE or CANCEL. # of CC’s CC % = ----------------------------------------- # of INVITE Requests w/ associated BYE 2.6. Call Success Rate (CSR) CSR is intended to provide a metric, which summarizes the overall service provided. A recommended method for calculating CSR is through utilizing a combination of IA and CC. CSR = 100% - (IA% - CC%) malas Expires November 22, 2006 [Page 5] Internet-Draft SIP Reliability and Performance Metrics May 2006 2.7. Defects Per Million (DPM) DPM was originally developed by Motorola, and is now more commonly referred to as Six Sigma quality management. It defines a method for calculating the number of defects per million “items” produced. Many service providers are using this metric. In SIP, this would be calculated as the number of “defective” failure responses per million final release responses received in response to an INVITE request. The following failure responses may provide a guideline for this defective criterion: 1. 500 Server Internal Error 2. 503 Service Unavailable 3. 504 Server Timeout 3. Security Considerations Security should be considered in the aspect of securing the relative data utilized in providing input to the above calculations. All other aspects of security should be considered as described in [2]. 4. IANA Considerations There are no IANA considerations at this time. 5. Conclusions The proposed guideline provides a description of common reliability and performance metrics, and their defined use with SIP. The use of these metrics will provide a common viewpoint across all vendors, service providers, and customers. 6. Acknowledgments TBD 7. References 7.1. Normative References [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. malas Expires November 22, 2006 [Page 6] Internet-Draft SIP Reliability and Performance Metrics May 2006 [2] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M. and E. Schooler, "SIP: Session Initiation Protocol", RFC 3261, June 2002. [3] Bellcore, “LSSGR: Reliability, Section 12”, GR-512-CORE Issue 2, January 1998. [4] ITU-T, “Terms and Defintions of Traffic Engineering”, Recommendation E.600, March 1993. 7.2. Informative References Author's Addresses Daryl Malas Level 3 Communications LLC 1025 Eldorado Blvd. Broomfield, CO 80021 USA EMail: daryl.malas@level3.com Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. malas Expires November 22, 2006 [Page 7] Internet-Draft SIP Reliability and Performance Metrics May 2006 Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society (2006). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. malas Expires November 22, 2006 [Page 8]