Network Working Group INTERNET-DRAFT Expires in: August 2004 Scott Poretsky Quarry Technologies Shankar Rao Qwest Communications February 2004 Terminology for Accelerated Stress Benchmarking Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. ABSTRACT This document provides the Terminology for performing Stress Benchmarking of networking devices. The three phases of the Stress Test: Startup, Instability and Recovery are defined along with the benchmarks and configuration terms associated with the each phase. Also defined are the Benchmark Planes fundamental to stress testing configuration, setup and measurement. The terminology is to be used with the companion framework and methodology documents. Table of Contents 1. Introduction ............................................... 3 2. Existing definitions ....................................... 3 3. Term definitions............................................ 3 Poretsky and Rao [Page 1] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 3.1 General Terms............................................. 3 3.1.1 Benchmark Planes...................................... 3 3.1.2 Configuration Sets.................................... 4 3.1.3 Startup Conditions.................................... 4 3.1.4 Instability Conditions................................ 5 3.1.5 Aggregate Forwarding Rate............................. 6 3.1.6 Controlled Session Loss............................... 6 3.1.7 Uncontrolled Session Loss............................. 6 3.2 Benchmark Planes.......................................... 7 3.2.1 Control Plane......................................... 7 3.2.2 Data Plane............................................ 7 3.2.3 Management Plane...................................... 8 3.2.4 Security Plane........................................ 8 3.3 Startup................................................... 9 3.3.1 Startup Phase......................................... 9 3.3.2 Benchmarks............................................10 3.3.2.1 Stable Aggregate Forwarding Rate..................10 3.3.2.2 Stable Session Count..............................11 3.3.3 Control Plane.........................................11 3.3.3.1 Control Plane Configuration Set...................11 3.3.3.2 Control Plane Startup Conditions..................12 3.3.4 Data Plane............................................12 3.3.4.1 Data Plane Configuration Set......................12 3.3.4.2 Traffic Profile...................................13 3.3.5 Management Plane......................................13 3.3.5.1 Management Plane Configuration Set................13 3.3.6 Security Plane........................................14 3.3.6.1 Security Plane Configuration Set..................14 3.3.6.2 Security Plane Startup Conditions.................15 3.4 Instability...............................................15 3.4.1 Instability Phase.....................................15 3.4.2 Benchmarks............................................16 3.4.2.1 Unstable Aggregate Forwarding Rate................16 3.4.2.2 Degraded Aggregate Forwarding Rate................16 3.4.2.3 Average Degraded Aggregate Forwarding Rate........17 3.4.2.4 Unstable Uncontrolled Sessions Lost...............17 3.4.3 Instability Conditions................................18 3.4.3.1 Control Plane Instability Conditions..............18 3.4.3.2 Data Plane Instability Conditions.................18 3.4.3.3 Management Plane Instability Conditions...........19 3.4.3.4 Security Plane Instability Conditions.............19 3.5 Recovery..................................................20 3.5.1 Recovery Phase........................................20 3.5.2 Benchmarks............................................20 3.5.2.1 Recovered Aggregate Forwarding Rate...............20 3.5.2.2 Recovery Time.....................................21 3.5.2.3 Recovered Uncontrolled Sessions Lost..............21 4. Security Considerations.....................................22 5. References..................................................22 6. Author's Address............................................22 7. Full Copyright Statement....................................23 Appendix 1 - White Box Benchmarks..............................23 Poretsky and Rao [Page 2] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 1. Introduction Routers in an operational network are simultaneously configured with multiple protocols and security policies while forwarding traffic and being managed. To accurately benchmark a router for deployment it is necessary to test that router in operational conditions by simultaneously configuring and scaling network protocols and security policies, forwarding traffic, and managing the device. It is helpful to accelerate these network operational conditions so that the router under test can be benchmarked with faster test duration. Testing a router in accelerated network conditions is known as Accelerated Stress Testing. This document provides the Terminology for performing Stress Benchmarking of networking devices. The three phases of the Stress Test: Startup, Instability and Recovery are defined along with the benchmark and configuration terms associated with the each phase. Benchmarks for stress testing are defined using the Aggregate Forwarding Rate and control plane Session Count during each phase of the test. Also defined are the Benchmark Planes fundamental to stress testing configuration, setup and measurement. These are the Control Plane, Data Plane, Management Plane and Security Plane For each plane, the Configuration Set, Startup Conditions, and Instability Conditions are defined. White Box benchmarks are provided in Appendix 1 for additional DUT behavior measurements. The terminology is to be used with the companion framework document [6] and methodology document [7]. 2. Existing definitions RFC 1242 "Benchmarking Terminology for Network Interconnect Devices" and RFC 2285 "Benchmarking Terminology for LAN Switching Devices" should be consulted before attempting to make use of this document. For the sake of clarity and continuity this RFC adopts the template for definitions set out in Section 2 of RFC 1242. Definitions are indexed and grouped together in sections for ease of reference. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 3. Term definitions 3.1 General Terms 3.1.1 Benchmark Planes Definition: The features, conditions, and behavior for the Accelerated Stress Benchmarking. Poretsky and Rao [Page 3] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 Discussion: There are four Benchmark Planes: Control Plane, Data Plane, Management Plane, and Security Plane as shown in Figure 1. The Benchmark Planes define the Configuration, Startup Conditions, Instability Conditions, and Failure Conditions used for the test. ___________ ___________ | Control | | Management| | Plane |___ ___| Plane | | | | | | | ----------- | | ----------- \/ \/ ___________ ___________ | Security | | |<-----------| Plane | | DUT | | | |--->| |<---| ----------- | ----------- | | | | ___________ | | | Data | | |--->| Plane |<---| | | ----------- Figure 1. Router Accelerated Stress Benchmarking Planes Measurement units: N/A Issues: None See Also: Control Plane Data Plane Management Plane Security Plane 3.1.2 Configuration Sets Definition: The features and scaling limits used during the Accelerated Stress Benchmarking. Discussion: There are four Configuration Sets: Control Plane Configuration Set, Data Plane Configuration Set, Management Plane Configuration Set, and Security Plane Configuration Set. Measurement units: N/A Poretsky and Rao [Page 4] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 Issues: None See Also: Control Plane Configuration Set Data Plane Configuration Set Management Plane Configuration Set Security Plane Configuration Set 3.1.3 Startup Conditions Definition: Test conditions that occur at the start of the Accelerated Life Benchmark to establish conditions for the remainder of the test. Discussion: Startup Conditions may cause stress on the DUT and produce failure. Startup Conditions are defined for the Control Plane and Security Plane. Measurement units: N/A Issues: None See Also: Control Plane Startup Conditions Data Plane Startup Conditions Management Plane Startup Conditions Security Plane Startup Conditions 3.1.4 Instability Conditions Definition: Test conditions that occur during the Accelerated Stress Benchmark to produce instability and stress the DUT. Discussion: Instability Conditions are applied to the DUT after the Startup Conditions have completed. Instability Conditions occur for the Control Plane, Data Plane, Management Plane, and Security Plane. Measurement units: N/A Issues: None Poretsky and Rao [Page 5] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 See Also: Control Plane Instability Conditions Data Plane Instability Conditions Management Plane Instability Conditions Security Plane Instability Conditions 3.1.5 Aggregate Forwarding Rate Definition: Sum of forwarding rates for all interfaces on the DUT during the Startup Phase. Discussion: Each interface of the DUT forwards traffic at some measured rate. The Aggregate Forwarding Rate is the sum of forwarding rates for all interfaces on the DUT. Measurement units: pps Issues: None See Also: Startup Phase 3.1.6 Controlled Session Loss Definition: Control Plane sessions that are intentionally brought down during the Stress test. Discussion: The test equipment is able to control protocol session state with the DUT. Measurement units: None Issues: None See Also: Uncontrolled Session Loss 3.1.7 Uncontrolled Session Loss Definition: Control Plane sessions that are in the down state but were not intentionally brought down during the Stress test. Poretsky and Rao [Page 6] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 Discussion: The test equipment is able to control protocol session state with the DUT. The test equipment is also to monitor for sessions lost with the DUT which the test equipment itself did not intentionally bring down. Measurement units: N/A Issues: None See Also: Controlled Session Loss 3.2 Benchmark Planes 3.2.1 Control Plane Definition: The Description of the control protocols enabled for the Accelerated Stress Benchmarking. Discussion: The Control Plane defines the Configuration, Startup Conditions, Instability Conditions, and Failure Conditions of the control protocols used for the test. Control Plane protocols may include routing protocols, multicast protocols, and MPLS protocols. These can be enabled or disabled for a benchmark test. Measurement units: N/A Issues: None See Also: Benchmark Planes Control Plane Configuration Set Control Plane Startup Conditions Control Plane Instability Conditions Control Plane Failure Conditions 3.2.2 Data Plane Definition: The data traffic profile used for the Accelerated Stress Benchmarking. Poretsky and Rao [Page 7] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 Discussion: The Data Plane defines the Configuration, Startup Conditions, Instability Conditions, and Failure Conditions of the data traffic used for the test. The Data Plane includes the traffic and interface profile. Measurement Units: N/A See Also: Benchmark Planes Data Plane Configuration Set Data Plane Startup Conditions Data Plane Instability Conditions Data Plane Failure Conditions 3.2.3 Management Plane Definition: The Management features and tools used for the Accelerated Stress Benchmarking. Discussion: A key component of the Accelerated Stress Benchmarking is the Management Plane to assess manageability of the router under stress. The Management Plane defines the Configuration, Startup Conditions, Instability Conditions, and Failure Conditions of the management protocols and features used for the test. The Management Plane includes SNMP, Logging/Debug, and Statistics Collection. Measurement units: N/A Issues: None See Also: Benchmark Planes Management Plane Configuration Set Management Plane Startup Conditions Management Plane Instability Conditions Management Plane Failure Conditions 3.2.4 Security Plane Definition: The Security features used during the Accelerated Stress Benchmarking. Poretsky and Rao [Page 8] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 Discussion: The Control Plane defines the Configuration, Startup Conditions, Instability Conditions, and Failure Conditions of the security features and protocols used for the test. The Security Plane includes the ACLs, Firewall, Secure Protocols, and User Login. Measurement units: N/A Issues: None See Also: Benchmark Planes Security Plane Configuration Set Security Plane Startup Conditions Security Plane Instability Conditions Security Plane Failure Conditions 3.3 Startup 3.3.1 Startup Phase Definition The portion of the benchmarking test in which the Startup Conditions are generated with the DUT. This begins with the attempt to establish the first session and ends when the last Control Plane session is established. Discussion: The Startup Phase is the first Phase of the benchmarking test preceding the Instability Phase and Recovery Phase. It is specified by the Configuration Sets and Startup Conditions for each Benchmark Plane. The Startup Phase ends and Instability Phase may begin when the Configuration Sets are achieved with the DUT. Measurement Units: None Issues: The 'last control plane session is established' may not be a sufficient indicator that steady-state is achieved and Instability Conditions can be applied to begin the Instability Phase. Poretsky and Rao [Page 9] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 See Also: Benchmark Plane Configuration Sets Startup Conditions Instability Phase Recovery Phase 3.3.2 Benchmarks 3.3.2.1 Stable Aggregate Forwarding Rate Definition: Average rate of traffic forwarded by the DUT during the Startup Phase. Discussion: Stable Aggregate Forwarding Rate is the calculated average the Aggregate Forwarding Rates measured during the Startup Phase. It is recommended that the Aggregate Forwarding Rate is measured at one-second intervals until the Startup Phase ends. Measurement units: pps Issues: The act of the DUT establishing the Startup Conditions could influence the forwarding rate in certain implementations so that this "baseline" for the remainder of the test is lowered. The alternative is to change the definition of Startup Aggregate Forwarding Rate so that it measured during the Startup Phase, but after Startup Conditions are achieved. The disadvantage of this definition would be that it loses measurement of any impact establishing Startup Conditions would have on forwarding rate. When comparing the Startup Aggregate Forwarding Rate benchmark of two devices it is preferred to know the impact establishing Startup Conditions has on Forwarding Rate. The definition was therefore selected so that Stable Aggregate Forwarding Rate is calculated from measurement samples throughout the entire Startup Phase. See Also: Instability Conditions Aggregate Forwarding Rate Stable Aggregate Forwarding Rate Poretsky and Rao [Page 10] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 3.3.2.2 Stable Session Count Definition: Total number of control plane sessions/adjacencies established and maintained by the DUT prior to Instability Conditions being initiated. Discussion: This measurement should be made after the Control Plane Startup Conditions are applied to the DUT. Measurement units: sessions Issues: None See Also: Instability Conditions 3.3.3 Control Plane 3.3.3.1 Control Plane Configuration Set Definition: The routing protocols and scaling values used for the Accelerated Life Benchmarking. Discussion: Control Plane Configuration Set is shown in Figure 2 and specifies the Routing Protocols, Multicast, and MPLS configuration. Specific protocols can be enabled or disabled for a benchmark test. Measurement units: N/A Issues: None ____________ ____________ ____________ | Routing | | Multicast | | MPLS | | Protocols |___ | Protocols | __| Protocols | | | | | | | | | ------------ | ------------ | ------------ | | | | | | | \/ | | ___________ | | | | | |------->| DUT |<------| ``| | ----------- Figure 2. Control Plane Configuration Module Poretsky and Rao [Page 11] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 See Also: Data Plane Configuration Set Management Configuration Set Security Configuration Set 3.3.3.2 Control Plane Startup Conditions Definition: Control Plane conditions that occur at the start of the Accelerated Stress Benchmarking to establish conditions for the remainder of the test. Discussion: Startup Conditions may cause stress on the DUT and produce failure. Startup Conditions for the Control Plane include session establishment rate, number of sessions established and number of routes learned. Measurement units: N/A Issues: None See Also: Startup Conditions Security Plane Startup Conditions Control Plane Configuration Set 3.3.4 Data Plane 3.3.4.1 Data Plane Configuration Set Definition: The data traffic profile enabled for the Accelerated Stress Benchmarking. Discussion: Data Plane Configuration Set includes the Traffic Profile and interfaces used for the Accelerated Stress Benchmarking. Measurement Units: N/A Issues: None See Also: Traffic Profile Poretsky and Rao [Page 12] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 3.3.4.2 Traffic Profile Definition The characteristics of the Offered Load to the DUT used for the Accelerated Stress Benchmarking. Discussion The Traffic Profile specifies the number of packet size(s), packet rate per interface, number of flows, and encapsulation used for the offered load to the DUT. Measurement Units: Traffic Profile is reported as follows: Parameter Units --------- ------ Packet Size(s) bytes Packet Rate(interface) array of packets per second Number of Flows number Encapsulation(flow) array of encapsulation type Issues: None See Also: Data Plane Configuration Set 3.3.5 Management Plane 3.3.5.1 Management Plane Configuration Set Definition: The router management features enabled for the Accelerated Stress Test. Discussion: A key component of the Accelerated Stress Test is the Management Configuration Set to assess manageability of the router under stress. The Management Configuration Set defines the management configuration of the DUT. Features that are part of the Management Configuration Set include SNMP, Logging/Debug, and Statistics Collection, and services such as FTP, as shown in Figure 3. Measurement units: N/A Issues: None See Also: Control Plane Configuration Set Data Plane Configuration Set Security Plane Configuration Set Poretsky and Rao [Page 13] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 ____________ ____________ | | | Logging/ | | SNMP | __| Debug | | | | | | ------------ | ------------ | | | | \/ | ___________ | | | | | DUT |<---| | | ----------- | | \/ ___________ | Packet | | Statistics| | Collector | | | ----------- Figure 3. Management Plane Configuration Set 3.3.6 Security Plane 3.3.6.1 Security Plane Configuration Set Definition: Security features and scaling enabled for the Accelerated Stress Test. Discussion: The Security Plane Configuration Set includes the configuration and scaling of ACLs, Firewall, IPsec, and User Access, as shown in Figure 4. ____________ ____________ ____________ | | | Secure | | User | |ACL/Firewall| | Protocol | __| Access | | | | | | | | ------------ ------------ | ------------ | | | | | | | \/ | | ___________ | | | | | |------->| DUT |<--------| | | ----------- Figure 4. Security Configuration Module Poretsky and Rao [Page 14] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 Measurement units: N/A Issues: None See Also: ACL Configuration Set Secure Protocol Configuration Set Password Login Configuration Set 3.3.6.2 Security Plane Startup Conditions Definition: Security Plane conditions that occur at the start of the Accelerated Stress Benchmarking to establish conditions for the remainder of the test. Discussion: Startup Conditions may cause stress on the DUT and produce failure. Startup Conditions for the Security Plane include session establishment rate, number of sessions established and number of policies learned, and number of user access sessions opened. Measurement units: N/A Issues: None See Also: Startup Conditions Data Plane Startup Conditions Management Plane Startup Conditions Security Plane Startup Conditions 3.4 Instability 3.4.1 Instability Phase Definition: The portion of the benchmarking test in which the Instability Conditions are offered to the DUT. Discussion: The Instability Phase is the middle Phase of of the benchmarking test following the Startup Phase and preceding the Recovery Phase. Measurement Units: None Poretsky and Rao [Page 15] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 Issues: None See Also: Instability Conditions Startup Phase Recovery Phase 3.4.2 Benchmarks 3.4.2.1 Unstable Aggregate Forwarding Rate Definition: Rate of taffic forwarded by the DUT during the Instability Phase. Discussion: Unstable Aggregated Forwarding Rate is an instantaneous measurement of the Aggregate Forwarding Rate during the Instability Phase. It is recommended that the Unstable Aggregate Forwarding Rate is measured at one-second intervals. Measurement units: pps Issues: None See Also: Instability Conditions Aggregate Forwarding Rate 3.4.2.2 Degraded Aggregate Forwarding Rate Definition: The reduction in Aggregate Forwarding Rate during the Instability Phase. Discussion: The Degraded Aggregate Forwarding Rate is calculated for each measurement of the Unstable Aggregate Forwarding Rate. The Degraded Aggregate Forwarding Rate is calculated by subtracting each measurement of the Unstable Aggregate Forwarding Rate from the Stable Aggregate Forwarding Rate, such that Degraded Forwarding Rate = Stable Aggregate Forwarding Rate - Unstable Aggregate Forwarding Rate Ideally, the Degraded Aggregate Forwarding Rate is zero. Poretsky and Rao [Page 16] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 Measurement Units: pps Issues: None See Also: Instability Phase Unstable Aggregate Forwarding Rate 3.4.2.3 Average Degraded Aggregate Forwarding Rate Definition DUT Benchmark that is the calculated average of the obtained Degraded Forwarding Rates. Discussion: Measurement Units: pps Issues: None See Also: Degraded Aggregate Forwarding Rate 3.4.2.4 Unstable Uncontrolled Sessions Lost Definition: Control Plane sessions that are in the down state but were not intentionally brought down during the Instability Phase. Discussion: The test equipment is able to control protocol session state with the DUT. The test equipment is also to monitor for sessions lost with the DUT which the test equipment itself did not intentionally bring down. Measurement units: sessions Issues: None See Also: Controlled Session Loss Uncontrolled Session Loss Poretsky and Rao [Page 17] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 3.4.3 Instability Conditions 3.4.3.1 Control Plane Instability Conditions Definition: Control Plane conditions that occur during the Accelerated Stress Benchmark to produce instability and stress the DUT. Discussion: Control Plane Instability Conditions are experienced by the DUT after the Startup Conditions have completed. Control Plane Instability Conditions experienced by the DUT include session loss, route withdrawal, and route cost changes. Measurement units: N/A Issues: None See Also: Instability Conditions Data Plane Instability Conditions Management Plane Instability Conditions Security Plane Instability Conditions 3.4.3.2 Data Plane Instability Conditions Definition: Data Plane conditions that occur during the Accelerated Stress Benchmark to produce instability and stress the DUT. Discussion: Data Plane Instability Conditions are experienced by the DUT after the Startup Conditions have completed. Data Plane Instability Conditions experienced by the DUT include interface shutdown, link loss, and overloaded links. Measurement units: N/A Issues: None See Also: Instability Conditions Control Plane Instability Conditions Management Plane Instability Conditions Security Plane Instability Conditions Poretsky and Rao [Page 18] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 3.4.3.3 Management Plane Instability Conditions Definition: Management Plane conditions that occur during the Accelerated Life Benchmark to produce instability and stress the DUT. Discussion: Management Plane Instability Conditions are experienced by the DUT after the Startup Conditions have completed. Management Plane Instability Conditions experienced by the DUT include repeated FTP of large files. Measurement units: N/A Issues: None See Also: Instability Conditions Control Plane Instability Conditions Data Plane Instability Conditions Security Plane Instability Conditions 3.4.3.4 Security Plane Instability Conditions Definition: Security Plane conditions that occur during the Accelerated Life Benchmark to produce instability and stress the DUT. Discussion: Security Plane Instability Conditions are experienced by the DUT after the Startup Conditions have completed. Security Plane Instability Conditions experienced by the DUT include session loss and uninitiated policy changes. Measurement units: N/A Issues: None See Also: Instability Conditions Control Plane Instability Conditions Data Plane Instability Conditions Management Plane Instability Conditions Poretsky and Rao [Page 19] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 3.5 Recovery 3.5.1 Recovery Phase Definition: The portion of the benchmarking test in which the Startup Conditions are generated with the DUT, but the Instability Conditions are no longer offered to the DUT. Discussion: The Recovery Phase is the final Phase of the benchmarking test following the Startup Phase and Instability Phase. Startup Conditions must not be Restarted. Measurement Units: None Issues: None See Also: Startup Conditions Startup Phase Instability Conditions Instability Phase 3.5.2 Benchmarks 3.5.2.1 Recovered Aggregate Forwarding Rate Definition Rate of traffic forwarded by the DUT during the Recovery Phase. Discussion: Recovered Aggregate Forwarding Rate is an instantaneous measurement of the Aggregate Forwarding Rate during the Recovery Phase. It is recommended that the Recovered Aggregate Forwarding Rate is measured at one-second intervals. Ideally, each measurement of the Recovered Aggregate Forwarding Rate equals the Stable Aggregate Forwarding Rate because the Instability Conditions do not exist in both the Startup and Recovery Phases. Measurement Units: pps Issues: None Poretsky and Rao [Page 20] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 See Also: Aggregate Forwarding Rate Recovery Phase Recovered Aggregate Forwarding Rate Startup Phase Stable Aggregate Forwarding Rate 3.5.2.2 Recovery Time Definition The amount of time for the Recovered Aggregate Forwarding Rate to become equal to the Stable Aggregate Forwarding Rate. Discussion Recovery Time is measured beginning at the instant the Instability Phase ends until the Recovered Aggregate Forwarding Rate equals the Stable Aggregate Forwarding Rate for a minimum of 180 consecutive seconds. Measurement Units: seconds Issues: None See Also: Recovered Aggregate Forwarding Rate Stable Aggregate Forwarding Rate 3.5.2.3 Recovered Uncontrolled Control Plane Sessions Lost Definition: Control Plane sessions that are in the down state but were not intentionally brought down during the Recovery Phase. Discussion: The test equipment is able to control protocol session state with the DUT. The test equipment is also to monitor for sessions lost with the DUT which the test equipment itself did not intentionally bring down. Measurement units: sessions Issues: None See Also: Controlled Session Loss Uncontrolled Session Loss Poretsky and Rao [Page 21] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 4. Security Considerations Documents of this type do not directly effect the security of the Internet or of corporate networks as long as benchmarking is not performed on devices or systems connected to operating networks. 5. References [1] Bradner, S., Editor, "Benchmarking Terminology for Network Interconnection Devices", RFC 1242, July 1991. [2] Mandeville, R., "Benchmarking Terminology for LAN Switching Devices", RFC 2285, June 1998. [3] Bradner, S. and McQuaid, J., "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, March 1999. [4] "Core Router Evaluation for Higher Availability", Scott Poretsky, NANOG 25, June 8, 2002, Toronto, CA. [5] "Router Stress Testing to Validate Readiness for Network Deployment", Scott Poretsky, IEEE CQR 2003. [6] Poretsky, S. and Rao, S., "Framework for Accelerated Stress Benchmarking", draft-ietf-bmwg-acc-bench-frame-01, work in progress, February 2004. [7] Poretsky, S. and Rao, S., "Methodology for Accelerated Stress Benchmarking", draft-ietf-bmwg-acc-bench-meth-00, work in progress, February 2004. 6. Author's Address Scott Poretsky Quarry Technologies 8 New England Executive Park Burlington, MA 01803 USA Phone: + 1 781 395 5090 EMail: sporetsky@quarrytech.com Shankar Rao Qwest Communications Denver, CO USA Phone: + 1 303 437 6643 Email: shankar.rao@qwest.com Poretsky and Rao [Page 22] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 7. Full Copyright Statement Copyright (C) The Internet Society (1998). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Appendix 1. White Box Benchmarking Terminology Minimum Available Memory Definition: Minimum DUT Available Memory during the duration of the Accelerated Stress Test. Discussion: It is necessary to monitor DUT memory to measure this benchmark. Measurement units: bytes Issues: None See Also: Maximum CPU Utilization Poretsky and Rao [Page 23] INTERNET-DRAFT Terminology for Accelerated Stress Benchmarking February 2004 Maximum CPU Utilization Definition: Maximum DUT CPU utilization during the duration of the Accelerated Stress Test. Discussion: It is necessary to monitor DUT CPU Utilization to measure this benchmark. Measurement units: % Issues: None See Also: Minimum Available Memory Poretsky and Rao [Page 24]