Network Working Group INTERNET-DRAFT Expires in: August 2003 Scott Poretsky Avici Systems February 2003 Benchmarking Methodology for IGP Route Convergence Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Table of Contents 1. Introduction ...............................................2 2. Existing definitions .......................................2 3. Test Setup..................................................2 3.1 Test Topologies............................................2 3.2 Test Considerations........................................3 4. Test Cases..................................................4 4.1 Local Events...............................................4 4.1.1 Convergence Due to SONET Link Failure....................4 4.1.2 Convergence Due to PPP Session Failure...................5 4.1.3 Convergence Due to IGP Adjacency Failure.................5 4.1.4 Convergence Due to Route Withdrawal......................6 4.1.5 Convergence Due to Cost Change...........................7 4.2 Remote Events..............................................7 4.2.1 Convergence Due to Remote SONET Link Failure.............7 5. Measuring Convergence Times.................................8 5.1 Measuring Peak-to-Peak Convergence Time....................8 5.2 Measuring Impact of Components for Convergence.............8 6. Security Considerations.....................................9 Poretsky [Page 1] INTERNET-DRAFT Benchmarking Methodology for February 2003 IGP Route Convergence 7. Acknowledgements............................................9 8. References..................................................9 9. Author's Address............................................10 10. Full Copyright Statement...................................11 1. Introduction This draft describes the methodology for benchmarking IGP Route Convergence. The applicability of this testing is described in [1] and the new terminology that it introduces is defined in [2]. Service Providers use IGP Convergence time as a key metric of router design and architecture. Customers of Service Providers observe convergence time by packet loss. IGP Route Convergence is a Direct Measure of Quality (DMOQ) when benchmarking the data plane and not the control plane. The test cases in this document are black-box tests that emulate the network events that cause route convergence, as described in [1]. Black-box test design accounts for all of the factors for route convergence time, as provided in [1]. The methodology and terminology is to be used for benchmarking route convergence and can be applied to any link-state IGP such as ISIS [3] and OSPF [4]. 2. Existing definitions For the sake of clarity and continuity this RFC adopts the template for definitions set out in Section 2 of RFC 1242. Definitions are indexed and grouped together in sections for ease of reference. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 3. Test Setup 3.1 Test Topologies --------- Ingress Traffic Path --------- | |<------------------------------| | | | | | | | Preferred Egress Path | | | DUT |------------------------------>|Tester | | | | | | |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>| | | | Backup Egress Path | | --------- --------- Figure 1. IGP Route Convergence Test Topology for Local Changes Figure 1 shows the test topology to measure IGP Route Convergence due to local changes such as SONET Link Failure, PPP Session Failure, IGP Adjacency Failure, Route Withdrawal, and Route cost change. These test cases are described in Poretsky [Page 2] INTERNET-DRAFT Benchmarking Methodology for February 2003 IGP Route Convergence section 4.1. These test cases provide IGP Route Convergence times that consider the Event Detection time, SPF Processing time, and FIB Update time. These times are measured by observing packet loss in the data plane. Physical Links may be of any type, such as Sonet or Ethernet, and any speed. ----- ----------- | | Preferred | | ----- |R2 |------------->| | | |---->| | Egress Path | | | | ----- | | |R1 | | Tester | | | ----- | | | |---->| | Backup | | ----- |R3 |~~~~~~~~~~~~~>| | ^ | | Egress Path | | | ----- ----------- | | |-------------------------------- Ingress Traffic Path Figure 1. IGP Route Convergence Test Topology for Remote Changes Figure 2 shows the test topology to measure IGP Route Convergence time due to remote changes in the network topology. These times are measured by observing packet loss in the data plane. Physical Links may be of any type, such as Sonet or Ethernet, and any speed. In this topology, the three routers are considered a System Under Test (SUT). Application of this topology and test cases described in section 4.2 account for the impact of IGP Advertisement on Route Convergence, as described in [1]. 3.2 Test Considerations 3.2.1 IGP Selection The test cases described in section 4 can be used for ISIS or OSPF. The Route Convergence test methodology for both is identical. The IGP adjacencies are established on the Preferred Egress Path and Backup Egress Path. 3.2.2 BGP Configuration The obtained results for IGP Route Convergence may vary if BGP routes are installed. For results similar to those that would be observed in an operational network it is recommended that a BGP session be established on the Ingress Traffic Path with routes installed. Poretsky [Page 3] INTERNET-DRAFT Benchmarking Methodology for February 2003 IGP Route Convergence 3.2.3 IGP Route Scaling The number of IGP routes will impact the measured IGP Route Convergence because convergence for the entire IGP route table is measured. For results similar to those that would be observed in an operational network it is recommended that the number of installed routes closely approximate that for routers in the network. 3.2.4 BGP Route Scaling The number of installed BGP routes may impact the IGP Convergence time. For results similar to those that would be observed in an operational network it is recommended that the number of installed routes closely approximate that for routers in the network. 3.2.5 Timers There are some timers that will impact the measured IGP Convergence time. The following timers should be configured to the minimum value prior to beginning execution of the test cases: SONET Failure Indication Delay IGP Hello Timer IGP Dead-Interval LSA Generation Delay LSA Flood Packet Pacing LSA Retransmission Packet Pacing SPF Delay 4. Test Cases 4.1 Local Events The test cases in this section use the test topology shown in Figure 1. 4.1.1 Convergence Due to Local SONET Link Failure Objective To obtain the IGP Route Convergence due to a Local SONET Link failure event. Procedure 1. Advertise BGP routes from Tester to DUT on Ingress Traffic Path. 2. Advertise matching IGP routes from Tester to DUT on Preferred Egress Path and Backup Egress Path. Set the cost of the routes so that the IGP routes along the Preferred Egress Path is the preferred next-hop. 3. Send traffic at maximum forwarding rate to destinations matching all IGP routes from Tester to DUT on Ingress Traffic Path. Poretsky [Page 4] INTERNET-DRAFT Benchmarking Methodology for February 2003 IGP Route Convergence 4. Verify traffic routed over Preferred Egress Path. 5. Remove SONET on Tester Interface connected to Preferred Egress Path. 6. Measure Peak-to-Peak Convergence Time [2] as DUT detects the link down event and converges all IGP routes and traffic over the Backup Egress Path. Results The measured IGP Convergence time is influenced by the Local SONET indication, SPF delay, SPF Holdtime, SPF Execution Time, Tree Build Time, and Hardware Update Time. 4.1.2 Convergence Due to PPP Session Failure Objective To obtain the IGP Route Convergence due to a Local PPP Session failure event. Procedure 1. Advertise BGP routes from Tester to DUT on Ingress Traffic Path. 2. Advertise matching IGP routes from Tester to DUT on Preferred Egress Path and Backup Egress Path. Set the cost of the routes so that the IGP routes along the Preferred Egress Path is the preferred next-hop. 3. Send traffic at maximum forwarding rate to destinations matching all IGP routes from Tester to DUT on Ingress Traffic Path. 4. Verify traffic routed over Preferred Egress Path. 5. Remove PPP session from Tester Interface connected to Preferred Egress Path. 6. Measure Peak-to-Peak Convergence Time as DUT detects the PPP session down event and converges all IGP routes and traffic over the Backup Egress Path. Results The measured IGP Convergence time is influenced by the Local PPP failure indication, SPF delay, SPF Holdtime, SPF Execution Time, Tree Build Time, and Hardware Update Time. 4.1.3 Convergence Due to IGP Adjacency Failure Objective To obtain the IGP Route Convergence due to a Local IGP Adjacency failure event. Procedure 1. Advertise BGP routes from Tester to DUT on Ingress Traffic Path. 2. Advertise matching IGP routes from Tester to DUT Poretsky [Page 5] INTERNET-DRAFT Benchmarking Methodology for February 2003 IGP Route Convergence on Preferred Egress Path and Backup Egress Path. Set the cost of the routes so that the IGP routes along the Preferred Egress Path is the preferred next-hop. 3. Send traffic at maximum forwarding rate to destinations matching all IGP routes from Tester to DUT on Ingress Traffic Path. 4. Verify traffic routed over Preferred Egress Path. 5. Remove IGP adjacency from Tester interface connected to Preferred Egress Path. 6. Measure Peak-to-Peak Convergence Time as DUT detects the IGP session failure event and converges all IGP routes and traffic over the Backup Egress Path. Results The measured IGP Convergence time is influenced by the IGP Hello Interval, IGP Dead Interval, SPF delay, SPF Holdtime, SPF Execution Time, Tree Build Time, and Hardware Update Time. 4.1.4 Convergence Due to Route Withdrawal Objective To obtain the IGP Route Convergence due to Route Withdrawal. Procedure 1. Advertise BGP routes from Tester to DUT on Ingress Traffic Path. 2. Advertise matching IGP routes from Tester to DUT on Preferred Egress Path and Backup Egress Path. Set the cost of the routes so that the IGP routes along the Preferred Egress Path is the preferred next-hop. 3. Send traffic at maximum forwarding rate to destinations matching all IGP routes from Tester to DUT on Ingress Traffic Path. 4. Verify traffic routed over Preferred Egress Path. 5. Tester withdraws all IGP routes from DUT's Local Interface on Preferred Egress Path. 6. Measure Peak-to-Peak Convergence Time as DUT processes the route withdrawal event and converges all IGP routes and traffic over the Backup Egress Path. Results The measured IGP Convergence time is the SPF Processing and FIB Update time as influenced by the SPF delay, SPF Holdtime, SPF Execution Time, Tree Build Time, and Hardware Update Time. Poretsky [Page 6] INTERNET-DRAFT Benchmarking Methodology for February 2003 IGP Route Convergence 4.1.5 Convergence Due to Cost Change Objective To obtain the IGP Route Convergence due to route cost change. Procedure 1. Advertise BGP routes from Tester to DUT on Ingress Traffic Path. 2. Advertise matching IGP routes from Tester to DUT on Preferred Egress Path and Backup Egress Path. Set the cost of the routes so that the Preferred Egress Path is the preferred path. 3. Send traffic at maximum forwarding rate to destinations matching all IGP routes from Tester to DUT on Ingress Traffic Path. 4. Verify traffic routed over Preferred Egress Path. 5. Tester increases cost for all IGP routes at DUT's Local Interface on Preferred Egress Path so that Backup Egress Path have lower cost and becomes preferred path. 6. Measure Reroute Convergence Time [2] as DUT detects the cost change event and converges all IGP routes and traffic over the Backup Egress Path. Results The measured IGP Convergence time is the SPF Processing and FIB Update time as influenced by the SPF delay, SPF Holdtime, SPF Execution Time, Tree Build Time, and Hardware Update Time. There should be no packet loss for this case. 4.2 Remote Events The test cases in this section use the test topology shown in Figure 2. 4.2.1 Convergence Due to Remote SONET Link Failure Objective To obtain the IGP Route Convergence due to a Remote SONET Link failure event. Procedure 1. Advertise BGP routes from Tester to DUT on Ingress Traffic Path. 2. Advertise matching IGP routes from Tester to DUT on Preferred Egress Path and Backup Egress Path. Set the cost of the routes so that the IGP routes along the Preferred Egress Path is the preferred next-hop. 3. Send traffic at maximum forwarding rate to destinations matching all IGP routes from Tester to DUT on Ingress Traffic Path. Poretsky [Page 7] INTERNET-DRAFT Benchmarking Methodology for February 2003 IGP Route Convergence 4. Verify traffic routed over Preferred Egress Path. 5. Remove SONET on Neighbor Interface connected to Preferred Egress Path. 6. Measure Peak-to-Peak Convergence time as DUT detects the link down event and converges all IGP routes and traffic over the Backup Egress Path. Results The measured IGP Convergence time is influenced by the SONET failure indication, LSA/LSP Flood Packet Pacing, LSA/LSP Retransmission Packet Pacing, LSA/LSP Generation time, SPF delay, SPF Holdtime, SPF Execution Time, Tree Build Time, and Hardware Update Time. 5. Measuring Convergence Times 5.1 Measuring Peak-to-Peak Convergence Time Figure 3 shows a graph model of Convergence Time as measured from the data plane. Refer to [2] for defintions of the terms used. IGP Route Convergence Time is the amount of time for the Forwarding Rate to begin its downward slope upon occurrence of a network event and then fully recover to the Maximum Forwarding Rate. This is calculated as (eq 1) Time(Convergence) = Time(Recovery) - Time(Network Event). Title: Forwarding Rate versus Time Time=Recovery Time=Network Event Time = 0sec Maximum ^ ^ ^ Forwarding Rate--> ----\ /----------- \ /<----Route Convergence Route Convergence------->\ / Event Slope Recovery Slope \_______/<------100% Packet Loss X-axis = Time Y-axis = Forwarding Rate Figure 3. Convergence Graph 5.2 Measuring Impact of Components for Convergence The factors for IGP Route Convergence Time are provided in [1]. The results of the test cases in section 4 above can be used to calculate the impact each factor has on the Convergence results, as follow: Poretsky [Page 8] INTERNET-DRAFT Benchmarking Methodology for February 2003 IGP Route Convergence SPF Processing and FIB Update time = Result (4.1.4) SONET failure indication time = Result(4.1.1) - Result(4.1.4) PPP failure indication time = Result(4.1.2) - Result(4.1.4) IGP failure indication time = Result(4.1.3) - Result(4.1.4) IGP Advertisement time = Result(4.1.1) - Result(4.2.1) 6. Security Considerations Documents of this type do not directly effect the security of the Internet or of corporate networks as long as benchmarking is not performed on devices or systems connected to operating networks. 7. Acknowledgements Thanks to Jayant Kulkarni for doing as most Test Engineers do - working beyond the call of duty to help advance technology. Especially thanks to the many Network Engineers and Network Architects at the Service Providers who are always eager to discuss Route Convergence. 8. References [1] Poretsky, S., "Benchmarking Applicability for IGP Convergence", draft-ietf-poretsky-igp-convergence-app-00, work in progress, February 2003. [2] Poretsky, S., "Benchmarking Terminology for IGP Convergence", draft-ietf-poretsky-igp-convergence-term-00, work in progress, February 2003. [3] Callon, R., "Use of OSI IS-IS for Routing in TCP/IP and Dual Environments", RFC 1195, December 1990. [4] Moy, J., "OSPF Version 2", RFC 2328, IETF, April 1998. 9. Author's Address Scott Poretsky Avici Systems, Inc. 101 Billerica Avenue N. Billerica, MA 01862 USA Phone: + 1 978 964 2287 EMail: sporetsky@avici.com Poretsky [Page 9] INTERNET-DRAFT Benchmarking Methodology for February 2003 IGP Route Convergence 10. Full Copyright Statement Copyright (C) The Internet Society (1998). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Poretsky [Page 10]