Network Working Group S. Poretsky Internet-Draft Allot Communications Intended status: Informational B. Imhoff Expires: January 14, 2010 Juniper Networks K. Michielsen Cisco Systems July 13, 2009 Terminology for Benchmarking Link-State IGP Data Plane Route Convergence draft-ietf-bmwg-igp-dataplane-conv-term-18 Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on January 14, 2010. Copyright Notice Copyright (c) 2009 IETF Trust and the persons identified as the Poretsky, et al. Expires January 14, 2010 [Page 1] Internet-Draft IGP Convergence Benchmark Terminology July 2009 document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Abstract This document describes the terminology for benchmarking Interior Gateway Protocol (IGP) Route Convergence. The terminology is to be used for benchmarking IGP convergence time through externally observable (black box) data plane measurements. The terminology can be applied to any link-state IGP, such as ISIS and OSPF. Table of Contents 1. Introduction and Scope . . . . . . . . . . . . . . . . . . . . 4 2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 4 3. Term Definitions . . . . . . . . . . . . . . . . . . . . . . . 4 3.1. Convergence Types . . . . . . . . . . . . . . . . . . . . 5 3.1.1. Route Convergence . . . . . . . . . . . . . . . . . . 5 3.1.2. Full Convergence . . . . . . . . . . . . . . . . . . . 5 3.1.3. Network Convergence . . . . . . . . . . . . . . . . . 6 3.2. Instants . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.2.1. Convergence Event Instant . . . . . . . . . . . . . . 6 3.2.2. Convergence Recovery Instant . . . . . . . . . . . . . 7 3.2.3. First Route Convergence Instant . . . . . . . . . . . 7 3.3. Transitions . . . . . . . . . . . . . . . . . . . . . . . 8 3.3.1. Convergence Event Transition . . . . . . . . . . . . . 8 3.3.2. Convergence Recovery Transition . . . . . . . . . . . 9 3.4. Interfaces . . . . . . . . . . . . . . . . . . . . . . . . 9 3.4.1. Local Interface . . . . . . . . . . . . . . . . . . . 9 3.4.2. Remote Interface . . . . . . . . . . . . . . . . . . . 10 3.4.3. Preferred Egress Interface . . . . . . . . . . . . . . 10 3.4.4. Next-Best Egress Interface . . . . . . . . . . . . . . 10 3.5. Benchmarking Methods . . . . . . . . . . . . . . . . . . . 11 3.5.1. Rate-Derived Method . . . . . . . . . . . . . . . . . 11 3.5.2. Loss-Derived Method . . . . . . . . . . . . . . . . . 12 3.5.3. Route-Specific Loss-Derived Method . . . . . . . . . . 13 3.6. Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . 15 3.6.1. Full Convergence Time . . . . . . . . . . . . . . . . 15 3.6.2. First Route Convergence Time . . . . . . . . . . . . . 15 3.6.3. Route-Specific Convergence Time . . . . . . . . . . . 16 3.6.4. Loss-Derived Convergence Time . . . . . . . . . . . . 18 3.6.5. Route Loss of Connectivity Period . . . . . . . . . . 19 Poretsky, et al. Expires January 14, 2010 [Page 2] Internet-Draft IGP Convergence Benchmark Terminology July 2009 3.6.6. Loss-Derived Loss of Connectivity Period . . . . . . . 20 3.7. Measurement Terms . . . . . . . . . . . . . . . . . . . . 21 3.7.1. Convergence Event . . . . . . . . . . . . . . . . . . 21 3.7.2. Packet Loss . . . . . . . . . . . . . . . . . . . . . 21 3.7.3. Convergence Packet Loss . . . . . . . . . . . . . . . 21 3.7.4. Connectivity Packet Loss . . . . . . . . . . . . . . . 22 3.7.5. Packet Sampling Interval . . . . . . . . . . . . . . . 23 3.7.6. Sustained Convergence Validation Time . . . . . . . . 23 3.8. Miscellaneous Terms . . . . . . . . . . . . . . . . . . . 24 3.8.1. Stale Forwarding . . . . . . . . . . . . . . . . . . . 24 3.8.2. Nested Convergence Event . . . . . . . . . . . . . . . 24 4. Security Considerations . . . . . . . . . . . . . . . . . . . 25 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 25 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 25 7.1. Normative References . . . . . . . . . . . . . . . . . . . 25 7.2. Informative References . . . . . . . . . . . . . . . . . . 26 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 26 Poretsky, et al. Expires January 14, 2010 [Page 3] Internet-Draft IGP Convergence Benchmark Terminology July 2009 1. Introduction and Scope This draft describes the terminology for benchmarking Link-State Interior Gateway Protocol (IGP) Convergence. The motivation and applicability for this benchmarking is provided in [Po09a]. The methodology to be used for this benchmarking is described in [Po09m]. The purpose of this document is to introduce new terms required to complete execution of the IGP Route Methodology [Po09m]. IGP convergence time is measured on the data plane at the Tester by observing packet loss through the DUT. The methodology and terminology to be used for benchmarking IGP Convergence can be applied to IPv4 and IPv6 traffic and link-state IGPs such as ISIS [Ca90][Ho08], OSPF [Mo98][Co08], and others. 2. Existing Definitions This document uses existing terminology defined in other BMWG work. Examples include, but are not limited to: Frame Loss Rate [Ref.[Br91], section 3.6] Throughput [Ref.[Br91], section 3.17] Offered Load [Ref.[Ma98], section 3.5.2] Forwarding Rate [Ref.[Ma98], section 3.6.1] Device Under Test (DUT) [Ref.[Ma98], section 3.1.1] System Under Test (SUT) [Ref.[Ma98], section 3.1.2] Out-of-order Packet [Ref.[Po06], section 3.3.2] Duplicate Packet [Ref.[Po06], section 3.3.3] Packet Reordering [Ref.[Mo06], section 3.3] Stream [Ref.[Po06], section 3.3.2] Flow [Ref.[Po06], section 3.1.5] Forwarding Delay [Ref.[Po06], section 3.2.4] Loss Period [Ref.[Ko02], section 4] The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14, RFC 2119 [Br97]. RFC 2119 defines the use of these key words to help make the intent of standards track documents as clear as possible. While this document uses these keywords, this document is not a standards track document. 3. Term Definitions Poretsky, et al. Expires January 14, 2010 [Page 4] Internet-Draft IGP Convergence Benchmark Terminology July 2009 3.1. Convergence Types 3.1.1. Route Convergence Definition: The process of updating all components of the router, including the Routing Information Base (RIB) and Forwarding Information Base (FIB), along with software and hardware tables, with the most recent route change(s) such that forwarding for a route entry is successful on the Next-Best Egress Interface. Discussion: Route Convergence MUST occur after a Convergence Event. Route Convergence can be observed externally by the rerouting of data traffic for a destination matching a route entry to the Next-best Egress Interface. Completion of Route Convergence may or may not be sustained over time. Measurement Units: N/A Issues: None See Also: Network Convergence, Full Convergence, Convergence Event 3.1.2. Full Convergence Definition: Route Convergence for all routes in the FIB. Discussion: Full Convergence MUST occur after a Convergence Event. Full Convergence can be observed externally by the rerouting of data traffic to destinations matching all route entries to the Next-best Egress Interface. Completion of Full Convergence is externally observable from the data plane when the Forwarding Rate of the data plane traffic on the Next-Best Egress Interface equals the Offered Load. Completion of Full Convergence may or may not be sustained over time. Measurement Units: N/A Poretsky, et al. Expires January 14, 2010 [Page 5] Internet-Draft IGP Convergence Benchmark Terminology July 2009 Issues: None See Also: Network Convergence, Route Convergence, Convergence Event, Full Convergence Time, Convergence Recovery Instant 3.1.3. Network Convergence Definition: Full Convergence in all routers throughout the network. Discussion: Network Convergence includes all Route Convergence operations for all routers in the network following a Convergence Event. Completion of Network Convergence can be observed by recovery of the network Forwarding Rate to equal the Offered Load, with no Stale Forwarding, and no Blenders [Ca01][Ci03]. Completion of Network Convergence may or may not be sustained over time. Measurement Units: N/A Issues: None See Also: Route Convergence, Full Convergence, Stale Forwarding 3.2. Instants 3.2.1. Convergence Event Instant Definition: The time instant that a Convergence Event occurs. Discussion: If the Convergence Event causes instantaneous traffic loss on the Preferred Egress Interface, the Convergence Event Instant is observable from the data plane as the instant that the DUT begins to exhibit packet loss. Poretsky, et al. Expires January 14, 2010 [Page 6] Internet-Draft IGP Convergence Benchmark Terminology July 2009 The Tester SHOULD collect a timestamp on the Convergence Event Instant if it is not observable from the data plane. Measurement Units: hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is microseconds. Issues: None See Also: Convergence Event 3.2.2. Convergence Recovery Instant Definition: The time instant that Full Convergence has completed. Discussion: The Full Convergence completed state MUST be maintained for an interval of duration equal to the Sustained Convergence Validation Time in order to validate the Convergence Recovery Instant. The Convergence Recovery Instant is observable from the data plane as the instant the DUT forwards traffic to all destinations over the Next-Best Egress Interface. When using the Rate-Derived Method, the Convergence Recovery Instant falls within the Packet Sampling Interval preceding the first interval where the observed Forwarding Rate on the Next-Best Egress Interface equals the Offered Load. Measurement Units: hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is microseconds. Issues: None See Also: Sustained Convergence Validation Time, Full Convergence 3.2.3. First Route Convergence Instant Definition: Poretsky, et al. Expires January 14, 2010 [Page 7] Internet-Draft IGP Convergence Benchmark Terminology July 2009 The time instant the first route entry completes Route Convergence following a Convergence Event Discussion: Any route may be the first to complete Route Convergence. The First Route Convergence Instant is observable from the data plane as the instant that the first packet is received from the Next-Best Egress Interface. Measurement Units: hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is microseconds. Issues: None See Also: Route Convergence 3.3. Transitions 3.3.1. Convergence Event Transition Definition: A time interval following a Convergence Event in which Forwarding Rate on the Preferred Egress Interface gradually reduces to zero. Discussion: The Forwarding Rate during a Convergence Event Transition may not decrease linearly. The Forwarding Rate observed on all DUT egress interfaces may or may not decrease to zero. The Offered Load, the number of routes, and the Packet Sampling Interval influence the observations of the Convergence Event Transition using the Rate-Derived Method. This is further discussed with the term "Rate-Derived Method". Measurement Units: seconds Issues: None See Also: Convergence Event, Rate-Derived Method Poretsky, et al. Expires January 14, 2010 [Page 8] Internet-Draft IGP Convergence Benchmark Terminology July 2009 3.3.2. Convergence Recovery Transition Definition: A time interval following the First Route Convergence Instant in which Forwarding Rate on the Next-Best Egress Interface gradually increases to equal the Offered Load. Discussion: The Forwarding Rate observed during a Convergence Recovery Transition may not increase linearly. The Offered Load, the number of routes, and the Packet Sampling Interval influence the observations of the Convergence Recovery Transition using the Rate-Derived Method. This is further discussed with the term "Rate-Derived Method". Measurement Units: seconds Issues: None See Also: Full Convergence,First Route Convergence Instant, Rate-Derived Method 3.4. Interfaces 3.4.1. Local Interface Definition: An interface on the DUT. Discussion: A failure of the Local Interface indicates that the failure occurred directly on the DUT. Measurement Units: N/A Issues: None See Also: Remote Interface Poretsky, et al. Expires January 14, 2010 [Page 9] Internet-Draft IGP Convergence Benchmark Terminology July 2009 3.4.2. Remote Interface Definition: An interface on a neighboring router that is not directly connected to any interface on the DUT. Discussion: A failure of a Remote Interface indicates that the failure occurred on a neighbor router's interface that is not directly connected to the DUT. Measurement Units: N/A Issues: None See Also: Local Interface 3.4.3. Preferred Egress Interface Definition: The outbound interface from the DUT for traffic routed to the preferred next-hop. Discussion: The Preferred Egress Interface is the egress interface prior to a Convergence Event. Measurement Units: N/A Issues: None See Also: Next-Best Egress Interface 3.4.4. Next-Best Egress Interface Definition: The outbound interface from the DUT for traffic routed to the second- best next-hop. Discussion: The Next-Best Egress Interface becomes the egress interface after a Convergence Event. Poretsky, et al. Expires January 14, 2010 [Page 10] Internet-Draft IGP Convergence Benchmark Terminology July 2009 The Next-Best Egress Interface is of the same media type and link speed as the Preferred Egress Interface. Measurement Units: N/A Issues: None See Also: Preferred Egress Interface 3.5. Benchmarking Methods 3.5.1. Rate-Derived Method Definition: The method to calculate convergence time benchmarks from observing Forwarding Rate each Packet Sampling Interval. Discussion: Figure 1 shows an example of the Forwarding Rate change in time during convergence as observed when using the Rate-Derived Method. ^ Convergence Fwd | Recovery Rate | Instant | Offered ^ | Load --> ----------\ /----------- | \ /<--- Convergence | \ Packet / Recovery | Convergence --->\ Loss / Transition | Event \ / | Transition \---------/ <-- Max Packet Loss | +---------------------------------------------------------> ^ ^ time Convergence First Route Event Instant Convergence Instant Figure 1: Rate-Derived Convergence Graph The Offered Load SHOULD consists of a single Stream [Po06]. If sending multiple Streams, the measured traffic rate statistics for all Streams MUST be added together. The destination addresses for the Offered Load MUST be distributed such that all routes in the FIB are matched and each route is offered an equal share of the total Offered Load. Poretsky, et al. Expires January 14, 2010 [Page 11] Internet-Draft IGP Convergence Benchmark Terminology July 2009 At least one packet per route in the FIB for all routes in the FIB MUST be offered to the DUT within each Packet Sampling Interval. The Offered Load, the number of routes, and the Packet Sampling Interval influence the observations for the Rate-Derived Method. It may be difficult to identify the different convergence time instants in the Rate-Derived Convergence Graph. For example, it is possible that a Convergence Event causes the Forwarding Rate to drop to zero, while this may not be observed in the Forwarding Rate measurements if the Packet Sampling Interval is too high. Metrics measured at the Packet Sampling Interval MUST include Forwarding Rate and packet loss. Rate-Derived Method is a RECOMMENDED method to measure convergence time benchmarks. To measure convergence time benchmarks for Convergence Events that do not cause instantaneous traffic loss for all routes at the Convergence Event Instant, the Tester SHOULD collect a timestamp of the Convergence Event Instant and the Tester SHOULD observe Forwarding Rate seperately on the Next-Best Egress Interface. Since the Rate-Derived Method does not distinguish between individual traffic destinations, it SHOULD NOT be used for any route specific measurements. Therefor Rate-Derived Method SHOULD NOT be used to benchmark Route Loss of Connectivity Period. Measurement Units: N/A Issues: None See Also: Packet Sampling Interval, Convergence Event, Convergence Event Instant, Full Convergence 3.5.2. Loss-Derived Method Definition: The method to calculate the Loss-Derived Convergence Time and Loss- Derived Loss of Connectivity Period benchmarks from the amount of packet loss. Discussion: The Offered Load SHOULD consists of a single Stream [Po06]. If Poretsky, et al. Expires January 14, 2010 [Page 12] Internet-Draft IGP Convergence Benchmark Terminology July 2009 sending multiple Streams, the measured traffic rate statistics for all Streams MUST be added together. The destination addresses for the Offered Load MUST be distributed such that all routes in the FIB are matched and each route is offered an equal share of the total Offered Load. Loss-Derived Method SHOULD always be combined with Rate-Derived Method in order to observe Full Convergence completion. The total amount of Convergence Packet Loss is collected after Full Convergence completion. To measure convergence time and loss of connectivity benchmarks, the Tester SHOULD in general observe packet loss on all DUT egress interfaces (Connectivity Packet Loss). To measure convergence time benchmarks for Convergence Events that do not cause instantaneous traffic loss for all routes at the Convergence Event Instant, the Tester SHOULD collect a timestamp of the Convergence Event Instant and the Tester SHOULD observe packet loss seperately on the Next-Best Egress Interface (Convergence Packet Loss). Since Loss-Derived Method does not distinguish between traffic destinations and the packet loss statistics are only collected after Full Convergence completion, this method can only be used to measure average values over all routes. For these reasons Loss-Derived Method can only be used to benchmark Loss-Derived Convergence Time and Loss-Derived Loss of Connectivity Period. Note that the Loss-Derived Method measures an average over all routes, including the routes that may not be impacted by the Convergence Event, such as routes via non-impacted members of ECMP or parallel links. Measurement Units: seconds Issues: None See Also: Loss-Derived Convergence Time, Loss-Derived Loss of Connectivity Period, Convergence Packet Loss 3.5.3. Route-Specific Loss-Derived Method Definition: Poretsky, et al. Expires January 14, 2010 [Page 13] Internet-Draft IGP Convergence Benchmark Terminology July 2009 The method to calculate the Route-Specific Convergence Time benchmark from the amount of packet loss during convergence for a specific route entry. Discussion: To benchmark Route-Specific Convergence Time, the Tester provides an Offered Load that consists of multiple Streams [Po06]. Each Stream has a single destination address matching a different route entry, for every route entry in the FIB. Convergence Packet Loss is measured for each Stream separately. Route-Specific Loss-Derived Method SHOULD always be combined with Rate-Derived Method in order to observe Full Convergence completion. The total amount of Convergence Packet Loss for each Stream is collected after Full Convergence completion. Route-Specific Loss-Derived Method is a RECOMMENDED method to measure convergence time benchmarks. To measure convergence time and loss of connectivity benchmarks, the Tester SHOULD in general observe packet loss on all DUT egress interfaces (Connectivity Packet Loss). To measure convergence time benchmarks for Convergence Events that do not cause instantaneous traffic loss for all routes at the Convergence Event Instant, the Tester SHOULD collect a timestamp of the Convergence Event Instant and the Tester SHOULD observe packet loss seperately on the Next-Best Egress Interface (Convergence Packet Loss). Since Route-Specific Loss-Derived Method uses traffic streams to individual routes, it measures packet loss as it would be experienced by a network user. For this reason Route-Specific Loss-Derived Method is RECOMMENDED to measure Route-Specific Convergence Time benchmarks and Route Loss of Connectivity Period benchmarks. Measurement Units: seconds Issues: None See Also: Route-Specific Convergence Time, Route Loss of Connectivity Period, Convergence Packet Loss Poretsky, et al. Expires January 14, 2010 [Page 14] Internet-Draft IGP Convergence Benchmark Terminology July 2009 3.6. Benchmarks 3.6.1. Full Convergence Time Definition: The time duration of the period between the Convergence Event Instant and the Convergence Recovery Instant as observed using the Rate- Derived Method. Discussion: Using the Rate-Derived Method, Full Convergence Time can be calculated as the time difference between the Convergence Event Instant and the Convergence Recovery Instant, as shown in Equation 1. Full Convergence Time = Convergence Recovery Instant - Convergence Event Instant Equation 1 The Convergence Event Instant can be derived from the Forwarding Rate observation or from a timestamp collected by the Tester. For the testcases described in [Po09m], it is expected that Full Convergence Time equals the maximum Route-Specific Convergence Time when benchmarking all routes in FIB using the Route-Specific Loss- Derived Method. It is not possible to measure Full Convergence Time using the Loss- Derived Method. Measurement Units: seconds Issues: None See Also: Full Convergence, Rate-Derived Method, Route-Specific Loss-Derived Method 3.6.2. First Route Convergence Time Definition: The duration of the period between the Convergence Event Instant and the First Route Convergence Instant as observed using the Rate- Derived Method. Poretsky, et al. Expires January 14, 2010 [Page 15] Internet-Draft IGP Convergence Benchmark Terminology July 2009 Discussion: Using the Rate-Derived Method, First Route Convergence Time can be calculated as the time difference between the Convergence Event Instant and the First Route Convergence Instant, as shown with Equation 2. First Route Convergence Time = First Route Convergence Instant - Convergence Event Instant Equation 2 The Convergence Event Instant can be derived from the Forwarding Rate observation or from a timestamp collected by the Tester. For the testcases described in [Po09m], it is expected that First Route Convergence Time equals the minimum Route-Specific Convergence Time when benchmarking all routes in FIB using the Route-Specific Loss-Derived Method. It is not possible to measure First Route Convergence Time using the Loss-Derived Method. Measurement Units: seconds Issues: None See Also: Rate-Derived Method, Route-Specific Loss-Derived Method, First Route Convergence Instant 3.6.3. Route-Specific Convergence Time Definition: The amount of time it takes for Route Convergence to be completed for a specific route, as calculated from the amount of packet loss during convergence for a single route entry. Discussion: Route-Specific Convergence Time can only be measured using the Route- Specific Loss-Derived Method. If the applied Convergence Event causes instantaneous traffic loss for all routes at the Convergence Event Instant, Connectivity Packet Loss should be observed. Connectivity Packet Loss is the combined Poretsky, et al. Expires January 14, 2010 [Page 16] Internet-Draft IGP Convergence Benchmark Terminology July 2009 packet loss observed on Preferred Egress Interface and Next-Best Egress Interface. When benchmarking Route-Specific Convergence Time, Connectivity Packet Loss is measured and Equation 3 is applied for each measured route. The calculation is equal to Equation 7 in Section 3.6.5. Route-Specific Convergence Time = Connectivity Packet Loss for specific route/Offered Load per route Equation 3 If the applied Convergence Event does not cause instantaneous traffic loss for all routes at the Convergence Event Instant, then the Tester SHOULD collect a timestamp of the Convergence Event Instant and the Tester SHOULD observe Convergence Packet Loss separately on the Next- Best Egress Interface. When benchmarking Route-Specific Convergence Time, Convergence Packet Loss is measured and Equation 4 is applied for each measured route. Route-Specific Convergence Time = Convergence Packet Loss for specific route/Offered Load per route - (Convergence Event Instant - start traffic instant) Equation 4 The Convergence Event Instant and start traffic instant SHOULD be collected by the Tester. The Route-Specific Convergence Time benchmarks enable minimum, maximum, average, and median convergence time measurements to be reported by comparing the results for the different route entries. It also enables benchmarking of convergence time when configuring a priority value for route entry(ies). Since multiple Route-Specific Convergence Times can be measured it is possible to have an array of results. The format for reporting Route-Specific Convergence Time is provided in [Po09m]. Measurement Units: seconds Issues: None See Also: Convergence Event, Convergence Packet Loss, Connectivity Packet Loss, Route Convergence Poretsky, et al. Expires January 14, 2010 [Page 17] Internet-Draft IGP Convergence Benchmark Terminology July 2009 3.6.4. Loss-Derived Convergence Time Definition: The average Route Convergence time for all routes in FIB, as calculated from the amount of packet loss during convergence. Discussion: Loss-Derived Convergence Time is measured using the Loss-Derived Method. If the applied Convergence Event causes instantaneous traffic loss for all routes at the Convergence Event Instant, Connectivity Packet Loss should be observed. Connectivity Packet Loss is the combined packet loss observed on Preferred Egress Interface and Next-Best Egress Interface. When benchmarking Loss-Derived Convergence Time, Connectivity Packet Loss is measured and Equation 5 is applied. Loss-Derived Convergence Time = Connectivity Packet Loss/Offered Load Equation 5 If the applied Convergence Event does not cause instantaneous traffic loss for all routes at the Convergence Event Instant, then the Tester SHOULD collect a timestamp of the Convergence Event Instant and the Tester SHOULD observe Convergence Packet Loss separately on the Next- Best Egress Interface. When benchmarking Loss-Derived Convergence Time, Convergence Packet Loss is measured and Equation 6 is applied. Loss-Derived Convergence Time = Convergence Packet Loss/Offered Load - (Convergence Event Instant - start traffic instant) Equation 6 The Convergence Event Instant and start traffic instant SHOULD be collected by the Tester. Measurement Units: seconds Issues: None See Also: Convergence Packet Loss, Connectivity Packet Loss, Route Convergence Poretsky, et al. Expires January 14, 2010 [Page 18] Internet-Draft IGP Convergence Benchmark Terminology July 2009 3.6.5. Route Loss of Connectivity Period Definition: The time duration of traffic loss for a specific route entry following a Convergence Event until Full Convergence completion, as observed using the Route-Specific Loss-Derived Method. Discussion: In general the Route Loss of Connectivity Period is not equal to the Route-Specific Convergence Time. If the DUT continues to forward traffic to the Preferred Egress Interface after the Convergence Event is applied then the Route Loss of Connectivity Period will be smaller than the Route-Specific Convergence Time. This is also specifically the case after reversing a failure event. The Route Loss of Connectivity Period may be equal to the Route- Specific Convergence Time, as a characteristic of the Convergence Event, traffic for all routes starts dropping instantaneously on the Convergence Event Instant. See discussion in [Po09m]. For the testcases described in [Po09m] the Route Loss of Connectivity Period is expected to be a single Loss Period [Ko02]. When benchmarking Route Loss of Connectivity Period, Connectivity Packet Loss is measured for each route and Equation 7 is applied for each measured route entry. The calculation is equal to Equation 3 in Section 3.6.3. Route Loss of Connectivity Period = Connectivity Packet Loss for specific route/Offered Load per route Equation 7 Route Loss of Connectivity Period SHOULD be measured using Route- Specific Loss-Derived Method. Measurement Units: seconds Issues: None See Also: Route-Specific Convergence Time, Route-Specific Loss-Derived Method, Connectivity Packet Loss Poretsky, et al. Expires January 14, 2010 [Page 19] Internet-Draft IGP Convergence Benchmark Terminology July 2009 3.6.6. Loss-Derived Loss of Connectivity Period Definition: The average time duration of traffic loss for all routes following a Convergence Event until Full Convergence completion, as observed using the Loss-Derived Method. Discussion: In general the Loss-Derived Loss of Connectivity Period is not equal to the Loss-Derived Convergence Time. If the DUT continues to forward traffic to the Preferred Egress Interface after the Convergence Event is applied then the Loss-Derived Loss of Connectivity Period will be smaller than the Loss-Derived Convergence Time. This is also specifically the case after reversing a failure event. The Loss-Derived Loss of Connectivity Period may be equal to the Loss-Derived Convergence Time if, as a characteristic of the Convergence Event, traffic for all routes starts dropping instantaneously on the Convergence Event Instant. See discussion in [Po09m]. For the testcases described in [Po09m] each route's Route Loss of Connectivity Period is expected to be a single Loss Period [Ko02]. When benchmarking Loss-Derived Loss of Connectivity Period, Connectivity Packet Loss is measured for all routes and Equation 8 is applied. The calculation is equal to Equation 5 in Section 3.6.4. Loss-Derived Loss of Connectivity Period = Connectivity Packet Loss for all routes/Offered Load Equation 8 Loss-Derived Loss of Connectivity Period SHOULD be measured using Loss-Derived Method. Measurement Units: seconds Issues: None See Also: Loss-Derived Convergence Time, Loss-Derived Method, Connectivity Packet Loss Poretsky, et al. Expires January 14, 2010 [Page 20] Internet-Draft IGP Convergence Benchmark Terminology July 2009 3.7. Measurement Terms 3.7.1. Convergence Event Definition: The occurrence of a planned or unplanned event in the network that will result in a change in the egress interface of the Device Under Test (DUT) for routed packets. Discussion: Convergence Events include but are not limited to link loss, routing protocol session loss, router failure, configuration change, and better next-hop learned via a routing protocol. Measurement Units: N/A Issues: None See Also: Convergence Event Instant 3.7.2. Packet Loss Definition: The number of packets that should have been forwarded by a DUT under a constant Offered Load that were not forwarded due to lack of resources. Discussion: Packet Loss is a modified version of the term "Frame Loss Rate" as defined in [Br91]. The term "Frame Loss" is intended for Ethernet Frames while "Packet Loss" is intended for IP packets. Measurement units: Number of offered packets that are not forwarded. Issues: None See Also: Convergence Packet Loss 3.7.3. Convergence Packet Loss Definition: Poretsky, et al. Expires January 14, 2010 [Page 21] Internet-Draft IGP Convergence Benchmark Terminology July 2009 The number of packets lost due to a Convergence Event until Full Convergence completes, as observed on the Next-Best Egress Interface. Discussion: Convergence Packet Loss is observed on the Next-Best Egress Interface. It only needs to be observed for Convergence Events that do not cause instantaneous traffic loss at Convergence Event Instant. Convergence Packet Loss includes packets that were lost and packets that were delayed due to buffering. The magnitude of an acceptable Forwarding Delay is a parameter of the methodology. If a maximum acceptable Forwarding Delay threshold is applied it MUST be reported. Measurement Units: number of packets Issues: None See Also: Packet Loss, Full Convergence, Convergence Event, Connectivity Packet Loss 3.7.4. Connectivity Packet Loss Definition: The number of packets lost due to a Convergence Event until Full Convergence completes. Discussion: Connectivity Packet Loss is observed on all DUT egress interfaces. Convergence Packet Loss includes packets that were lost and packets that were delayed due to buffering. The magnitude of an acceptable Forwarding Delay is a parameter of the methodology. If a maximum acceptable Forwarding Delay threshold is applied it MUST be reported. Measurement Units: number of packets Issues: None See Also: Packet Loss, Route Loss of Connectivity Period, Convergence Event, Convergence Packet Loss Poretsky, et al. Expires January 14, 2010 [Page 22] Internet-Draft IGP Convergence Benchmark Terminology July 2009 3.7.5. Packet Sampling Interval Definition: The interval at which the Tester (test equipment) polls to make measurements for arriving packets. Discussion: At least one packet per route in the FIB for all routes MUST be offered to the DUT within the Packet Sampling Interval. Metrics measured at the Packet Sampling Interval MUST include Forwarding Rate and received packets. Packet Sampling Interval can influence the Convergence Graph as observed with the Rate-Derived Method. This is particularly true when implementations complete Full Convergence in less time than the Packet Sampling Interval. The Convergence Event Instant and First Route Convergence Instant may not be easily identifiable and the Rate-Derived Method may produce a larger than actual convergence time. The recommended value for configuration of the Packet Sampling Interval when using the Rate-Derived Method is provided in [Po09m]. For the other benchmark methods the value of the Packet Sampling Interval does not contribute to the measurement accuracy. Measurement Units: seconds Issues: None See Also: Rate-Derived Method 3.7.6. Sustained Convergence Validation Time Definition: The amount of time for which the completion of Full Convergence is maintained without additional packet loss. Discussion: The purpose of the Sustained Convergence Validation Time is to produce convergence benchmarks protected against fluctuation in Forwarding Rate after the completion of Full Convergence is observed. The RECOMMENDED Sustained Convergence Validation Time to be used is 5 seconds. The BMWG selected 5 seconds based upon RFC 2544 [Br99] which recommends waiting 2 seconds for residual frames to arrive and Poretsky, et al. Expires January 14, 2010 [Page 23] Internet-Draft IGP Convergence Benchmark Terminology July 2009 5 seconds for DUT restabilization. Measurement Units: seconds Issues: None See Also: Full Convergence, Convergence Recovery Instant 3.8. Miscellaneous Terms 3.8.1. Stale Forwarding Definition: Forwarding of traffic to route entries that no longer exist or to route entries with next-hops that are no longer preferred. Discussion: Stale Forwarding can be caused by a Convergence Event and can manifest as a "black-hole" or microloop that produces packet loss, or out-of-order packets, or delayed packets. Stale Forwarding can exist until Network Convergence is completed. Measurement Units: N/A Issues: None See Also: Network Convergence 3.8.2. Nested Convergence Event Definition: The occurrence of a Convergence Event while the route table is converging from a prior Convergence Event. Discussion: The Convergence Events for a Nested Convergence Event MUST occur with different neighbors. A possible observation from a Nested Convergence Event will be the withdrawal of routes from one neighbor while the routes of another neighbor are being installed. Measurement Units: N/A Poretsky, et al. Expires January 14, 2010 [Page 24] Internet-Draft IGP Convergence Benchmark Terminology July 2009 Issues: None See Also: Convergence Event 4. Security Considerations Documents of this type do not directly affect the security of Internet or corporate networks as long as benchmarking is not performed on devices or systems connected to production networks. Security threats and how to counter these in SIP and the media layer is discussed in RFC3261, RFC3550, and RFC3711 and various other drafts. This document attempts to formalize a set of common methodology for benchmarking IGP convergence performance in a lab environment. 5. IANA Considerations This document requires no IANA considerations. 6. Acknowledgements Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward, Peter De Vriendt and the BMWG for their contributions to this work. 7. References 7.1. Normative References [Br91] Bradner, S., "Benchmarking terminology for network interconnection devices", RFC 1242, July 1991. [Br97] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, March 1999. [Ca90] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual environments", RFC 1195, December 1990. [Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for IPv6", RFC 5340, July 2008. [Ho08] Hopps, C., "Routing IPv6 with IS-IS", RFC 5308, Poretsky, et al. Expires January 14, 2010 [Page 25] Internet-Draft IGP Convergence Benchmark Terminology July 2009 October 2008. [Ko02] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample Metrics", RFC 3357, August 2002. [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching Devices", RFC 2285, February 1998. [Mo06] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, S., and J. Perser, "Packet Reordering Metrics", RFC 4737, November 2006. [Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998. [Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, "Terminology for Benchmarking Network-layer Traffic Control Mechanisms", RFC 4689, October 2006. [Po09a] Poretsky, S., "Considerations for Benchmarking Link-State IGP Data Plane Route Convergence", draft-ietf-bmwg-igp-dataplane-conv-app-17 (work in progress), March 2009. [Po09m] Poretsky, S. and B. Imhoff, "Benchmarking Methodology for Link-State IGP Data Plane Route Convergence", draft-ietf-bmwg-igp-dataplane-conv-meth-18 (work in progress), July 2009. 7.2. Informative References [Ca01] Casner, S., Alaettinoglu, C., and C. Kuan, "A Fine-Grained View of High Performance Networking", NANOG 22, June 2001. [Ci03] Ciavattone, L., Morton, A., and G. Ramachandran, "Standardized Active Measurements on a Tier 1 IP Backbone", IEEE Communications Magazine p90-97, May 2003. Poretsky, et al. Expires January 14, 2010 [Page 26] Internet-Draft IGP Convergence Benchmark Terminology July 2009 Authors' Addresses Scott Poretsky Allot Communications 67 South Bedford Street, Suite 400 Burlington, MA 01803 USA Phone: + 1 508 309 2179 Email: sporetsky@allot.com Brent Imhoff Juniper Networks 1194 North Mathilda Ave Sunnyvale, CA 94089 USA Phone: + 1 314 378 2571 Email: bimhoff@planetspork.com Kris Michielsen Cisco Systems 6A De Kleetlaan Diegem, BRABANT 1831 Belgium Email: kmichiel@cisco.com Poretsky, et al. Expires January 14, 2010 [Page 27]