Benchmarking Methodology WG R.Papneja
Internet Draft Isocore
Intended status: Informational
Expires: April 2011 B.Parise
Cisco Systems
Susan Hares
Huawei
October 18, 2010
Basic BGP Convergence Benchmarking Methodology for Data Plane
Convergence
draft-papneja-bgp-basic-dp-convergence-00.txt
Abstract
BGP is widely deployed and used by several service providers as the
default Inter AS routing protocol. It is of utmost importance to
ensure that when a BGP peer or a downstream link of a BGP peer
fails, the alternate paths are rapidly used and routes via these
alternate paths are installed. This document provides the basic BGP
Benchmarking Methodology using existing BGP Convergence Terminology,
RFC-4098.
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with
the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
This Internet-Draft will expire on April 18, 2009.
Papneja, et al. Expires April 18, 2011 [Page 1]
Internet-Draft BGP Convergence Methodology October 2010
Copyright Notice
Copyright (c) 2010 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described
in Section 4.e of the Trust Legal Provisions and are provided
without warranty as described in the Simplified BSD License.
This document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November
10, 2008. The person(s) controlling the copyright in some of this
material may not have granted the IETF Trust the right to allow
modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s)
controlling the copyright in such materials, this document may not
be modified outside the IETF Standards Process, and derivative
works of it may not be created outside the IETF Standards Process,
except to format it for publication as an RFC or to translate it
into languages other than English.
Table of Contents
1. Introduction 3
1.1. Document Scope 5
2. Existing definitions and requirements 5
3. Test Topologies 6
3.1. General Reference Topology 6
4. Test Considerations 8
4.1. Number of Peers 8
4.2. Number of Routes per Peer 8
4.3. Policy Processing/Reconfiguration 9
4.4. Configured Parameters (Timers, etc..) 9
4.5. Interface Types 10
4.6. Measurement Accuracy 10
4.7. Measurement Statistics 11
4.8. Authentication 11
4.9. Convergence Events 11
4.10. High Availability 12
Papneja, et al. Expires April 18, 2011 [Page 2]
Internet-Draft BGP Convergence Methodology October 2010
5. Test Cases 12
5.1. Basic Convergence Tests 12
5.1.1. RIB-IN Convergence 12
5.1.2. RIB-OUT Convergence 14
5.1.3. eBGP Convergence 15
5.1.4. iBGP Convergence 15
5.1.5. eBGP Multihop Convergence 16
5.2. BGP Failure/Convergence Events 17
5.2.1. Physical Link Failure on DUT End 17
5.2.2. Physical Link Failure on Remote/Emulator End 18
5.2.3. ECMP Link Failure on DUT End 19
5.3. BGP Adjacency Failure (Non-Physical Link Failure) on
Emulator 19
5.4. BGP Hard Reset Test cases 20
5.4.1. BGP Non-Recovering Hard Reset Event on DUT 20
5.5. BGP Soft Reset 22
5.6. BGP Route Withdrawal Convergence Time 23
5.7. BGP Path Attribute Change Convergence Time 25
5.8. BGP Graceful Restart Convergence Time 26
6. Reporting Format 28
7. Security Considerations 30
8. IANA Considerations 31
9. References 31
9.1. Normative References 31
9.2. Informative References 31
Authors Addresses 33
1. Introduction
This document defines the methodology for benchmarking data plane
FIB convergence performance of BGP in router and switches for simple
topologies of 3 or 4 nodes.
The methodology proposed in this document applies to both IPv4 and
IPv6 and if a particular test is unique to one version, it is marked
accordingly. For IPv6 benchmarking the device under test will
require the support of Multi-Protocol BGP (MP-BGP) [RFC2858,
RFC2545].
The scope of this companion document is limited to basic BGP
protocol FIB convergence measurements. BGP extensions outside of
Papneja, et al. Expires April 18, 2011 [Page 3]
Internet-Draft BGP Convergence Methodology October 2010
carrying IPv6 in (MP-BGP) [RFC2858, RFC2545] are outside the scope
of this document. Interaction with IGPs (IGP interworking) is
outside the scope of this document.
1.1 Precise Benchmarking definition
Since benchmarking is science of precision, let us restate the
purpose of this document in benchmarking terms. This document
defines methodology to test
- data plane convergence on a single BGP device that supports the
BGP [RFC4271] functionality;
- in test topology of 3 or 4 nodes,
- using Basic BGP.
Data plane convergence is defined as the completion of all FIB
changes so that all forwarded traffic now takes the new proposed
route. RFC 4098 defines the terms BGP device, FIB and the forwarded
traffic. Data plane convergence is different than control plane
convergence within a node.
Basic BGP is defined as RFC 4271 functional with Multi-Protocol BGP
(MP-BGP) [RFC2858, RFC2545] for IPv6. The use of other extensions of
BGP to support layer-2, layer-3 virtual private networks (VPN) are
out of scope of this document.
The terminology used in this document is defined in [RFC4098]. One
additional term is defined in this draft: data plane BGP
convergence.
1.2 Purpose of BGP FIB (data plane) convergence
In the current Internet architecture the Inter-Autonomous System
(inter-AS) transit is primarily available through BGP. To maintain a
reliable connectivity within intra-domains or across inter-domains,
fast recovery from failures remains most critical. To ensure minimal
traffic losses, many service providers are requiring BGP
implementations to converge the entire Internet routing table within
sub-seconds at FIB level.
Furthermore, to compare these numbers amongst various devices,
service providers are also looking at ways to standardize the
convergence measurement methods. This document offers test methods
for simple topologies. These simple tests will provide a quick high-
Papneja, et al. Expires April 18, 2011 [Page 4]
Internet-Draft BGP Convergence Methodology October 2010
level check, of the BGP data plane convergence across multiple
implementations.
1.2 Control Plane Convergence
The convergence of BGP occurs at two levels: RIB and FIB
convergence. RFC 4098 defines terms for BGP control plane
convergence. Methodologies which test control plane convergence are
out of scope for this draft.
1.3 Benchmarking Testing
In order to ensure that the results obtained in tests are
repeatable, careful setup of initial conditions and exact steps are
required.
This document proposes these initial conditions, test steps, and
result checking. To ensure uniformity of the results all optional
parameters SHOULD be disabled and all settings SHOULD be changed to
default, these may include BGP timers as well.
2. Existing definitions and requirements
RFC 1242, "Benchmarking Terminology for Network Interconnect
Devices" [RFC1242] and RFC 2285, "Benchmarking Terminology for LAN
Switching Devices" [RFC2285] SHOULD be reviewed in conjunction with
this document. WLAN-specific terms and definitions are also
provided in Clauses 3 and 4 of the IEEE 802.11 standard [802.11].
Commonly used terms may also be found in RFC 1983 [RFC1983].
For the sake of clarity and continuity, this document adopts the
general template for benchmarking terminology set out in Section 2
of RFC 1242. Definitions are organized in alphabetical order, and
grouped into sections for ease of reference.
The following terms are assumed to be taken as defined in RFC 1242
[RFC1242]: Throughput, Latency, Constant Load, Frame Loss Rate, and
Overhead Behavior. In addition, the following terms are taken as
defined in RFC 2285 [RFC2285]: Forwarding Rates, Maximum Forwarding
Rate, Loads, Device Under Test (DUT), and System Under Test (SUT).
Papneja, et al. Expires April 18, 2011 [Page 5]
Internet-Draft BGP Convergence Methodology October 2010
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
3. Test Topologies
This section describes simple test setups for use in BGP
benchmarking tests measuring convergence of the FIB (data plane)
after the BGP updates has been received.
These simple test nodes have 3 or 4 nodes with the following
configuration:
1. Basic Test Setup
2. Three node setup for iBGP or eBGP convergence
3. Setup for eBGP multihop test scenario
4. Four node setup for iBGP or eBGP convergence
Individual tests refer to these topologies.
Figures 1-4 use the following conventions
AS-X: Autonomous System X
Loopback Int: Loopback interface on the BGP enabled device
R2: Helper router
3.1. General Reference Topology
Emulator acts as 1 or more BGP peers for different testcases.
----------- -------------
| | traffic interfaces | |
| |-----------------------1---- | tx |
| |-----------------------2---- | tr1 |
| |-----------------------3-----| tr2 |
| DUT | routing interfaces | Emulator |
| | | |
| Drr1|--------------------------- |Err1 |
| | BGP Peering | |
| Drr2|---------------------------- |Err2 |
| | BGP Peering | |
----------- -------------
Figure 1 Basic Test Setup
Papneja, et al. Expires April 18, 2011 [Page 6]
Internet-Draft BGP Convergence Methodology October 2010
------------- ------------- -------------
| | | | | |
| | | | | |
| HLP | | DUT | | Emulator |
| (AS-X) |--------| (AS-Y) |-----------| (AS-Z) |
| | | | | |
| | | | | |
| | | | | |
------------- ------------- -------------
Figure 2 Three Node Setup for eBGP and iBGP Convergence
------------- ------------- -------------
| | | | | |
| | | | | |
| R1 | | DUT | | Emulator |
| (AS-X) |--------| (AS-Y) |-----------| (AS-Z) |
| | | | | |
| | | | | |
| | | | | |
------------- ------------- -------------
|Loopback-Int |Loopback-Int
| |
+ +
Figure 3 BGP Convergence for eBGP Multihop Scenario
---------- ---------- ---------- ----------
| | | | | | | |
| | | | | | | |
| R1 | | DUT2 | | DUT1 | |Emulator |
| (AS-X) |-----| (AS-X) |-----| (AS-Y) |-----| (AS-Z) |
| | | | | | | |
| | | | | | | |
| | | | | | | |
---------- ---------- ---------- ----------
Figure 4 Four Node Setup for EBGP and IBGP Convergence
Papneja, et al. Expires April 18, 2011 [Page 7]
Internet-Draft BGP Convergence Methodology October 2010
4. Test Considerations
The test cases for measuring convergence for iBGP and eBGP are
different. Both iBGP and eBGP use different mechanisms to advertise,
install and learn the routes. Typically, an iBGP route on the DUT
is installed and exported only when the next-hop is reachable. For
eBGP the route is installed on the DUT with the remote interface
address as the next-hop with the exception of the multihop case.
4.1. Number of Peers
Number of Peers is defined as the number of BGP neighbors or
sessions the DUT has at the beginning of the test.
The peers are established before the tests begin.
The relationship could be either, iBGP or eBGP peering depending
upon the test case requirement.
The DUT establishes one or more BGP sessions with one more emulated
routers or helper nodes. Additional peers can be added based on the
testing requirements. The number of peers enabled during the testing
should be well documented in the report matrix.
4.2. Number of Routes per Peer
It Number of Routes per Peer is defined as the number of routes
advertized or learnt by the DUT per session or through neighbor
relationship with an emulator or helper node. The tester, emulating
as neighbor MUST advertise at least one route per peer.
Each test must run must identify the route stream in terms of route
packing, route mixture, and number of routes. This route stream must
be well documented in the reporting stream. RFC 4098 defines these
terms.
It is RECOMMENDED that the user may consider advertizing the entire
current Internet routing table per peering session using an Internet
route mixture with unique or non-unique routes.
If multiple peers are used, it is important to precisely document
the timing sequence between the peer sending routes (as defined in
RFC 4098).
Papneja, et al. Expires April 18, 2011 [Page 8]
Internet-Draft BGP Convergence Methodology October 2010
4.3. Policy Processing/Reconfiguration
The DUT MUST run one baseline test where policy is Minimum policy as
defined in RFC 4098. Additional runs may be done with policy set-up
before the tests begin. Exact policy settings should be documented
as part of the test.
4.4. Configured Parameters (Timers, etc..)
There are configured parameters and timers that may impact the
measured BGP convergence times.
The benchmark metrics MAY be measured at any fixed values for these
configured parameters.
It is RECOMMENDED these configure parameters have two settings: a)
basic-test, and b)values as expected in the operational network.
All optional BGP settings MUST be kept consistent across iterations
of any specific tests
Examples of the configured parameters that may impact measured BGP
convergence time include, but are not limited to:
1. Interface failure detection timer
2. BGP Keepalive timer
3. BGP Holdtime
4. BGP update delay timer
5. ConnectRetry timer
6. TCP Segment Size
7. Minimum Route Advertisement Interval (MRAI)
8. MinASOriginationInterval (MAOI)
9. Route Flap Dampening parameters
10. TCP MD5
The basic-test settings for the parameters should be:
Papneja, et al. Expires April 18, 2011 [Page 9]
Internet-Draft BGP Convergence Methodology October 2010
1. Interface failure detection timer (0 ms)
2. BGP Keepalive timer (1 min)
3. BGP Holdtime (3 min)
4. BGP update delay timer (0 s)
5. ConnectRetry timer (1 s)
6. TCP Segment Size (4096)
7. Minimum Route Advertisement Interval (MRAI)(0 s)
8. MinASOriginationInterval (MAOI) (0 s)
9. Route Flap Dampening parameters (off)
10. TCP MD5 (off)
4.5. Interface Types
The type of media dictate which test cases may be executed, each
interface type has unique mechanism for detecting link failures and
the speed at which that mechanism operates will influence the
measurement results. All interfaces MUST be of the same media and
throughput for each test case.
4.6. Measurement Accuracy
Since observed packet loss is used to measure the route convergence
time, the time between two successive packets offered to each
individual route is the highest possible accuracy of any packet-loss
based measurement. When packet jitter is much less than the
convergence time, it is a negligible source of error and hence it
will be treated as within tolerance.
An exterior measurement on the input media (such Ethernet)is defined
by this specification.
Papneja, et al. Expires April 18, 2011 [Page 10]
Internet-Draft BGP Convergence Methodology October 2010
4.7. Measurement Statistics
The benchmark measurements may vary for each trial, due to the
statistical nature of timer expirations, CPU scheduling, etc.
It is recommended to repeat the test multiple times. Evaluation of
the test data must be done with an understanding of generally
accepted testing practices regarding repeatability, variance and
statistical significance of a small number of trials.
For any repeated tests that are averaged to remove variance, all
parameters MUST remain the same.
4.8. Authentication
Authentication in BGP is done using the TCP MD5 Signature Option
[RFC2385]. The processing of the MD5 hash, particularly in devices
with a large number of BGP peers and a large amount of update
traffic, can have an impact on the control plane of the device. If
authentication is enabled, it SHOULD be documented correctly in the
reporting format
4.9. Convergence Events
Convergence events or triggers are defined as abnormal occurrences
in the network, which initiate route flapping in the network, and
hence forces the re-convergence of a steady state network. In a real
network, a series of convergence events may cause convergence
latency operators desire to test.
These convergence events must be defined in terms of the sequences
defined in RFC 4098. This basic document begins all tests with a
router initial set-up. Additional documents will define BGP data
plane convergence based on peer initialization.
The convergence events may or may not be tied to the actual failure
A Soft Reset (RFC 4098) does not clear the RIB or FIB tables. A
Hard reset clears the BGP peer sessions, the RIB tables, and FIB
tables.
Papneja, et al. Expires April 18, 2011 [Page 11]
Internet-Draft BGP Convergence Methodology October 2010
4.10. High Availability
Due to the different Non-Stop-Routing (sometimes referred to High-
Availability) solutions available from different vendors, it is
RECOMMENDED that any redundancy available in the routing processors
should be disabled during the convergence measurements.
5. Test Cases
All tests defined under this section assume the following:
BGP peers should be brought to BGP Peer established state.
a. Furthermore the traffic generation and routing should be verified
in the topology
5.1. Basic Convergence Tests
These test cases measure characteristics of a BGP implementation in
non-failure scenarios like:
a. RIB-IN Convergence
b. RIB-OUT Convergence
c. eBGP Convergence
d. iBGP Convergence
5.1.1. RIB-IN Convergence
Objective:
This test measures the convergence time taken to receive and install a
route in RIB using BGP
Reference Test Setup:
This test uses the setup as shown in figure 1
Procedure:
Papneja, et al. Expires April 18, 2011 [Page 12]
Internet-Draft BGP Convergence Methodology October 2010
a. All variables affecting Convergence should be set to a basic
test state (as defined in section 4-4).
b. Establish BGP adjacency between DUT and peer x of Emulator;
c. To ensure adjacency establishment, wait for 3 KeepAlives from
the DUT or a configurable delay before proceeding with the rest
of the test;
d. Start the traffic from the Emulator peer-x towards the DUT
targeted at a routes specified in route mixture (ex. route A)
Initially no traffic SHOULD be observed on the egress interface
as the route A is not installed in the forwarding database of
the DUT.
e. Advertise route A from the Peer-x to the DUT and record the
time;
This is Tup(EMx,Rt-A). (nick-name XMT-Rt-time)
f. Record the time when the route-A from Peer-x is received at the
DUT.
This Tup(DUT,Rt-A). It is nick named is RCV-Rt-time
g. Record the time when the traffic targeted towards route A is
received by Emulator on appropriate traffic egress interface.
rd
If 3 party route (traffic-egress 2), or BGP peer route
interfaces.
This is TR(TDx,Rt-A). This is "nick-named" DUT-XMT-Data-Time.
h. The difference between the Tup(TDx,RT-A) and traffic received
time (TR (TDr, Rt-A) is the FIB Convergence Time for route-A in
the route mixture.
A full convergence for the route update is the measurement
between the 1st route (Route-A) and the last route (Rt-last)
Route update convergence is
TR(TDr, RT-last)- Tup(DUT, Rt-A) or
(DUT-XMT-Data-Time - RCV-Rt-Time)(rt-A)
Papneja, et al. Expires April 18, 2011 [Page 13]
Internet-Draft BGP Convergence Methodology October 2010
Note: It is recommended that a single test with the same route
mixture be repeated several times. A report should provide the
Stand deviation of all tests and the average.
Running tests with a varying number of routes and route mixtures
is important to get a full characterization of a single peer.
5.1.2. RIB-OUT Convergence
Objective:
This test measures the convergence time taken by an implementation to
receive, install and advertise a route using BGP
Reference Test Setup:
This test uses the setup as shown in figure 2
Procedure:
a. The Helper node (HLP) run same version of BGP as DUT;
b. All devices MUST be synchronized using NTP or some local
reference clock;
c. All configuration variables for HLP, DUT, and Emulator SHOULD
be set to the same values. These values MAY be basic-test or a
unique set completely described in the test set-up.
d. Establish BGP adjacency between DUT and Emulator
e. Establish BGP adjacency between DUT and Helper Node
f. To ensure adjacency establishment, wait for 3 KeepAlives from
the DUT or a configurable delay before proceeding with the rest
of the test
g. Start the traffic from the Emulator towards the Helper Node
targeted at a specific route say route A.
Initially no traffic SHOULD be observed on the egress interface
as the route-A is not installed in the forwarding database of
the DUT
h. Advertise routeA from the Emulator to the DUT and note the
time.
This is Tup(EMx, Route-A). (nick-name EM-XMT-Rt-Time)
Papneja, et al. Expires April 18, 2011 [Page 14]
Internet-Draft BGP Convergence Methodology October 2010
i. Record when Route-A is received by DUT.
This is Tup(DUTr, Route-A). (nick-name DUT-RCV-Rt-Time)
j. Record the time when the ROUTE forward by DUT toward the Helper
node.
This is Tup(DUTx, Rt-A). (nick-name DUT-XMT-Rt-Time).
k. Record the time when the traffic targeted towards route-A is
received on the Route Egress Interface toward peer-X.
This is TR(EMr, Route-A). (nick-name DUT-XMT-Data Time).
FIB convergence = (DUT-RCV-Rt-Time - DUT-XMT-Data-Time).
RIB convergence = (DUT-RCV-Rt-Time - DUT-XMT-Rt-Time).
Convergence for a route stream is characterized by
a) Individual route convergence for FIB, RIB
b) All route convergence of
FIB-convergence =DUT-RCV-Rt-Time(A)-DUT-XMT-Data-Time(last)
RIB-convergence =DUT-RCV-Rt-Time(A)-DUT-XMT-Rt-Time(last)
5.1.3. eBGP Convergence
Objective:
This test measures the convergence time taken by an implementation to
receive, install and advertise a route in an eBGP Scenario
Reference Test Setup:
This test uses the setup as shown in figure 2, and the scenarios
described in RIB-IN and RIB-OUT are applicable to this test case.
5.1.4. iBGP Convergence
Objective:
Papneja, et al. Expires April 18, 2011 [Page 15]
Internet-Draft BGP Convergence Methodology October 2010
This test measures the convergence time taken by an implementation to
receive, install and advertise a route in an iBGP Scenario
Reference Test Setup:
This test uses the setup as shown in figure 2, and the test scenarios
listed in RIB-IN and RIB-OUT are applicable to this test case.
5.1.5. eBGP Multihop Convergence
Objective
This test measures the convergence time taken by an implementation to
receive, install and advertise a route in an eBGP Multihop Scenario
Reference Test Setup:
This test uses the setup as shown in figure 3. Two DUTs are used along
with a helper node.
Procedure:
a. The DUT2 is the same model as DUT and runs the same BGP
implementation as DUT.
b. All devices to be synchronized using NTP
c. All variables affecting Convergence like authentication,
policies, timers should be set to basic-settings.
d. All 3 devices, DUT, Emulator and Helper Node are configured
as different Autonomous Systems
e. Loopback Interfaces configured on DUT and Helper Node and
connectivity is established between them using any config
options available on the DUT
f. Establish BGP adjacency between DUT1 and Emulator
g. Establish BGP adjacency between DUT2 and Helper Node
h. Establish BGP adjacency between DUT 1 and DUT 2
i. To ensure adjacency establishment, wait for 3 KeepAlives from
the DUT1 and DUT2 or a configurable delay before proceeding
with the rest of the test
j. Start the traffic from the Emulator towards the Helper Node
targeted at a specific route say routeA.
Papneja, et al. Expires April 18, 2011 [Page 16]
Internet-Draft BGP Convergence Methodology October 2010
k. Initially no traffic SHOULD be observed on the egress
interface as the routeA is not installed in the forwarding
database of the DUT
l. Advertise routeA from the Emulator to the DUT and note the
time. (Tup(EMx,RouteA) - this is nicknamed (Route-Rec-time).
m. Record the time when the traffic targeted towards routeA is
received from Egress Interface of DUT on emulator.
This is TR(EMr,DUT), nicknamed (Data Receive time)
n. The following equation represents the FIB Convergence multi-
node
eBGP Multihop Convergence Time =
(Rt-RecTime - Data-RcvTime).
Note: It is recommended that the test be repeated with varying number
of routes and route mixtures. With each set route mixture, the test
should be repeated multiple times. The results should record average,
mean, Standard Deviation.
5.2. BGP Failure/Convergence Events
5.2.1. Physical Link Failure on DUT End
Objective:
This test measures the route convergence time due to local link failure
event at DUT's Local Interface
Reference Test Setup:
This test uses the setup as shown in figure 1. Shutdown event is
defined as an administrative shutdown event on the DUT.
Procedure:
a. All variables affecting Convergence like authentication,
policies, timers should be set to basic-test policy.
Papneja, et al. Expires April 18, 2011 [Page 17]
Internet-Draft BGP Convergence Methodology October 2010
b. Establish 2 BGP adjacencies from DUT to Emulator, one over
the peer interface and the other using a second peer
interface.
c. Advertise the same route, route A over both the adjacencies
and (Tx1)Interface to be the preferred next hop.
d. To ensure adjacency establishment, wait for 3 KeepAlives from
the DUT or a configurable delay before proceeding with the
rest of the test.
e. Start the traffic from the Emulator towards the DUT targeted
at a specific route say route A. Initially traffic would be
observed on the best egress route (Err1) instead of Trr2
f. Trigger the shutdown event of Best Egress Interface on DUT
(Drr1).
g. Measure the Convergence Time for the event to be detected and
traffic to be forwarded to Next-Best Egress Interface (rr2).
Time = Data-detect(rr2) - Shutdown time.
h. Stop the offered load and wait for the queues to drain and
Restart
i. Bring up the link on DUT Best Egress Interface
j. Measure the convergence time taken for the traffic to be
rerouted from (rr2) to Best Interface (rr1)
Time = Data-Detect(rr1) - Shutdown-time.
k. It is recommended that the test be repeated with varying
number of routes and route mixtures or with number of routes
& route mixtures closer to what is deployed in operational
networks
5.2.2. Physical Link Failure on Remote/Emulator End
Objective:
This test measures the route convergence time due to local link failure
event at Tester's Local Interface
Papneja, et al. Expires April 18, 2011 [Page 18]
Internet-Draft BGP Convergence Methodology October 2010
Reference Test Setup:
This test uses the setup as shown in figure 1. Shutdown event is
defined as shutdown of the local interface of Tester via logical
shutdown event. The procedure used in 5.2.1 is used for the
termination.
5.2.3. ECMP Link Failure on DUT End
Objective:
This test measures the route convergence time due to local link failure
event at ECMP Member. The FIB configuration and BGP is set to allow two
ECMP routes to be installed. However, policy directs the routes to be
sent only over one of the paths.
Reference Test Setup:
This test uses the setup as shown in figure 1, and the procedure uses
5.2.1.
5.3. BGP Adjacency Failure (Non-Physical Link Failure) on Emulator
Objective:
This test measures the route convergence time due to BGP Adjacency
Failure on Emulator
Reference Test Setup:
This test uses the setup as shown in figure 1
Procedure:
a. All variables affecting Convergence like authentication,
policies, timers should be basic-policy set.
Papneja, et al. Expires April 18, 2011 [Page 19]
Internet-Draft BGP Convergence Methodology October 2010
b. Establish 2 BGP adjacencies from DUT to Emulator, one over
the Best Egress Interface and the other using the Next-Best
Egress Interface
c. Advertise the same route, routeA over both the adjacencies
and make Best Egress Interface to be the preferred next hop
d. To ensure adjacency establishment, wait for 3 KeepAlives from
the DUT or a configurable delay before proceeding with the
rest of the test
e. Start the traffic from the Emulator towards the DUT targeted
at a specific route say routeA. Initially traffic would be
observed on the Best Egress interface
f. Remove BGP adjacency via a software adjacency down on the
Emulator on the Best Egress Interface
Time = BGPadj-down-time - nicknamed BGPpeer-down.
g. Measure the Convergence Time for the event to be detected and
traffic to be forwarded to Next-Best Egress Interface
This time is Tr-rr2 nicknamed - TR2-traffic-on
Convergence = TR2-traffic-on - BGPpeer-down
h. Stop the offered load and wait for the queues to drain and
Restart
i. Bring up BGP adjacency on the Emulator over the Best Egress
Interface
Time = BGP-adj-up - nicknamed BGPpeer-up
j. Measure the convergence time taken for the traffic to be
rerouted to Best Interface
Time = Tr-rr1 is nicknamed TR1-traffic-on.
5.4. BGP Hard Reset Test cases
5.4.1. BGP Non-Recovering Hard Reset Event on DUT
Objective:
Papneja, et al. Expires April 18, 2011 [Page 20]
Internet-Draft BGP Convergence Methodology October 2010
This test measures the route convergence time due to Hard Reset on the
DUT
Reference Test Setup:
This test uses the setup as shown in figure 1
Procedure:
a. The requirement for this test case is that the Hard Reset
Event should be non-recovering and should affect only the
adjacency between DUT and Emulator on the Best Egress
Interface
b. All variables affecting SHOULD be set to basic-test values
c. Establish 2 BGP adjacencies from DUT to Emulator, one over
the Best Egress Interface and the other using the Next-Best
Egress Interface
d. Advertise the same route, routeA over both the adjacencies
and make Best Egress Interface to be the preferred next hop
e. To ensure adjacency establishment, wait for 3 KeepAlives from
the DUT or a configurable delay before proceeding with the
rest of the test
f. Start the traffic from the Emulator towards the DUT targeted
at a specific route say routeA. Initially traffic would be
observed on the Best Egress interface
g. Trigger the Hard Reset event of Best Egress Interface on DUT
h. Measure the Convergence Time for the event to be detected and
traffic to be forwarded to Next-Best Egress Interface
Time of convergence = time-traffic flow - time-reset.
i. Stop the offered load and wait for the queues to drain and
Restart
j. It is recommended that the test be repeated with varying
number of routes and route mixtures or with number of routes
& route mixtures closer to what is deployed in operational
networks
k. When varying number of routes are used, convergence Time is
measured using the Loss Derived method [IGP-Data]
Papneja, et al. Expires April 18, 2011 [Page 21]
Internet-Draft BGP Convergence Methodology October 2010
l. Convergence Time in this scenario is influenced by Failure
detection time on Tester, BGP Keep Alive Time and routing,
forwarding table update time
5.5. BGP Soft Reset
Objective:
This test measures the route convergence time taken by an
implementation to service a BGP Route Refresh message and advertise a
route
Reference Test Setup:
This test uses the setup as shown in figure 2
Procedure:
a. The BGP implementation on DUT & Helper Node needs to support
BGP Route Refresh Capability [RFC 2918]
b. All devices to be synchronized using NTP
c. All variables affecting Convergence like authentication,
policies, timers should be set to basic-test defaults.
d. DUT and Helper Node are configured in the same Autonomous
System whereas Emulator is configured under a different
Autonomous System
e. Establish BGP adjacency between DUT and Emulator
f. Establish BGP adjacency between DUT and Helper Node
g. To ensure adjacency establishment, wait for 3 KeepAlives from
the DUT or a configurable delay before proceeding with the
rest of the test
h. Configure a policy under BGP on Helper Node to deny routes
received from DUT
i. Advertise routeA from the Emulator to the DUT
j. The DUT will try to advertise the route to Helper Node will
be denied
k. Wait for 3 KeepAlives
l. Start the traffic from the Emulator towards the Helper Node
targeted at a specific route say routeA. Initially no traffic
Papneja, et al. Expires April 18, 2011 [Page 22]
Internet-Draft BGP Convergence Methodology October 2010
would be observed on the Egress interface, as routeA is not
present
m. Remove the policy on Helper Node and issue a Route Refresh
request towards DUT. Note the timestamp of this event. This
is the RefreshTime
n. Record the time when the traffic targeted towards routeA is
received on the Egress Interface. This is RecTime
o. The following equation represents the Route Refresh
Convergence Time per route
i. Route Refresh Convergence Time = (RecTime -
RefreshTime)
5.6. BGP Route Withdrawal Convergence Time
Objective:
This test measures the route convergence time taken by an
implementation to service a BGP Withdraw message and advertise the
withdraw
Reference Test Setup:
This test uses the setup as shown in figure 2
Procedure:
a. This test consists of 2 steps to determine the Total Withdraw
Processing Time
b. Step 1:
i. All devices to be synchronized using NTP
ii. All variables should be set to basic-test parameters.
iii. DUT and Helper Node are configured in the same
Autonomous System whereas Emulator is configured under a
different Autonomous System
iv. Establish BGP adjacency between DUT and Emulator
v. To ensure adjacency establishment, wait for 3 KeepAlives
from the DUT or a configurable delay before proceeding
with the rest of the test
vi. Start the traffic from the Emulator towards the DUT
targeted at a specific route say routeA. Initially no
Papneja, et al. Expires April 18, 2011 [Page 23]
Internet-Draft BGP Convergence Methodology October 2010
traffic would be observed on the Egress interface as the
routeA is not present on DUT.
vii. Advertise routeA from the Emulator to the DUT
viii. The traffic targeted towards routeA is received on the
Egress Interface
ix. Now the Tester sends request to withdraw routeA to DUT.
TRx(Awith) nicknamed WdrawTime1
x. Record the time when no traffic is observed on the
Egress Interface.
This is the RouteRemoveTime1(A)
WdrawConvTime1 = RouteRemoveTime1(A)
xi. The difference between the RouteRemoveTime1 and
WdrawTime1 is the WdrawConvTime1
c. Step 2:
i. Continuing from Step 1, re-advertise routeA back to DUT
from Tester
ii. The DUT will try to advertise the routeA to Helper Node
(assumption there exists a session between DUT and
helper node)
iii. Start the traffic from the Emulator towards the Helper
Node targeted at a specific route say routeA. Traffic
would be observed on the Egress interface after routeA
is received by the Helper Node
WATime=time traffic first flows
iv. Now the Tester sends a request to withdraw routeA to
DUT. This is the WdrawTime2
WAWtime-TRx(RouteA) = is nicknamed WdrawTime2
v. DUT processes the withdraw and sends it to Helper Node
vi. Record the time when no traffic is observed on the
Egress Interface of Helper Node. This is the
Papneja, et al. Expires April 18, 2011 [Page 24]
Internet-Draft BGP Convergence Methodology October 2010
TR-WAW(DUT,RouteA) = RouteRemoveTime2
vii. Total withdraw processing time is
TotalWdrawTime = ((RouteRemoveTime2 - WdrawTime2) -
WdrawConvTime1)
5.7. BGP Path Attribute Change Convergence Time
Objective:
This test measures the route convergence time taken by an
implementation to service a BGP Path Attribute Change
Reference Test Setup:
This test uses the setup as shown in figure 1
Procedure:
a. This test only applies to Well-Known Mandatory Attributes
like Origin, AS Path, Next Hop
b. In each iteration of test only one of these mandatory
attributes need to be varied whereas the others remain the
same
c. All devices to be synchronized using NTP
d. All variables should be set to basic-test parameters
e. Advertise the route, routeA over the Best Egress Interface
only, making it the preferred next hop
f. To ensure adjacency establishment, wait for 3 KeepAlives from
the DUT or a configurable delay before proceeding with the
rest of the test
g. Start the traffic from the Emulator towards the DUT targeted
at the specific route say routeA. Initially traffic would be
observed on the Best Egress interface
h. Now advertise the same route routeA on the Next-Best Egress
Interface but by varying one of the well-known mandatory
attributes to have a preferred value over that interface. The
Papneja, et al. Expires April 18, 2011 [Page 25]
Internet-Draft BGP Convergence Methodology October 2010
other values need to be same as what was advertised on the
Best-Egress adjacency.
TRx(Path-Change) = Path Change Event Time
i. Measure the Convergence Time for the event to be detected and
traffic to be forwarded to Next-Best Egress Interface
DUT(Path-Change, RouteA) = Path-switch time
Convergence = Path-switch time - Path Change Event Time.
j. Stop the offered load and wait for the queues to drain and
Restart
5.8. BGP Graceful Restart Convergence Time
Objective:
This test measures the route convergence time taken by an
implementation during a Graceful Restart Event
Reference Test Setup:
This test uses the setup as shown in figure 4
Procedure:
a. It measures the time taken by an implementation to service a
BGP Graceful Restart Event and advertise a route
b. The Helper Nodes are the same model as DUT and run the same
BGP implementation as DUT
c. The BGP implementation on DUT & Helper Node needs to support
BGP Graceful Restart Mechanism [RFC4724]
d. All devices to be synchronized using NTP
e. All variables are set to basic-test values.
f. DUT and Helper Node-1 are configured in the same Autonomous
System whereas Emulator and Helper Node-2 are configured
under different Autonomous Systems
g. Establish BGP adjacency between DUT and Helper Nodes
Papneja, et al. Expires April 18, 2011 [Page 26]
Internet-Draft BGP Convergence Methodology October 2010
h. Establish BGP adjacency between Helper Node-2 and Emulator
i. To ensure adjacency establishment, wait for 3 KeepAlives from
the DUT or a configurable delay before proceeding with the
rest of the test
j. Configure a policy under BGP on Helper Node-1 to deny routes
received from DUT
k. Advertise routeA from the Emulator to Helper Node-2
l. Helper Node-2 advertises the route to DUT and DUT will try to
advertise the route to Helper Node-1 which will be denied
m. Wait for 3 KeepAlives
n. Start the traffic from the Emulator towards the Helper Node-1
targeted at the specific route say routeA. Initially no
traffic would be observed on the Egress interface as the
routeA is not present
o. Perform a Graceful Restart Trigger Event on DUT and note the
time. This is the GREventTime
p. Remove the policy on Helper Node-1
q. Record the time when the traffic targeted towards routeA is
received on the Egress Interface.
TRr(DUT, routeA). This is nicknamed RecTime.
r. The following equation represents the Graceful Restart
Convergence Time
i. Graceful Restart Convergence Time = ((GREventTime -
RecTime) - RIB-IN)
s. It is assumed in this test case that after a Switchover is
triggered on the DUT, it will not have any cycles to process
BGP Refresh messages.
The reason for this assumption is that there is a narrow
window of time where after switchover when we remove the
policy from Helper Node -1, implementations might generate
Route-Refresh automatically and this request might be
serviced before the DUT actually switches over and
reestablishes BGP adjacencies with the peers
Papneja, et al. Expires April 18, 2011 [Page 27]
Internet-Draft BGP Convergence Methodology October 2010
6. Reporting Format
For each test case, it is recommended that the reporting tables
below are completed and all time values SHOULD be reported with
resolution as specified in [RFC 4098].
Parameter Units
Test case Test case number
Test topology 1,2,3 or 4
Parallel links Number of parallel links
Interface type GigE, POS, ATM, other
Convergence Event Hard reset, Soft reset, link
failure, or other defined
eBGP sessions Number of eBGP sessions
iBGP sessions Number of iBGP sessions
eBGP neighbor Number of eBGP neighbors
iBGP neighbor Number of iBGP neighbors
Routes per peer Number of routes
Total unique routes Number of routes
Total non-unique routes Number of routes
IGP configured ISIS, OSPF, static, or other
Route Mixture Description of Route mixture
Route Packing Number of routes in an update
Policy configured Yes, No
Packet size offered to the DUT Bytes
Offered load Packets per second
Packet sampling interval on Seconds
tester
Forwarding delay threshold Seconds
Timer value configured on DUT
Interface failure indication Seconds
delay
Hold timer Seconds
MinRouteAdvertisementInterval Seconds
(MRAI)
MinASOriginationInterval Seconds
(MAOI)
Keepalive Seconds
ConnectRetry Seconds
TCP Parameters for DUT and
Papneja, et al. Expires April 18, 2011 [Page 28]
Internet-Draft BGP Convergence Methodology October 2010
tester
MSS Bytes
Slow start threshold Bytes
Maximum window size Bytes
Test Details:
a. If the Offered Load matches a subset of routes, describe how
this subset is selected.
b. Describe how the Convergence Event is applied; does it cause
instantaneous traffic loss or not.
c. If there is any policy configured, describe the configured
policy.
Complete the table below for the initial Convergence Event and
the reversion Convergence Event.
Parameter Unit
Conversion Event Initial or reversion
Traffic Forwarding Metrics
Total number of packets Number of packets
offered to DUT
Total number of packets Number of packets
forwarded by DUT
Connectivity Packet Loss Number of packets
Convergence Packet Loss Number of packets
Out-of-order packets Number of packets
Duplicate packets Number of packets
Convergence Benchmarks
Rate-derived Method[IGP-
Data]:
First route convergence Seconds
time
Full convergence time Seconds
Loss-derived Method [IGP-
Data]:
Loss-derived convergence Seconds
time
Route-Specific Loss-Derived
Method:
Papneja, et al. Expires April 18, 2011 [Page 29]
Internet-Draft BGP Convergence Methodology October 2010
Minimum R-S convergence Seconds
time
Maximum R-S convergence Seconds
time
Median R-S convergence Seconds
time
Average R-S convergence Seconds
time
Loss of Connectivity Benchmarks
Loss-derived Method:
Loss-derived loss of Seconds
connectivity period
Route-Specific loss-derived
Method:
Minimum LoC period [n] Array of seconds
Minimum Route LoC period Seconds
Maximum Route LoC period Seconds
Median Route LoC period Seconds
Average Route LoC period Seconds
7. Security Considerations
Benchmarking activities as described in this memo are limited to
technology characterization using controlled stimuli in a laboratory
environment, with dedicated address space and the constraints
specified in the sections above.
The benchmarking network topology will be an independent test setup
and MUST NOT be connected to devices that may forward the test
traffic into a production network, or misroute traffic to the test
management network.
Further, benchmarking is performed on a "black-box" basis, relying
solely on measurements observable external to the DUT/SUT.
Special capabilities SHOULD NOT exist in the DUT/SUT specifically
for benchmarking purposes. Any implications for network security
arising from the DUT/SUT SHOULD be identical in the lab and in
production networks.
Papneja, et al. Expires April 18, 2011 [Page 30]
Internet-Draft BGP Convergence Methodology October 2010
8. IANA Considerations
This document requires no IANA considerations.
9. References
9.1. Normative References
[RFC1771] Rekhter, Y. and Li, T., "A Border Gateway Protocol 4(BGP-
4)", RFC 4271, March 1995.
[RFC4098] Berkowitz, H. et al., "Terminology for benchmarking BGP
device convergence in control plane", RFC4098, June 2005
[RFC1242] Bradner, S., "Benchmarking terminology for network
interconnection devices", RFC 1242, July 1991.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999
[IGP-Data] Poretsky, S. et al., "Terminology for benchmarking Link-
state IGP data plane convergence," draft-ietf-bmwg-igp-
dataplane-conv-term-21, May 2010
9.2. Informative References
[RFC2858] Bates, T. et al., "Multiprotocol extensions for BGP-4,"
RFC 2858, June 2000
[RFC2545] Marques, P. and F. Dupont, "Use of BGP-4 multiprotocol
extensions for IPv6 Inter-Domain Routing," RFC2545, March
1999
[RFC3107] Rekhter, Y. and E. Rosen, "Carrying label information in
BGP-4," RFC 3107, May 2001
[RFC2385] Heffernan, A., "Protection of BGP Sessions via the TCP MD5
signature option," RFC2385, August 1998
[RFC2918] Chen, E., "Route Refresh capability for BGP-4," RFC 2918,
September 2000
Papneja, et al. Expires April 18, 2011 [Page 31]
Internet-Draft BGP Convergence Methodology October 2010
[RFC4724] Sangli, S. et al., "Graceful restart mechanism for BGP,"
RFC 4724, Jan 2007
[RFC1983] Malkin, G.,"Internet Users' Glossary", RFC 1983, August
1996
Papneja, et al. Expires April 18, 2011 [Page 32]
Internet-Draft BGP Convergence Methodology October 2010
Authors Addresses
Rajiv Papneja
Isocore
12359 Sunrise Valley Dr. STE100
Reston, VA 20191
Phone: +1.703.860.9273
Email: rpapneja@isocore.com
Bhavani Parise
Cisco Systems
170 West Tasman Drive
San Jose, CA 95134
Phone: +408-853-6346
Email: bhavani@cisco.com
Susan Hares
Huawei Technologies (USA)
2330 Central Expressway
Santa Clara, CA 95050
Phone: +408-330-4581
Cell: +1-734-604-0332
Email shares@huawei.com
Eric Brendel
Independent Consultant
154 3rd St,
Fair Haven, NJ 07704
Phone:+1.732.895.1504
Email: brendel@pektel.com
Mohan Nanduri
Microsoft
12012 Sunset Hills Rd.
Reston, VA 20190
Phone: +1.703.627.6455
Email: mnanduri@microsoft.com
Jay Karthik
Cisco Systems
170 West Tasman Drive
San Jose, CA 95134
Phone: +1.978.319.0527
Email: jkarthik@cisco.com
Papneja, et al. Expires April 18, 2011 [Page 33]