Internet DRAFT - draft-varlashkin-router-conv-bench
draft-varlashkin-router-conv-bench
Internet Engineering Task Force I. Varlashkin
Internet-Draft Easynet Global Services
Intended status: Informational R. Papneja
Expires: April 22, 2012 Huawei Technologies (USA)
B. Parise
Cisco
T. Van Unen
Ixia
October 20, 2011
Convergence benchmarking on contemporary routers
draft-varlashkin-router-conv-bench-00
Abstract
This document specifies methodology for benchmarking convergence of
routers without making assumptions about relation and dependencies
between data- and control-planes. Provided methodology is primary
intended for testing routers running BGP and some form of link-state
IGP with or without MPLS. It may also be applicable for environments
using MPLS-TE or GRE, however they're beyond scope of this document
and such application is left for further study.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on April 22, 2012.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
Varlashkin, et al. Expires April 22, 2012 [Page 1]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
This document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November
10, 2008. The person(s) controlling the copyright in some of this
material may not have granted the IETF Trust the right to allow
modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of it may
not be created outside the IETF Standards Process, except to format
it for publication as an RFC or to translate it into languages other
than English.
Varlashkin, et al. Expires April 22, 2012 [Page 2]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Test topology . . . . . . . . . . . . . . . . . . . . . . . . 5
3. TEST PARAMETERS . . . . . . . . . . . . . . . . . . . . . . . 6
3.1. Packing ratios . . . . . . . . . . . . . . . . . . . . . . 7
3.2. Test traffic . . . . . . . . . . . . . . . . . . . . . . . 7
3.3. IGP metrics . . . . . . . . . . . . . . . . . . . . . . . 7
3.4. Internal routers matrix . . . . . . . . . . . . . . . . . 7
3.5. Number of next-hops . . . . . . . . . . . . . . . . . . . 8
3.6. 'e' - Failure and Restoration start entropy . . . . . . . 8
4. TEST PROCEDURES . . . . . . . . . . . . . . . . . . . . . . . 8
4.1. Initialisation time . . . . . . . . . . . . . . . . . . . 8
4.2. Generic data-plane failure test . . . . . . . . . . . . . 9
4.3. Generic test procedure for . . . . . . . . . . . . . . . . 9
5. Failure and restoration scenarios . . . . . . . . . . . . . . 10
5.1. Loss of Signal on the link attached to DUT . . . . . . . . 10
5.2. Link failure without LoS . . . . . . . . . . . . . . . . . 10
5.3. Non-direct link failure . . . . . . . . . . . . . . . . . 11
5.4. Best route withdrawal . . . . . . . . . . . . . . . . . . 11
5.5. iBGP next-hop failure . . . . . . . . . . . . . . . . . . 12
6. Test report . . . . . . . . . . . . . . . . . . . . . . . . . 12
7. Link bundling and Equal Cost Multi-Path . . . . . . . . . . . 13
8. Graceful Restart and Non-Stop Forwarding . . . . . . . . . . . 13
9. Security considerations . . . . . . . . . . . . . . . . . . . 13
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14
11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 14
12. Normative References . . . . . . . . . . . . . . . . . . . . . 14
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 14
Varlashkin, et al. Expires April 22, 2012 [Page 3]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
1. Introduction
Ability of the network to restore traffic flow when primary path
fails has always been important subject for network engineers,
researchers and equipment manufacturers. Time to recover from a link
or node failure has often been linked to routeing protocols
convergence; and benchmarking of a routeing protocol convergence has
often been considered sufficient for quantifying recovery
performance. As long as routers could obtain new best path only
after relevant routeing protocols perform their calculations such
methodology was reasonable. However continuous improvements in
hardware and software result in more and more routers being able to
restore traffic flow even before routeing protocols converge.
Methodology described in this document takes such fact into account.
When a failure occurs on the network a router needs to:
1. select new best path so that the packets, which already arrived
to the router, can be forwarded
2. let other routers know about new network state so they can find
new best path from their perspective
How fast a router can perform these two functions characterise
router's performance with regards to convergence. Note that in
general case each of these characteristics may or may not be related
to the other. For example, some platform may need to perform
calculations to find new best path and only then update local FIB and
send relevant protocol updates to other routers, another platform can
update local FIB without waiting for calculations to complete but
still needs to wait for calculations before sending routeing protocol
updates, third platform can use different optimisation for both FIB
changes and routeing protocol updates without waiting for completion
of the calculations. Other variations are also possible. This
document makes no assumption about whether local FIB changes and
routeing protocol updates dependencies on each other or on routeing
protocol calculations.
Since it is not known whether local FIB is updated before or after
routeing protocol calculations, forwarding-plane method is proposed
to benchmark local convergence. And because it is not known whether
routeing protocol updates are linked to FIB modification or not the
control-plane approach is used to benchmark how fast updates are
propagated. However both characteristics are benchmarked using very
similar test topologies and procedures. Also, an attempt is made to
to minimise dependency on performance on non-DUT elements involved in
the tests.
Varlashkin, et al. Expires April 22, 2012 [Page 4]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
At the time of writing of this document it is not known whether
existing network testers and protocol emulators are able to execute
described tests out of the box. Nevertheless the authors believe
that required functionality can be added with reasonable effort.
Alternatively the tests can be performed with help of physical
routers to create necessary test topology, which may have impact on
time required to perform the test but expected to provide same degree
of the test results accuracy. This also means that tests performed
using a protocol simulator can be repeated using physical routers and
results expected to be comparable.
This document complements draft-papneja-bgp-basic-dp-convergence.
2. Test topology
Unless specified otherwise all tests use same basic test topology
outlined below:
[R1]-----1----[R3]
/ \ / \
1 9 C2 \
/ \ / \
[S]---[DUT] [M1] [NetA]
\ / || \ /
3 C1 || 2 /
\ / / \ \ /
[R2] / \ [R4]
/ \
/ \
[ER1] ... [ERn]
| |
[NetB-1] [NetB-N]
S is source of test traffic for data-plane tests, while for control-
plane tests S is an emulated or physical router with packet capturing
(sniffing) capability.
Unidirectional test traffic goes from Source to NetA.
IGP between DUT and R1-R4; BGP between DUT and R3, R4; no BGP between
R3 and R4 (important). If tunnelling (e.g. MPLS or GRE) is used
then R1 and R2 do not need to run BGP, otherwise they MUST run BGP.
Source has static default to DUT; R3 and R4 have static to NetA.
NetA is in BGP but not in IGP. M1 is K*M matrix of internal routers.
Metrics C1 is used to control whether R2 is LFA for DUT to NetA.
Metric C2 is used to control whether R3 or R4 are best exit towards
NetA. All other metrics are fixed for all tests and MUST be set to
Varlashkin, et al. Expires April 22, 2012 [Page 5]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
exact values provided in the above diagram. IGP metrics from M1 to
ER1 throughout ERn can be set arbitrarily, their exact values are
irrelevant to this test as long as they're valid for given IGP.
Routers ER1 throughout ERn together with prefixes NetB-1 throughout
NetB-N are presented to create realistic environment but not used
directly in measurements. NetB-1 throughout NetB-N are distinct
single-prefix sets.
Traffic restoration depends on ability of R2 and M1 to forward
traffic after failure. To eliminate this dependency R2 is set to
always forward traffic to R3 and NetA via M1 which in turn always
forwards traffic directly via R3 or R2 depending on the test. One
possibility to achieve this is to use static routes. Another
alternative is to use different IGP between R2 and R3 from the one
used by DUT and make routes learned via this IGP preferred on R2.
E.g. DUT uses OSPF, then in addition to it R2&R3 also run ISIS and
prefer ISIS routes over OSPF ones. A protocol simulator can have
internal mechanism to provide required behaviour. There are no other
dependencies on non-DUT devices in this tests.
For evaluating eBGP performance following topology is used:
[R1]
/ \
/ \
/ \
[S]----[DUT] [NetA]
\ /
\ /
\ /
[R2]
Test topology for eBGP
In "Link failure without LoS" test direct cable between DUT and R1 is
replaced with connection over an L2 switch as follow:
[DUT]---[SW1]---[R1]
3. TEST PARAMETERS
Varlashkin, et al. Expires April 22, 2012 [Page 6]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
3.1. Packing ratios
Routes with different prefixes but same attributes can potentially be
packed into single update message. Since both number of update
messages and number of prefixes per update can affect convergence
time, the tests SHOULD be performed with various prefix packing
ratios. This document does not specify values of individual BGP
attributes used to control packing ratio.
3.2. Test traffic
Traffic is sent from single source address located at the Source port
of the tester to one address in each prefix in NetA set. Packets are
sent at rate 1000 per second, which provides 1ms resolution of the
convergence time as measured by tests in this document. All packets
SHOULD be 64 bytes at IP layer, that is IP header plus IP payload.
3.3. IGP metrics
Basic test topology specifies fixed IGP metrics for some links.
These metrics SHOULD be used verbatim. There are also two variable
metrics - C1 and C2 - intended for controlling whether R2 is Loop-
Free-Alternate (LFA) for DUT towards NetA, and whether R3 remains
best exit towards NetA after path failure between DUT and R3.
Following values SHOULD be used for C1 and C2 depending on required
behaviour:
+------------+----------+----+----+
| R2 is LFA? | R3 best? | C1 | C2 |
+------------+----------+----+----+
| yes | yes | 1 | 1 |
| yes | no | 1 | 3 |
| no | yes | 5 | 1 |
| no | no | 5 | 3 |
+------------+----------+----+----+
3.4. Internal routers matrix
Basic test topology has N*K grid of internal routers denoted as M1.
When N>1 or K>1 the cost of all links within grid MUST be set to 1
(one). This matrix is intended for controlling topology size, which
has affect on particularly SPF run-time.
If traffic is forwarded using a tunneling mechanism, such as MPLS or
GRE, the internal routers only need to have reachability information
about tunnel end-points. However if traditional hop-by-hop
forwarding is used, then internal routers MUST have routes to each
and every prefix within NetA set.
Varlashkin, et al. Expires April 22, 2012 [Page 7]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
This document does not specify how internal routers should obtain
necessary reachability information. The only requirement is that
after primary DUT-NetA path failure internal routers are able to
forward traffic to NetA instantly. Using values of IGP metrics as
described earlier addresses this requirement. Also, protocol
simulator may have built-in mechanism to achieve desired behaviour.
3.5. Number of next-hops
Basic test topology has set of N edge routers ER1 throughout ERn,
each advertising unique prefix. Some BGP implementations may exhibit
different performance depending on number of next-hops for which IGP
cost has changed after failure. By varying overall number of next-
hops such dependency can be detected.
Note that prefixes NetB-1 throughout NetB-n are not used as
destinations for test traffic, they're only present for creating
"background environment".
3.6. 'e' - Failure and Restoration start entropy
Tests described in this document use fixed time T2 and variable
offset 'e' as starting point for simulating failure or restoration
event.
Fixing time T2 is necessary as reference point to which variable
offset e is added for each iteration of the test. Introduction of
such variable offset allows better analysis of the test results. For
example, DUT may run FIB changes at certain intervals. If failure
introduced close to the end of such interval, shorter outage will be
observed, and if introduced close to the beginning of such interval
longer outage will be observed. Running test multiple times each
time using different offset will help to profile DUT better.
Test report must contain value of T2 (same for all iterations) and
values of e for each iterations. This document recommends to use
T2=T1+8s and e from 0 to 1s in 0.01s (10ms) increments.
4. TEST PROCEDURES
This section provides generic steps that are used in all tests.
4.1. Initialisation time
The objective of this test is to measure time that must elapse
between starting protocols and ability of the test topology to
forward traffic. This test is not intended to reflect DUT
Varlashkin, et al. Expires April 22, 2012 [Page 8]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
performance but used only as a way to find time T1 that is used in
all subsequent tests.
To execute test perform following steps:
1. Configure DUT and protocol simulator (or auxiliary nodes)
2. At T0 start traffic and then immediately start routeing protocols
3. When traffic starts arriving Sink Port 1 stop test.
The time of arrival of the first packet is T1.
4.2. Generic data-plane failure test
The purpose of failure test is to measure time required by DUT to
resume traffic flow after best path to destination fails. Following
steps are common for all failure tests:
1. Start protocols and mark time as T0
2. At time T1 start traffic to each prefix in set NetA
3. At T2+e simulate failure or restoration event (see Section 5)
4. From T2+e until T3 packets do not arrive to NetA
5. After packets are seen again at NetA (T3) wait until time T4
6. Stop traffic
7. Measure total number of lost packets and calculate outage knowing
packet-per-second
4.3. Generic test procedure for
1. At T0 bring up all interfaces and protocols, and start capturing
BGP packets at RS1
2. At T1+e simulate failure/restoration event (see Section 5)
3. At T2-d1 first UPDATE message is sent by DUT and at T2 it will be
observed at RS1
4. At T3-d2 last UPDATE message is sent by DUT and at T3 it will be
observed at RS1
d1 and d2 represent serialisation and propagation delay and can be
Varlashkin, et al. Expires April 22, 2012 [Page 9]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
disregarded unless DUT-RS1 link has large delay. With this in mind,
T2-(T1+e) and T3-(T1+e) represent convergence time for the first and
last prefix respectively.
5. Failure and restoration scenarios
This section defines set of various failure and restoration scenarios
used in step 3 of the generic test procedures described in previous
section. Unless otherwise specified all scenarios are applicable to
both data- and control-plane test procedures.
5.1. Loss of Signal on the link attached to DUT
This scenario simulates situation where link attached to DUT fails
and Loss of Signal (LoS) can be observed by DUT. In other words link
fails and results in interface on the DUT going down.
To simulate LoS failure at the time defined by the test procedure
shut down R1 side of the link to DUT.
To simulate LoS restoration at the time defined by the test procedure
re-activate R1 side of the link to DUT.
5.2. Link failure without LoS
This scenario simulates situation where link between DUT and adjacent
node fails but DUT does not observe LoS. In practice such failure
can occur when, for example, link between DUT and adjacent node is
implemented via carrier equipment that does not shut link down when
remote side of the link fails.
DUT can use various methods to detect such failures, including but
not limited to protocol HELLO or Keep-alive packets, BFD, OAM. This
document does not restrict methods which DUT can use, but requires
use of particular method to be recorded in the test report.
Basic network topology is modified for the purpose of this test only
as follow: rather than using direct cabling between DUT and R1 the
link is implemented via intermediate L2 switch that supports concept
of VLAN's. Initially switch ports connected to DUT and R1 are placed
into the same VLAN (same L2 broadcast domain).
To simulate failure at the time defined by the test procedure move
switch port connected to R1 to a VLAN different from the one used for
switch port connected to DUT.
To simulate restoration at the time defined by the test procedure
Varlashkin, et al. Expires April 22, 2012 [Page 10]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
move switch port connected to R1 back to the same VLAN as the one
used for switch port connected to DUT.
5.3. Non-direct link failure
This scenario simulates situation where a link not directly connected
to DUT but located on the primary path to destination fails.
Unmodified basic network topology is used.
Depending on technologies used in the setup different failure
detection techniques can be employed by DUT. This document assumes
that DUT relies exclusively on IGP information to learn about failure
and that nodes adjacent to the failed link flood this information
within D seconds since the event. If required exact value of D can
be obtained through simple additional test, but in this document D is
assumed to be 0 (zero).
It is possible, though undesirable, that some traffic and protocol
simulators may contunue accepting packets coming through the port
that leads to simulated failed link. It is essential to assert such
behaviour prior to the tests and if confirmed, exclude packets
received after failure from calculations in step 7 of the test.
Failure event is triggered by simulating shutdown of R3 side of the
link to R1 at the time defined by the test procedure. R1 MUST send
IGP update (depending on which protocol is used) to DUT within D
seconds.
Restoration event is triggered by simulating recovery of R3 side of
the link to R1 at the time defined by the test procedure. R1 MUST
send IGP update (depending on which protocol is used) to DUT within D
seconds.
5.4. Best route withdrawal
This scenario sumulates situation where best AS exit path to a
destination is no longer valid and ASBR sends BGP UPDATE to its iBGP
peers. Unmodified basic network topology is used.
Disconnecting R3 from NetA implies that R3 will send BGP WITHDRAW for
this prefixes in its update to DUT. It is possible, though
undesirable, that some protocol simulator and traffic generators will
still count packets received at sink port 1 even after prefixes were
withdrawn. To correctly execute this test it's mandatory that
traffic received at sink port 1 after withdrawing prefixes is ignored
and not counted as delivered. If traffic generator is not able to
assure such functionality (should be asserted prior to the test),
then packets received at the sink port 1 MUST be excluded from
Varlashkin, et al. Expires April 22, 2012 [Page 11]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
calculation in step 7 of the test.
Failure event is triggered by simulating failure of the link between
R3 and NetA and immediate withdrawal of all corresponding prefixes by
R3.
Restoration event is triggered by simulating recovery of the link
between R3 and NetA and immediate BGP UPDATE for all corresponding
prefixes by R3.
5.5. iBGP next-hop failure
This scenario simulates situation where ASBR used as best exit to a
destination unexpectedly fails both at control and forwarding plane.
Both R1 and a router within M1 connected to R3 MUST send appropriate
IGP update message to the rest of the network within D seconds. To
detect failure DUT MAY rely on IGP information provided by rest of
the network or it MAY employ additional techniques. This document
does not restrict what detection mechanism should DUT use but
requires that particular mechanism is recorded in the test report.
Failure event is triggered by simulating removal of R3 from the test
topology at the time defined by the test procedure, followed by IGP
update as described in previous paragraph.
Recovery event is triggered by re-introducing R3 into the test
topology, followed by IGP update as described in first paragraph of
this section and immediate re-activation of BGP session between R3
and DUT. Note that recovery time calculated by this method depends
on DUT performance in respect to bringing up new BGP session. This
is intentional. Control plane convergence benchmarking can be
performed separately by a method that is outside of the scope of this
document and two results can be correlated netto data-plane
convergence value should that be necessary.
6. Test report
TODO: Report format is to be discussed.
Test report MUST contain following data for each test:
1. T1 and 'e'
2. Number of prefixes NetA and NetB
3. Size of M1 (recored as N*K)
Varlashkin, et al. Expires April 22, 2012 [Page 12]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
4. Traffic rate, in packets per second, and packet size at IP layer
in octets
5. Number of lost packets during falure, and number of lost packets
during restoration
7. Link bundling and Equal Cost Multi-Path
Scenarios where DUT can balance traffic to NetA across multiple best
paths is explicitly excluded from scope of this document. There are
two reasons.
First, two different DUT may choose different path (out of all equal)
to forward given packet, which makes it unreasonably difficult to
define generic traffic that would produce comparable results when
testing different platforms.
Second, mechanisms used to handle failures in ECMP (but not
necessarily in link-bundling) environment are similar to those
handling single-path failures. Therefore it's expected that
convergence in ECMP scenario will be of the same order as in single-
path scenario.
8. Graceful Restart and Non-Stop Forwarding
While Graceful Restart and Non-Stop Forwarding mechanisms are related
to DUT ability to forward traffic under certain failure conditions,
the test covering DUT own ability to restore or preserve traffic flow
already covered in RFC6201.
9. Security considerations
The tests described in this document intended to be performed in
isolated lab environment, which inheretently has no security
implication on the live network of the organisation or Internet as
whole.
Authors foresee that some people or organisations might be interested
to benchmark performance of the live networks. The tests described
in this document are disruptive by their nature and will have impact
at least on the network where they're executed, and depending on the
role of that network effect can extend to other parts of the
Internet. Such tests MUST NOT be attempted in live environment
without careful consideration.
Varlashkin, et al. Expires April 22, 2012 [Page 13]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
The fact of publishing this document does not increase potential
negative consequences if tests are executed in live environment
because information provided here is mere recording of widely known
and used techniques.
10. IANA Considerations
None.
11. Acknowledgments
Authors would like to thank Gregory Cauchie, Rob Shakir, David
Freedman, Anton Elita, Saku Ytti, Andrew Yourtchenko, for their
valuable contribution and peer-review of this work.
12. Normative References
[RFC4760] Bates, T., Chandra, R., Katz, D., and Y. Rekhter,
"Multiprotocol Extensions for BGP-4", RFC 4760,
January 2007.
Authors' Addresses
Ilya Varlashkin
Easynet Global Services
Email: ilya.varlashkin@easynet.com
Rajiv Papneja
Huawei Technologies (USA)
Email: rajiv.papneja@huawei.com
Bhavani Parise
Cisco
Email: bhavani@cisco.com
Varlashkin, et al. Expires April 22, 2012 [Page 14]
Internet-Draft draft-varlashkin-router-conv-bench-00 October 2011
Tara Van Unen
Ixia
Email: TVanUnen@ixiacom.com
Varlashkin, et al. Expires April 22, 2012 [Page 15]
