Internet Engineering Task Force S. Jacob, Ed. Internet-Draft K. Tiruveedhula Intended status: Standards Track Juniper Networks Expires: May 30, 2019 November 26, 2018 Benchmarking Methodology for EVPN and PBB-EVPN draft-ietf-bmwg-evpntest-01 Abstract This document defines methodologies for benchmarking EVPN and PBB- EVPN performance. EVPN is defined in RFC 7432, and is being deployed in Service Provider networks. Specifically this document defines the methodologies for benchmarking EVPN/PBB-EVPN convergence, data plane performance, and control plane performance. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on May 30, 2019. Copyright Notice Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Jacob & Tiruveedhula Expires May 30, 2019 [Page 1] Internet-Draft EVPN Benchmarking Methodology November 2018 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3 2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.1. How long it takes to learn local mac address in EVPN . . 8 3.2. How long it takes to learn local mac address in PBB EVPN 9 3.3. How long it takes to learn the remote macs . . . . . . . 9 3.4. PBB-EVPN How long it takes to learn the mac from remote peer . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.5. How long it takes to flush the local macs due to CE link flap and measure the relearning rate of MACs . . . . . . 11 3.6. PBB-EVPN how long it takes to flush the local macs and measure the relearning rate of macs during PE-CE link flap . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.7. How long it takes to flush the remote macs, due to remote link failure. . . . . . . . . . . . . . . . . . . . . . . 12 3.8. PBB-EVPN How long it takes to flush the remote macs due to remote link failure . . . . . . . . . . . . . . . . . 13 3.9. To measure the MAC aging time. . . . . . . . . . . . . . 14 3.10. PBB-EVPN To measure the MAC aging time. . . . . . . . . . 15 3.11. How long it takes to age out the remote macs . . . . . . 15 3.12. PBB-EVPN How long it takes to age out the remote macs. . 16 3.13. How long it takes to learn both local and remote macs. . 17 3.14. PBB-EVPN How long it takes to learn both local and remote macs . . . . . . . . . . . . . . . . . . . . . . . . . . 17 4. High Availability . . . . . . . . . . . . . . . . . . . . . . 18 4.1. To Record the whether there is traffic loss due to routing engine failover for redundancy test. . . . . . . 18 4.2. PBB-EVPN To Record the whether there is traffic loss due to routing engine failover for redundancy test . . . . . 19 5. ARP/ND Scale And Prefix Scale . . . . . . . . . . . . . . . . 19 5.1. To find ARP/ND scale . . . . . . . . . . . . . . . . . . 19 5.2. To find the prefix( type 5 route) scale . . . . . . . . . 20 6. Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 6.1. To Measure the scale limit of DUT with trigger (Scale without traffic) . . . . . . . . . . . . . . . . . . . . 20 6.2. PBB-EVPN To measure the scale limit with trigger. . . . . 21 6.3. To measure the convergence time of DUT with scale and traffic. . . . . . . . . . . . . . . . . . . . . . . . . 21 6.4. .PBB-EVPN To measure the convergence time of DUT with scale and traffic. . . . . . . . . . . . . . . . . . . . 22 7. SOAK Test . . . . . . . . . . . . . . . . . . . . . . . . . . 23 7.1. To Measure the stability of the DUT with scale and traffic. . . . . . . . . . . . . . . . . . . . . . . . . 23 7.2. PBB-EVPN to measure the stability of DUT with scale and Jacob & Tiruveedhula Expires May 30, 2019 [Page 2] Internet-Draft EVPN Benchmarking Methodology November 2018 traffic. . . . . . . . . . . . . . . . . . . . . . . . . 23 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 24 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24 10. Security Considerations . . . . . . . . . . . . . . . . . . . 24 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 24 11.1. Normative References . . . . . . . . . . . . . . . . . . 24 11.2. Informative References . . . . . . . . . . . . . . . . . 24 Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 25 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25 1. Introduction EVPN is defined in RFC 7432, and describes BGP MPLS- based Ethernet VPNs (EVPN). PBB-EVPN is defined in RFC 7623, discusses how Ethernet Provider backbone Bridging can be combined with EVPNs to provide a new/combined solution. This draft defines methodologies that can be used to benchmark both RFC 7432 and RFC 7623 solutions. Further, this draft provides methodologies for benchmarking the performance of EVPN data and control planes, MAC learning, MAC flushing, MAC ageing, convergence, high availability, and scale. 1.1. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 1.2. Terminologies MHPE Multi homed Provide Edge router. RR Route Reflector. P Provider Router. CE Customer Router/Devices/Switch. MHPE2 Multi homed Provider Edge router 2. MHPE1 Multi homed Provider Edge router 1. SHPE3 Single homed Provider Edge Router 3. AA EVPN Terminologies AA All-Active. SA EVPN Terminologies SA Single-Active. RT Router Tester. Jacob & Tiruveedhula Expires May 30, 2019 [Page 3] Internet-Draft EVPN Benchmarking Methodology November 2018 Sub Interface Each physical Interfaces is subdivided in to Logical units. EVI EVPN Instances which will be running on sub interface or physical port of the provider Edge routers. DF Designated Forwarder. ESI Ethernet Segment Identifier. 2. Test Topology EVPN/PBB-EVPN Services running on SHPE3, MHPE1 and MHPE2 in Single Active Mode: Topology Diagram | [Traffic Generator ] Router Tester traffic receiver for layer 2 traffic from CE +----------+ | | | SHPE3 | | SHPE3 | +----------+ | |Core link +----------+ | | | RR | | | Route Reflector/Core router +----------+-------------| | | | Core links | +----------+ +-----------+ | | | MHPE2 | | DUT | | | | MHPE1 | | | +----------+ +-----------+ | PE-CE link | +----------+------------ | | | CE | | layer2 | |bridge | +----------+------------ [Traffic Generator](Router Tester sending layer 2 traffic with different VLAN ) Topology 1 Jacob & Tiruveedhula Expires May 30, 2019 [Page 4] Internet-Draft EVPN Benchmarking Methodology November 2018 | [Traffic Generator ] Router Tester sending layer 2 traffic. +----------+ | | | SHPE3 | | SHPE3 | +----------+ | |Core link +----------+ | | | RR | | | Route Reflector/Core router +----------+-------------| | | | Core links | +----------+ +-----------+ | | | MHPE2 | | DUT | | | | MHPE1 | | | +----------+ +-----------+ | PE-CE link | +----------+------------ | | | CE | | layer2 | |bridge | +----------+------------ [Traffic Generator](Router Tester receiver for layer 2 traffic with different vlans.) Topology 2 Jacob & Tiruveedhula Expires May 30, 2019 [Page 5] Internet-Draft EVPN Benchmarking Methodology November 2018 | [Traffic Generator ] Router Tester sending layer 2 bi directional traffic sender/receiver +----------+ | | | SHPE3 | | SHPE3 | +----------+ | |Core link +----------+ | | | RR | | | Route Reflector/Core router +----------+-------------| | | | Core links | +----------+ +-----------+ | | | MHPE2 | | DUT | | | | MHPE1 | | | +----------+ +-----------+ | PE-CE link | +----------+------------ | | | CE | | layer2 | |bridge | +----------+------------ [Traffic Generator](Router Tester sending bi directional layer 2 traffic with different VLAN sender/receiver) Topology 3 | Traffic generator ( no traffic) +----------+ | | | SHPE3 | | SHPE3 | Jacob & Tiruveedhula Expires May 30, 2019 [Page 6] Internet-Draft EVPN Benchmarking Methodology November 2018 +----------+ | |Core link +----------+ | | | RR | | | Route Reflector/Core router +----------+-------------| | | | Core links | +----------+ +-----------+ | | | MHPE2 | | DUT | | | | MHPE1 | | | +----------+ +-----------+ | | Traffic generator (no traffic) Traffic generator sending prefixes to DUT Topology 4 There are five routers in the topology. SHPE3, RR/P, MHPE1 and MHPE2 emulating a service provider network. CE is a customer device connected to MHPE1 and MHPE2, it is configured with bridge domains in different vlans. The router tester is connected to CE and SHPE3.The MHPE1 acts DUT.The RT will act as sender and receiver.The measurement will be taken in DUT. All routers except CE is configured with OSPF/IS-IS,LDP,MPLS,BGP with EVPN address family. All routers except CE must have IBGP configured with RR acting as route reflector. MHPE1,MHPE2,SHPE3 must be configured with "N" EVPN/PBB-EVPN instances depends up on the cases. MHPE1 and MHEPE2 must be configured with ESI per vlan or ESI on IFD. Jacob & Tiruveedhula Expires May 30, 2019 [Page 7] Internet-Draft EVPN Benchmarking Methodology November 2018 MHPE1 and MHEPE2 are running Single Active mode of EVPN. CE is acting as bridge configured with vlans that is configured on MHPE1,MHPE2,SHPE3. Depends up on the test traffic will be flowing uni directional or bi directional depends on the topology mentioned above. The above configuration will serve as base configuration for all the test cases. 3. Test Cases The following tests are conducted to measure the time taken to learn the "X" number of MAC's locally in EVI . The data plane learning of MAC will happen locally from connected interface. The control plane learning of MAC is through BGP advertisements from the remote PE(SHPE3). The control plane learning of "X" MAC. The data plane MAC learning can be measured using the parameters defined in RFC 2889 section 5.8. 3.1. How long it takes to learn local mac address in EVPN Objective: To Record the time taken to learn the MAC address locally in DUT. Topology : Topology 1 Procedure: Send "X" unicast frames from CE to MHPE1(DUT) working in SA mode with "X" different source and destination address from RT. The DUT must learn these "X" macs in data plane. After measuring the time taken to learn the macs, stop the traffic. Clear the mac table, then increase the scale of "X" by 10%. repeat the above procedure. After each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Measure the time taken to learn "X" MACs in DUT evpn mac table. The data plane measurement is taken by considering DUT as black box the range of X MAC is known from RT and the same must be learned in DUT, the time taken to learn "X" macs is measured. The same procedure must be used for increased scale. Jacob & Tiruveedhula Expires May 30, 2019 [Page 8] Internet-Draft EVPN Benchmarking Methodology November 2018 Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The mac learning time is calculated by averaging the values obtained from "N" samples. Mac learning rate in sec for "X" macs = (T1+T2+..Tn/N) Mac learning rate in sec for "X+10%" macs = (T1+T2+..Tn/N) 3.2. How long it takes to learn local mac address in PBB EVPN Objective: To Record the time taken to learn the MAC address locally. Topology : Topology 1 Procedure: Send "X" unicast frames from CE to MHPE1(DUT) working in SA mode with "X" different source and destination address from RT. The DUT must learn "X" macs in data plane.After measuring the time taken to learn the macs,stop the traffic and then clear mac table. Then increase the scale of "X" by 10%.repeat the above procedure. After each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Measure the time taken by the DUT to learn the "X" MACs in the data plane. The data plane measurement is taken by considering DUT as black box the range of "X" MAC is known from RT and the same must be learned in DUT, the time taken to learn "X" MAC is measured. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The mac learning time is calculated by averaging the values obtained from "N" samples. The same process is repeated for increased scale. Mac learning rate in for "X" mac's in sec = (T1+T2+..Tn/N) Mac learning rate for "X+10%" in sec = (T1+T2+..Tn/N) 3.3. How long it takes to learn the remote macs Objective: To Record the time taken to learn the remote macs. Jacob & Tiruveedhula Expires May 30, 2019 [Page 9] Internet-Draft EVPN Benchmarking Methodology November 2018 Topology : Topology 2 Procedure: Send X frames with X different SA and DA to SHPE3 from RT. SHPE3 will advertise these locally learned macs to MHPE1 and MHPE2 via control plane.Measure the time taken to learn these X MACs from remote peer in DUT EVPN MAC address table.The DUT and MHPE2 are running SA mode. After measuring the time taken to learn the macs.stop the traffic and then clear mac table.Then increase the scale of "X" by 10%. repeat the above procedure. After each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Measure the time taken by the DUT to learn the "X" MACs in the data plane.Repeat these test and plot the data.The test is repeated for "N" times and the values are collected.The mac learning time is calculated by averaging the values obtained from "N" samples.The same process is repeated for increased scale. Mac learning rate for "X" remote macs in sec = (T1+T2+..Tn/N) Mac learning rate for "X+10%" remote macs in sec = (T1+T2+..Tn/N) 3.4. PBB-EVPN How long it takes to learn the mac from remote peer Objective: To Record the time taken to learn the remote macs. Topology : Topology 2 Procedure: Send X frames with X different SA and DA to SHPE3 from RT.These macs will be flooded to MHPE1 and MHPE2 by SHPE3.The DUT and MHPE2 are running SA mode.After measuring the time taken to learn the macs. Stop the traffic and then clear mac table. Then increase the scale of "X" by 10%.repeat the above procedure. After each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Jacob & Tiruveedhula Expires May 30, 2019 [Page 10] Internet-Draft EVPN Benchmarking Methodology November 2018 Measure the time taken to learn X mac address in DUT mac table. Repeat these test and plot the data.The test is repeated for "N" times and the values are collected.The mac learning time is calculated by averaging the values obtained by "N" samples. The same process is repeated for increased scale. Mac learning rate for "X" remote macs in sec = (T1+T2+..Tn/N) Mac learning rate for "X+10%" remote macs in sec = (T1+T2+..Tn/N) 3.5. How long it takes to flush the local macs due to CE link flap and measure the relearning rate of MACs Objective: To record the time taken to flush the mac learned locally and the time taken to relearn the same amount of macs. Topology : Topology 1 Procedure: Send X frames with X different SA and DA to DUT from CE using traffic generator. Wait till the MHPE1 learns all X MAC address. Then fail the MHPE1 CE link and measure the time taken to flush these X MACs from the EVPN MAC table. Bring up the link which was made Down(the link between MHPE1 and CE).Measure time taken to relearn it. The DUT and MHPE2 are running SA mode.After measuring the time taken to re learn the macs.Stop the traffic and then clear mac table. Then increase the scale of "X" by 10%.repeat the above procedure. After each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Measure the time taken for flushing these X MAC address. Measure the time taken to relearn the X MACs in DUT. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The flush and the relearning time is calculated by averaging the values obtained by "N" samples. The same process is repeated for increased scale. Flush time for X Macs in sec = (T1+T2+..Tn/N) Relearning time for X macs in sec = (T1+T2+..Tn/N) Flush time for X+10% Macs in sec = (T1+T2+..Tn/N) Jacob & Tiruveedhula Expires May 30, 2019 [Page 11] Internet-Draft EVPN Benchmarking Methodology November 2018 Relearning time for X+10% macs in sec = (T1+T2+..Tn/N) 3.6. PBB-EVPN how long it takes to flush the local macs and measure the relearning rate of macs during PE-CE link flap Objective: To record the time taken to flush the mac learned locally and the time taken to relearn the same amount of macs. Topology : Topology 1 Procedure: Send X frames with X different SA and DA to DUT from CE using traffic generator. Wait till the MHPE1 learn all X MAC address. Then fail the MHPE1 CE link and measure the time taken to flush these X MACs from the PBB-EVPN MAC table. Then bring up the link. Measure the time taken to relearn X MACS. The DUT and MHPE2 are running SA mode. After measuring the time taken to re learn the macs.Stop the traffic and then clear mac table.Then increase the scale of "X" by 10%. repeat the above procedure. After each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Measure the time taken for flushing these X MAC address. Measure the time taken to relearn the X MACs in DUT. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The flush and the relearning time is calculated by averaging the values obtained by "N" samples.The same process is repeated for increased scale. Flush time for X Macs in sec = (T1+T2+..Tn/N) Relearning time for X macs in sec = (T1+T2+..Tn/N) Flush time for X+10% Macs in sec = (T1+T2+..Tn/N) Relearning time for X+10% macs in sec = (T1+T2+..Tn/N) 3.7. How long it takes to flush the remote macs, due to remote link failure. Objective: Jacob & Tiruveedhula Expires May 30, 2019 [Page 12] Internet-Draft EVPN Benchmarking Methodology November 2018 To record the time taken to flush the remote mac learned in DUT during remote link failure. Topology : Topology 2 Procedure: Send X frames with X different SA and DA to DUT from SHPE3 using traffic generator. Bring down the link between SHPE3 and traffic generator. Then measure the time taken to flush the DUT EVPN MAC table. The DUT and MHPE2 are running SA mode.Stop the traffic and then clear mac table.Then increase the scale of "X" by 10%. repeat the above procedure.After each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Measure the time taken to flush X remote MACs from EVPN MAC table of DUT. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The flush rate is calculated averaging the values obtained by "N" samples.The same process is repeated for increased scale. Flush time for X Macs in sec = (T1+T2+..Tn/N) Flush time for X+10% Macs in sec = (T1+T2+..Tn/N) 3.8. PBB-EVPN How long it takes to flush the remote macs due to remote link failure Objective: To record the time taken to flush the remote mac learned in DUT during remote link failure. Topology : Topology 2 Procedure: Send X frames with X different SA and DA to DUT from SHPE3 using traffic generator. Bring down the link between SHPE3 and traffic generator. Then measure the time taken to flush the DUT PBB-EVPN MAC address table. The remote MACs will be learned by Data plane, but the B-MAC will be learned by control plane. The DUT and MHPE2 are running SA mode.Stop the traffic and then clear mac table.Then increase the scale of "X" by 10%.repeat the above procedure. After Jacob & Tiruveedhula Expires May 30, 2019 [Page 13] Internet-Draft EVPN Benchmarking Methodology November 2018 each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Measure the time taken to flush X remote MACs from PBB-EVPN MAC table of DUT. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The flush rate is calculated by averaging the values obtained by "N" samples. The same process is repeated for increased scale. Flush time for X Macs in sec = (T1+T2+..Tn/N) Flush time for X+10% Macs in sec = (T1+T2+..Tn/N) 3.9. To measure the MAC aging time. Objective: To measure the mac aging time. Topology : Topology 1 Procedure: Send X frames with X different SA and DA to DUT from CE using traffic generator. Wait till X MAC address are learned. Then stop the traffic. Record the time taken to flush X MACS from DUT EVPN MAC table due to aging. The DUT and MHPE2 are running SA mode. Then increase the scale of "X" by 10%.repeat the above procedure. After each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Measure the time taken to flush X MAC address due to aging. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The aging is calculated averaging the values obtained by "N" samples.The same process is repeated for increased scale. Aging time for X Macs in sec = (T1+T2+..Tn/N) Aging time for X+10% Macs in sec = (T1+T2+..Tn/N) Jacob & Tiruveedhula Expires May 30, 2019 [Page 14] Internet-Draft EVPN Benchmarking Methodology November 2018 3.10. PBB-EVPN To measure the MAC aging time. Objective: To measure the mac aging time. Topology : Topology 1 Procedure: Send X frames with X different SA and DA to DUT from CE using traffic generator. Wait till X MAC address are learned in DUT PBB- EVPN MAC table. Then stop the traffic. Record the time taken to flush X MAC entries due to aging. The DUT and MHPE2 running in SA mode. Then increase the scale of "X" by 10%.repeat the above procedure. After each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Measure the time taken to flush X MAC address due to aging. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The aging is calculated by averaging the values obtained by "N" samples.The same process is repeated for increased scale. Aging time for X Macs in sec = (T1+T2+..Tn/N) Aging time for X+10% Macs in sec = (T1+T2+..Tn/N) 3.11. How long it takes to age out the remote macs Objective: To measure the remote mac aging time. Topology : Topology 2 Procedure: Send X frames with X different SA and DA to DUT from SHPE3 using traffic generator. Stop the traffic at remote PE SHPE3.Due to MAC aging SHPE3 will withdraw its routes from DUT and MHPE2. Measure the time taken to remove these MACs from DUT EVPN MAC table. DUT and MHPE2 are running in SA mode.Then increase the scale of "X" by Jacob & Tiruveedhula Expires May 30, 2019 [Page 15] Internet-Draft EVPN Benchmarking Methodology November 2018 10%.repeat the above procedure. After each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Measure the time taken to flush X remote MACs learned in DUT EVPN MAC table due to aging. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The aging is calculated by averaging the values obtained by "N" samples. the same process is repeated for increased scale. Aging time for X Macs in sec = (T1+T2+..Tn/N) Aging time for X+10% Macs in sec = (T1+T2+..Tn/N) 3.12. PBB-EVPN How long it takes to age out the remote macs. Objective: To measure the remote mac aging time. Topology : Topology 2 Procedure: Send X frames with X different SA and DA to DUT from SHPE3 using traffic generator. Stop the traffic at remote PE(SHPE3).Measure the time taken to remove these remote MACs from DUT PBB-EVPN MAC table. The DUT and MHPE2 are running in SA mode.Then increase the scale of "X" by 10%.repeat the above procedure. After each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Measure the time taken to flush the X remote MACs from DUT PBB-EVPN MAC table due to aging Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The aging is calculated by averaging the values obtained by "N" samples. The same process is repeated for increased scale. Aging time for X Macs in sec = (T1+T2+..Tn/N) Aging time for X+10% Macs in sec = (T1+T2+..Tn/N) Jacob & Tiruveedhula Expires May 30, 2019 [Page 16] Internet-Draft EVPN Benchmarking Methodology November 2018 3.13. How long it takes to learn both local and remote macs. Objective: To record the time taken to learn both local and remote macs. Topology : Topology 3 Procedure: Send X frames with X different SA and DA to DUT from SHPE3 using traffic generator. Send X frames with different SA and DA from traffic generator connected to CE. The SA and DA of flows must be complimentary to have unicast flows. Measure the time taken by the DUT to learn 2X in EVPN MAC. DUT and MHPE2 are running in SA mode. Stop the traffic, clear the mac table.Then increase the scale of "X" by 10%.repeat the above procedure. After each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Measure the time taken to learn 2X MAC address in DUT EVPN MAC table. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The mac learning time is calculated by averaging the values obtained by "N" samples.The same process is repeated for increased scale. Time to learn 2X Macs in sec = (T1+T2+..Tn/N) Time to learn 2(X+10%) Macs in sec = (T1+T2+..Tn/N) 3.14. PBB-EVPN How long it takes to learn both local and remote macs Objective: To record the time taken to learn both local and remote macs. Topology : Topology 3 Procedure: Send X frames with X different SA and DA to DUT from SHPE3 using traffic generator. Send X frames with different SA and DA from traffic generator connected to CE. The SA and DA of flows must be complimentary to have unicast flows. Measure the time taken by the DUT to learn 2X in MAC table. DUT and MHPE2 are running in SA mode. Jacob & Tiruveedhula Expires May 30, 2019 [Page 17] Internet-Draft EVPN Benchmarking Methodology November 2018 Stop the traffic, clear the mac table.Then increase the scale of "X" by 10%.repeat the above procedure. After each iteration the scale must be increased by 10% till the limit of the DUT is reached. Measurement : Measure the time taken to learn 2X MAC address table in DUT PBB-EVPN MAC table. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The mac learning time is calculated by averaging the values obtained by "N" samples.The same process must be repeated for increased scale. Time to learn 2X Macs in sec = (T1+T2+..Tn/N) Time to learn 2(X+10%) Macs in sec = (T1+T2+..Tn/N) 4. High Availability 4.1. To Record the whether there is traffic loss due to routing engine failover for redundancy test. Objective: To record traffic loss during routing engine failover. Topology : Topology 3 Procedure: Send X frames from CE to DUT from traffic generator withX different SA and DA. Send X frames from traffic generator to SHPE3 with X different SA and DA so that 2X MAC address will be learned in DUT. There is a bi directional traffic flow with X pps in each direction. Then do a routing engine fail-over. Measurement : There should be 0 traffic loss which is the ideal case, No change in the DF role. DUT should not withdraw any routes.Repeat the test "N" times and plot the data.The packet loss is calculated by averaging the values obtained from "N" samples. Packet loss in sec = (T1+T2+..Tn/N) Jacob & Tiruveedhula Expires May 30, 2019 [Page 18] Internet-Draft EVPN Benchmarking Methodology November 2018 4.2. PBB-EVPN To Record the whether there is traffic loss due to routing engine failover for redundancy test Objective: To record traffic loss during routing engine failover. Topology : Topology 3 Procedure: Send X frames to DUT with X different SA and DA from CE using the traffic generator. Send X frames from traffic generator to SHPE3 with X different SA and DA so that 2X MAC address will be Learned in DUT. There is a bi directional traffic flow with X pps in each direction. Then do a routing engine fail-over. Measurement : There should be 0 traffic loss which is the ideal case, No change in the DF role. DUT should not withdraw any routes.Repeat the test "N" times and plot the data.The packet loss is calculated by averaging the values obtained from "N" samples. Packet loss in sec = (T1+T2+..Tn/N) 5. ARP/ND Scale And Prefix Scale These tests are conducted to Record the scaling parameter of ARP/ND of the DUT. 5.1. To find ARP/ND scale Objective: To Record the ARP/ND scale of the DUT. Topology : Topology 1 Procedure: Send X arp/icmpv6 request from RT to DUT with different sender ip/ ipv6 address to the same target gateway ip address. Measure whether X MAC+IPv4 address/MAC+IPv6 address of the hosts are learned in DUT. Increase the scale by 10 percent, then measure the DUT in order to find the new scale is reached.continue till the limit of the DUT. Jacob & Tiruveedhula Expires May 30, 2019 [Page 19] Internet-Draft EVPN Benchmarking Methodology November 2018 that is DUT is no longer learn the arp/ND generated by the traffic generator. Measurement : The DUT must learn the arp and ND and it must advertise mac+ip/ mac+ipv6 to the remote PE's. Scale value is calculated based on the maximum number if mac+ip/mac+ipv6 is learned beyond this number DUT cant learn. The test is repeated "N" times and the average value is taken as the scale limit. 5.2. To find the prefix( type 5 route) scale Objective: To Record the Prefix scale limit of the DUT Topology : Topology 4 Procedure: Send X Prefix to the DUT. DUT must learn the X prefixes and advertise as type 5 route to the remote router. Increase the scale by 10 percent, then measure the DUT in order to find the new scale is reached. continue till the limit of the DUT is reached, that is DUT is no longer learn the prefixes which is generated by traffic generator. Measurement : The test is carried to find out the prefix scale of the DUT. The test is repeated "N" times. The final scale value will be the average of "N" samples. 6. Scale This is to measure the performance of DUT in scaling to "X" EVPN instances. The measured parameters are CPU usage, memory leak,crashes. 6.1. To Measure the scale limit of DUT with trigger (Scale without traffic) Objective: To measure the scale limit of DUT for EVPN. Jacob & Tiruveedhula Expires May 30, 2019 [Page 20] Internet-Draft EVPN Benchmarking Methodology November 2018 Topology : Topology 3 Procedure: The DUT,MHPE2 and SHPE3 are scaled to "N" EVI.Clear BGP neighbors of the DUT. Once adjacency is established in the DUT. Measure the routes received from MHPE2 and SHPE3 for "N" EVI in the DUT. Measurement : There should not be any loss of route types 1,2,3 and 4 in DUT. DUT must relearn all type 1,2,3 and 4 from remote routers. The DUT must be subjected to various values of N to find the optimal scale limit 6.2. PBB-EVPN To measure the scale limit with trigger. Objective: To measure the scale limit of DUT for PBB-EVPN. Topology : Topology 3 Procedure: The DUT,MHPE2 and SHPE3 are scaled to "N" PBB-EVPN instances. Clear BGP neighbors in the DUT Once adjacency is established in DUT, check routes received from SHPE3 and MHPE2. Measurement : There should not be any loss of route types 2,3 and 4 in DUT. The DUT must relearn all type 2,3 and 4 routes from remote routers. The DUT must be subjected to various values of N to find the optimal scale limit. 6.3. To measure the convergence time of DUT with scale and traffic. Objective: To measure the convergence time of DUT when the DUT is scaled with EVPN instance along with traffic. Topology : Topology 3 Procedure: Jacob & Tiruveedhula Expires May 30, 2019 [Page 21] Internet-Draft EVPN Benchmarking Methodology November 2018 Scale N EVIs in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE using traffic generator with X different SA and DA for N EVI's. Send F frames from traffic generator to SHPE3 with X different SA and DA. There will be 2X number of MAC address will be learned in DUT EVPN MAC table. There is a bi directional traffic flow with F pps in each direction. Then clear the BGP neighbors in the DUT. Once the adjacency is restored in DUT. Measure the time taken to learn 2X MAC address in DUT MAC table. Measurement : The DUT must learn 2X MAC address. Measure the time taken to learn 2X MAC in DUT. Repeat these test and plot the data.The test is repeated for "N" times and the values are collected.The convergence time is calculated by averaging the values obtained by "N" samples. Convergence time in sec = (T1+T2+..Tn/N) 6.4. .PBB-EVPN To measure the convergence time of DUT with scale and traffic. Objective: To measure the convergence time of DUT when the DUT is scaled with PBB-EVPN instance along with traffic. Topology : Topology 3 Procedure: Scale N PBB-EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE using traffic generator with X different SA and DA for N EVI's. Send F frames from traffic generator to SHPE3 with X different SA and DA. There will be 2X number of MAC address will be learned in DUT PBB- EVPN MAC table. There is a bi directional traffic flow with F pps in each direction. Then clear the BGP neighbors in the DUT. Once the adjacency is restored in DUT. Measure the time taken to learn 2X MAC address in DUT PBB-MAC table. Measurement : The DUT must learn 2X MAC address. Measure the time taken to learn 2X MAC in DUT. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The convergence time is calculated by averaging the values obtained by "N" samples. Convergence time in sec = (T1+T2+..Tn/N) Jacob & Tiruveedhula Expires May 30, 2019 [Page 22] Internet-Draft EVPN Benchmarking Methodology November 2018 7. SOAK Test This is measuring the performance of DUT running with scaled configuration with traffic over a peroid of time "T'". In each interval "t1" the parameters measured are CPU usage, memory usage, crashes. 7.1. To Measure the stability of the DUT with scale and traffic. Objective: To measure the stability of the DUT in a scaled environment with traffic. Topology : Topology 3 Procedure: Scale N EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE using traffic generator with different X SA and DA for N EVI's. Send F frames from traffic generator to SHPE3 with X different SA and DA. There will be 2X number of MAC address will be learned in DUT EVPN MAC table. There is a bi directional traffic flow with F pps in each direction. The DUT must run with traffic for 24 hours, every hour check for memory leak, crash. Measurement : Take the hourly reading of CPU, process memory. There should not be any leak, crashes, CPU spikes. 7.2. PBB-EVPN to measure the stability of DUT with scale and traffic. Objective: To measure the stability of the DUT in a scaled environment with traffic. Topology : Topology 3 Procedure: Scale N PBB-EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE using traffic generator with X different SA and DA for N EVI's. Send F frames from traffic generator to SHPE3 with X different SA and DA. There will be 2X number of MAC address will be learned in DUT PBB- Jacob & Tiruveedhula Expires May 30, 2019 [Page 23] Internet-Draft EVPN Benchmarking Methodology November 2018 EVPN MAC table. There is a bi directional traffic flow with F pps in Each direction. The DUT must run with traffic for 24 hours, every hour check the memory leak, crashes. Measurement : Take the hourly reading of CPU process, memory usages. There should not be any memory leak, crashes,CPU spikes. 8. Acknowledgements We would like to thank Fioccola Giuseppe of Telecom Italia reviewing our draft and commenting it. We would like to thank Sarah Banks for guiding and mentoring us. 9. IANA Considerations This memo includes no request to IANA. 10. Security Considerations There is no additional consideration from RFC 6192. 11. References 11.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, DOI 10.17487/RFC2544, March 1999, . [RFC2899] Ginoza, S., "Request for Comments Summary RFC Numbers 2800-2899", RFC 2899, DOI 10.17487/RFC2899, May 2001, . 11.2. Informative References [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015, . Jacob & Tiruveedhula Expires May 30, 2019 [Page 24] Internet-Draft EVPN Benchmarking Methodology November 2018 [RFC7623] Sajassi, A., Ed., Salam, S., Bitar, N., Isaac, A., and W. Henderickx, "Provider Backbone Bridging Combined with Ethernet VPN (PBB-EVPN)", RFC 7623, DOI 10.17487/RFC7623, September 2015, . Appendix A. Appendix Authors' Addresses Sudhin Jacob (editor) Juniper Networks Bangalore India Phone: +91 8061212543 Email: sjacob@juniper.net Kishore Tiruveedhula Juniper Networks 10 Technology Park Dr Westford, MA 01886 USA Phone: +1 9785898861 Email: kishoret@juniper.net Jacob & Tiruveedhula Expires May 30, 2019 [Page 25]