Internet Engineering Task Force S. Jacob, Ed. Internet-Draft K. Tiruveedhula Intended status: Informational Juniper Networks Expires: June 20, 2020 December 18, 2019 Benchmarking Methodology for EVPN and PBB-EVPN draft-ietf-bmwg-evpntest-04 Abstract This document defines methodologies for benchmarking EVPN and PBB- EVPN performance.EVPN is defined in RFC 7432, and is being deployed in Service Provider networks.Specifically, this document defines the methodologies for benchmarking EVPN/PBB-EVPN convergence, data plane and control plane performance. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on June 20, 2020. Copyright Notice Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Jacob & Tiruveedhula Expires June 20, 2020 [Page 1] Internet-Draft EVPN Benchmarking Methodology December 2019 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3 2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Test Cases for EVPN Benchmarking . . . . . . . . . . . . . . 7 3.1. Local MAC Learning . . . . . . . . . . . . . . . . . . . 7 3.2. Remote MAC Learning . . . . . . . . . . . . . . . . . . . 8 3.3. MAC Flush due to local link failure and Relearning . . . 8 3.4. MAC Flush due to remote link failure. . . . . . . . . . . 9 3.5. MAC Aging . . . . . . . . . . . . . . . . . . . . . . . . 9 3.6. Remote Mac Aging . . . . . . . . . . . . . . . . . . . . 10 3.7. Local and Remote MAC Learning . . . . . . . . . . . . . . 10 3.8. High Availability. . . . . . . . . . . . . . . . . . . . 11 3.9. ARP/ND Scale . . . . . . . . . . . . . . . . . . . . . . 12 3.10. Scaling of Services . . . . . . . . . . . . . . . . . . . 12 3.11. Scale Convergence . . . . . . . . . . . . . . . . . . . . 13 3.12. SOAK Test. . . . . . . . . . . . . . . . . . . . . . . . 14 4. Test Cases for PBB-EVPN Benchmarking . . . . . . . . . . . . 14 4.1. Local MAC Learning . . . . . . . . . . . . . . . . . . . 14 4.2. Remote Mac Learning . . . . . . . . . . . . . . . . . . . 15 4.3. MAC Flush due to link failure . . . . . . . . . . . . . . 15 4.4. MAC Flush due to remote Failure . . . . . . . . . . . . . 16 4.5. MAC aging . . . . . . . . . . . . . . . . . . . . . . . . 17 4.6. Remote MAC Aging. . . . . . . . . . . . . . . . . . . . . 17 4.7. Local and Remote MAC Learning . . . . . . . . . . . . . . 18 4.8. High Availability . . . . . . . . . . . . . . . . . . . . 18 4.9. Scale . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.10. Scale Convergence . . . . . . . . . . . . . . . . . . . . 19 4.11. Soak Test . . . . . . . . . . . . . . . . . . . . . . . . 20 5. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 21 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 7. Security Considerations . . . . . . . . . . . . . . . . . . . 21 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 8.1. Normative References . . . . . . . . . . . . . . . . . . 21 8.2. Informative References . . . . . . . . . . . . . . . . . 22 Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 22 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 1. Introduction EVPN is defined in RFC 7432, and describes BGP MPLS- based Ethernet VPNs (EVPN).PBB-EVPN is defined in RFC 7623, discusses how Ethernet Provider backbone Bridging can be combined with EVPNs to provide a new/combined solution.This draft defines methodologies that can be used to benchmark both RFC 7432 and RFC 7623 solutions.Further, this draft provides methodologies for benchmarking the performance of Jacob & Tiruveedhula Expires June 20, 2020 [Page 2] Internet-Draft EVPN Benchmarking Methodology December 2019 EVPN data and control planes, MAC learning, MAC flushing, MAC aging, convergence, high availability, and scale. 1.1. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 8174 [RFC8174]. 1.2. Terminologies All-Active Redundancy Mode: When all PEs attached to an Ethernet segment are allowed to forward known unicast traffic to/from that Ethernet segment for a given VLAN, then the Ethernet segment is defined to be operating in All-Active redundancy mode. AA All Active mode CE Customer Router/Devices/Switch. DF Designated Forwarder DUT Device under test. Ethernet Segment (ES): When a customer site (device or network) is connected to one or more PEs via a set of Ethernet links, then that set of links is referred to as an 'Ethernet segment'. EVI: An EVPN instance spanning the Provider Edge (PE) devices participating in that EVPN. Ethernet Segment Identifier (ESI): A unique non-zero identifier that identifies an Ethernet segment is called an 'Ethernet Segment Identifier'. Ethernet Tag: An Ethernet tag identifies a particular broadcast domain, e.g., a VLAN. An EVPN instance consists of one or more broadcast domains. Interface Physical interface of a router/switch. IRB Integrated routing and bridging interface MAC Media Access Control addresses on a PE. MHPE2 Multi homed Provider Edge router 2. MHPE1 Multi homed Provider Edge router 1. Jacob & Tiruveedhula Expires June 20, 2020 [Page 3] Internet-Draft EVPN Benchmarking Methodology December 2019 SHPE3 Single homed Provider Edge Router 3. PE: Provider Edge device. P Provider Router. RR Route Reflector. RT Traffic Generator. Sub Interface Each physical Interfaces is subdivided into Logical units. SA Single Active Single-Active Redundancy Mode: When only a single PE, among all the PEs attached to an Ethernet segment, is allowed to forward traffic to/from that Ethernet segment for a given VLAN, then the Ethernet segment is defined to be operating in Single-Active redundancy mode. 2. Test Topology EVPN/PBB-EVPN Services running on SHPE3, MHPE1 and MHPE2 in Single Active Mode: Jacob & Tiruveedhula Expires June 20, 2020 [Page 4] Internet-Draft EVPN Benchmarking Methodology December 2019 | Traffic Generator acts as a sender/receiver of layer 2 traffic with multiple vlan. +----------+ | | | SHPE3 | | | +----------+ | |Core link +----------+ | | | RR | | | Route Reflector /Provider router +----------+-------------| | | | Core links | +----------+ +-----------+ | | | MHPE2 | | DUT | | | | MHPE1 | | | +----------+ +-----------+ | PE-CE link | +----------+------------ | | | CE | | layer2 | |bridge | +----------+------------ Traffic Generator acts as a sender/receiver of layer 2 traffic with multiple vlan. Topology 1 Test Setup Figure 1 Jacob & Tiruveedhula Expires June 20, 2020 [Page 5] Internet-Draft EVPN Benchmarking Methodology December 2019 +-----------------+---------------------+---------------------+---------------------+----------------------+-----------------------+ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Mode | | | |Receiver | | | | Test |Traffic Direction |Sender | | | | | | | | | | | | | | | | | | | | | | | | +----------------------------------------------------------------------------------------------------------------------------------+ | | | | | | | | | | | | SHPE3 | | |Single Active | Local Mac | |CE | |Layer 2 traffic | | | Learning | Uni | | | | | | | | | | multiple MAC | | | | | | | | +-----------------------------------------------------------------------------------------------------------------------------------+ | | | | | | | |Single Active | Remote MAC | | | CE |Layer 2 traffic | | | Learning | uni | SHPE3 | | | | | | | | |multiple MAC | | | | | | | ++ +----------------------------------------------------------------------------------------------------------------------------------+ | | | | | | | |Single Active | Scale Convergence | Bi | | CE/SHPE3 | | | | | | CE/SHPE3 | |Layer 2 traffic | | | Local& Remote | | | |multiple mac& vlans | | | Learning | | | | | +-----------------+---------------------+---------------------+--------------------------------------------+-----------------------+ | ++ Table showing Traffic pattern for various test Figure 2 Test Setup Configurations: There are five routers in the Test setup.SHPE3, RR/P, MHPE1 and MHPE2 emulating a service provider network. CE is a customer device connected to MHPE1 and MHPE2, it is configured with bridge domains in multiple vlans.The traffic generator is connected to CE and SHPE3.The MHPE1 acts as DUT.The traffic generator will be used as sender and receiver of traffic.The measurement will be taken in DUT. Jacob & Tiruveedhula Expires June 20, 2020 [Page 6] Internet-Draft EVPN Benchmarking Methodology December 2019 All routers except CE are configured with OSPF/IS-IS,LDP,MPLS,BGP with EVPN address family. All routers except CE are configured with Interior border gateway protocol,RR acting as route reflector. MHPE1,MHPE2,SHPE3 must be configured with "N" EVPN/PBB-EVPN instances depends up on the cases. MHPE1 and MHEPE2 must be configured with ESI per vlan or ESI on interface. MHPE1 and MHEPE2 are running Single Active mode of EVPN. CE is acting as bridge configured with multiple vlans,the same vlans are configured on MHPE1,MHPE2,SHPE3. Depending up on the test scenarios the traffic generators will be used to generate uni directional or bi directional flows. The above configuration will be serving as the base configuration for all test cases. 3. Test Cases for EVPN Benchmarking 3.1. Local MAC Learning Objective: To Record the time taken to learn the MAC address locally in DUT. Topology : Topology 1 Procedure: The data plane MAC learning can be measured using the parameters defined in RFC 2889 section 5.8.Send "X" unicast frames from CE to MHPE1(DUT) working in SA mode.Traffic generator connected to CE must send frames with "X" different source and destination MAC address. The DUT must learn these "X" macs in data plane. Measurement : Measure the time taken to learn "X" MACs locally in DUT evpn mac table. The data plane measurement is taken by considering DUT as black box. The range of MACs are known from traffic generator,the same must be learned in DUT, the time taken to learn "X" macs is measured. Jacob & Tiruveedhula Expires June 20, 2020 [Page 7] Internet-Draft EVPN Benchmarking Methodology December 2019 The test is repeated for "N" times and the values are collected.The mac learning rate is calculated by averaging the values obtained from "N" samples. Mac learning rate = (T1+T2+..Tn)/N 3.2. Remote MAC Learning Objective: To Record the time taken to learn the remote macs. Topology : Topology 1 Procedure: Send X frames with X different source and destination mac addresses to SHPE3 from traffic generator.SHPE3 will advertise these locally learned macs to MHPE1 and MHPE2 via control plane.Measure the time taken to learn these X MACs from remote peer in DUT EVPN MAC address table.The DUT and MHPE2 are running SA mode. Measurement : Measure the time taken by the DUT to learn the "X" MACs in the data plane.The test is repeated for "N" times and the values are collected. The remote mac learning rate is calculated by averaging the values obtained from "N" samples. Remote mac learning rate = (T1+T2+..Tn)/N 3.3. MAC Flush due to local link failure and Relearning Objective: To record the time taken to flush the locally learned mac and the time taken to relearn the same amount of macs. Topology : Topology 1 Procedure: Send X frames with X different source and destination mac addresses to DUT from CE using traffic generator. Wait till the MHPE1 learns all X MAC addresses. Then fail the MHPE1 CE link and measure the time taken to flush these X MACs from the EVPN MAC table. Bring up Jacob & Tiruveedhula Expires June 20, 2020 [Page 8] Internet-Draft EVPN Benchmarking Methodology December 2019 the link which was made Down(the link between MHPE1 and CE).Measure time taken to relearn it.The DUT and MHPE2 are running SA mode. Measurement : Measure the time taken for flushing these X MAC addresses.Measure the time taken to relearn the X MACs in DUT.The test is repeated for "N" times and the values are collected.The flush and the relearning time is calculated by averaging the values obtained by "N" samples. Flush rate = (T1+T2+..Tn)/N Relearning rate = (T1+T2+..Tn)/N 3.4. MAC Flush due to remote link failure. Objective: To record the time taken to flush the remote mac learned in DUT during remote link failure. Topology : Topology 1 Procedure: Send X frames with X different source and destination mac addresses to DUT from SHPE3 using traffic generator. Bring down the link between SHPE3 and traffic generator.Then measure the time taken to flush the DUT EVPN MAC table.The DUT and MHPE2 are running SA mode. Measurement : Measure the time taken to flush X remote MACs from EVPN MAC table of the DUT.The test is repeated for "N" times and the values are collected.The flush rate is calculated by averaging the values obtained by "N" samples. Flush rate = (T1+T2+..Tn)/N 3.5. MAC Aging Objective: To measure the mac aging time. Topology : Topology 1 Jacob & Tiruveedhula Expires June 20, 2020 [Page 9] Internet-Draft EVPN Benchmarking Methodology December 2019 Procedure: Send X frames with X different source and destination mac addresses to DUT from CE using traffic generator.Wait till X MAC addresses are learned. Then stop the traffic.Record the time taken to flush X MACS from DUT EVPN MAC table due to aging.The DUT and MHPE2 are running SA mode. Measurement : Measure the time taken to flush X MAC addresses due to aging.The test is repeated for "N" times and the values are collected.The aging is calculated averaging the values obtained by "N" samples. Aging time for X Macs in sec = (T1+T2+..Tn)/N 3.6. Remote Mac Aging Objective: To measure the remote mac aging time. Topology : Topology 1 Procedure: Send X frames with X different source and destination mac addresses to DUT from SHPE3 using traffic generator. Stop the traffic at remote PE SHPE3.Due to MAC aging SHPE3 will withdraw its routes from DUT and MHPE2.Measure the time taken to remove these MACs from DUT EVPN MAC table.DUT and MHPE2 are running in SA mode Measurement : Measure the time taken to flush X remote MACs learned in DUT EVPN MAC table due to aging.The test is repeated for "N" times and the values are collected.The aging is calculated by averaging the values obtained by "N" samples. Aging time for X Macs in sec = (T1+T2+..Tn)/N 3.7. Local and Remote MAC Learning Objective: To record the time taken to learn both local and remote macs. Jacob & Tiruveedhula Expires June 20, 2020 [Page 10] Internet-Draft EVPN Benchmarking Methodology December 2019 Topology : Topology 1 Procedure: Send X frames with X different source and destination mac addresses to DUT from SHPE3 using traffic generator.Send X frames with different source and destination mac addresses from traffic generator connected to CE.The source and destination addresses of flows must be complimentary to have unicast flows.Measure the time taken by the DUT to learn 2X in EVPN MAC.DUT and MHPE2 are running in SA mode. Measurement : Measure the time taken to learn 2X MAC addresses in DUT EVPN MAC table. The test is repeated for "N" times and the values are collected. The mac learning time is calculated by averaging the values obtained by "N" samples. Mac learning rate = (T1+T2+..Tn)/N 3.8. High Availability. Objective: To record traffic loss during routing engine fail over. Topology : Topology 1 Procedure: Send X frames from CE to DUT from traffic generator with X different source and destination mac addresses.Send X frames from traffic generator to SHPE3 with X different source and destination mac addresses, so that 2X MAC address will be learned in the DUT.There is a bi directional traffic flow with X pps in each direction.Then do a routing engine fail-over. Measurement : The expectation of the test is 0 traffic loss with no change in the DF role.DUT should not withdraw any routes.But in cases where the DUT is not property synchronized between master and standby,due to that packet loss are observed.In that scenario the packet loss is measured.The test is repeated for "N" times and the values are Jacob & Tiruveedhula Expires June 20, 2020 [Page 11] Internet-Draft EVPN Benchmarking Methodology December 2019 collected.The packet loss is calculated by averaging the values obtained by "N" samples. Packet loss in sec with 2X mac addresses = (T1+T2+..Tn)/N 3.9. ARP/ND Scale To Record the DUT scaling limit of ARP/ND. Objective: To Record the ARP/ND scale of the DUT. Topology : Topology 1 Procedure: Send X arp/neighbor discovery(ND) from the traffic generator to DUT with different sender ip/ipv6,mac addresses to the target IRB address configured in EVPN instance. The EVPN instance learns the mac+ip and mac+ipv6 addresses from these request and advertise as type 2 mac+ip/ mac+ipv6 route to remote provide edge routers which have same EVPN configurations.The value of X must be increased at a incremental value of 5% of X, till the limit is reached.The limit is where the DUT cant learn any more type 2 mac+ip/mac+ipv6.The test must be separately conducted for arp and ND. Measurement : Measure the scale limit of type 2 mac+ip/mac+ipv6 route which DUT can learn.The test is repeated for "N" times and the values are collected.The scale limit is calculated by averaging the values obtained by "N" samples for both mac+ip and mac+ipv6. DUT scale limit for mac+ip = (v1+v2+..vn)/N DUT scale limit for mac+ipv6 = (v1+v2+..vn)/N 3.10. Scaling of Services Objective: This test is to measure the scale of EVPN instances that a DUT can hold. Topology : Topology 1 Jacob & Tiruveedhula Expires June 20, 2020 [Page 12] Internet-Draft EVPN Benchmarking Methodology December 2019 Procedure: The DUT,MHPE2 and SHPE3 are scaled to "N" EVI.Clear BGP neighbors of the DUT.Once adjacency is established in the DUT, check the routes received from MHPE2 and SHPE3 for "N" EVI in the DUT.Then increment the scale of N by 5% of N till the limit is reached.The limit is where the DUT cant learn any EVPN routes from peers. Measurement : There should not be any loss of route types 1,2,3 and 4 in DUT.DUT must relearn all type 1,2,3 and 4 from remote routers. The DUT must be subjected to various values of N to find the optimal scale limit. The scope of the test is find out the maximum evpn instance that a DUT can hold. 3.11. Scale Convergence Objective: To measure the convergence time of DUT when the DUT is scaled with EVPN instance along with traffic. Topology : Topology 1 Procedure: Scale N EVIs in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE using traffic generator with X different source and destination mac addresses for N EVI's.Send F frames from traffic generator to SHPE3 with X different source and destination mac addresses.There will be 2X number of MAC addresses will be learned in DUT EVPN MAC table. There is a bi directional traffic flow with F pps in each direction. Then clear the BGP neighbors in the DUT.Once the adjacency is restored in DUT.Measure the time taken to learn 2X MAC address in DUT MAC table. Measurement : The DUT must learn 2X MAC addresses. Measure the time taken to learn 2X MAC in DUT.The test is repeated for "N" times and the values are collected. The convergence time is calculated by averaging the values obtained by "N" samples. Time taken to learn 2X macs in DUT = (T1+T2+..Tn)/N Jacob & Tiruveedhula Expires June 20, 2020 [Page 13] Internet-Draft EVPN Benchmarking Methodology December 2019 3.12. SOAK Test. Objective: This test is carried out to measure the stability of the DUT in a scaled environment with traffic over a period of time "T'". In each interval "t1" the DUT CPU usage, memory usage are measured.The DUT is checked for any crashes during this time period. Topology : Topology 1 Procedure: Scale N EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE using traffic generator with different X source and destination mac addresses for N EVI's.Send F frames from traffic generator to SHPE3 with X different source and destination mac addresses.There will be 2X number of MAC addresses will be learned in DUT EVPN MAC table. There is a bi directional traffic flow with F pps in each direction. The DUT must run with traffic for 24 hours, every hour check for memory leak, CPU usage and crash. Measurement : Take the hourly reading of CPU, process memory. There should not be any leak, crashes, CPU spikes. Th CPU spike is determined as the CPU usage which shoots at 40 to 50 percent of the average usage.The average value vary from device to device. Memory leak is determined by increase usage of the memory for EVPN process. The expectation is under steady state the memory usage for EVPN process should not increase. 4. Test Cases for PBB-EVPN Benchmarking 4.1. Local MAC Learning Objective: To Record the time taken to learn the MAC address locally. Topology : Topology 1 Procedure: Jacob & Tiruveedhula Expires June 20, 2020 [Page 14] Internet-Draft EVPN Benchmarking Methodology December 2019 Send "X" unicast frames from CE to MHPE1(DUT) working in SA mode with "X" different source and destination mac addresses from traffic generator.The DUT must learn "X" macs in data plane. Measurement : Measure the time taken by the DUT to learn the "X" MACs in the data plane. The data plane measurement is taken by considering DUT as black box the range of "X" MAC is known from traffic generator and the same must be learned in DUT, the time taken to learn "X" MAC is measured.The test is repeated for "N" times and the values are collected.The mac learning time is calculated by averaging the values obtained from "N" samples. Mac learning rate = (T1+T2+..Tn)/N 4.2. Remote Mac Learning Objective: To Record the time taken to learn the remote macs. Topology : Topology 1 Procedure: Send X frames with X different source and destination mac addresses to SHPE3 from traffic generator.These macs will be flooded to MHPE1 and MHPE2 by SHPE3.The DUT and MHPE2 are running SA mode. Measurement : Measure the time taken to learn X mac addresses in DUT mac table. The test is repeated for "N" times and the values are collected.The mac learning time is calculated by averaging the values obtained by "N" samples. Remote Mac learning rate = (T1+T2+..Tn)/N 4.3. MAC Flush due to link failure Objective: To record the time taken to flush the locally learned mac and the time taken to relearn the same amount of macs. Topology : Topology 1 Jacob & Tiruveedhula Expires June 20, 2020 [Page 15] Internet-Draft EVPN Benchmarking Methodology December 2019 Procedure: Send X frames with X different source and destination mac addresses to DUT from CE using traffic generator.Wait till the MHPE1 learn all X MAC address.Then fail the MHPE1 CE link and measure the time taken to flush these X MACs from the PBB-EVPN MAC table.Then bring up the link.Measure the time taken to relearn X MACS.The DUT and MHPE2 are running SA mode. Measurement : Measure the time taken for flushing these X MAC addresses.Measure the time taken to relearn the X MACs in DUT.The test is repeated for "N" times and the values are collected. The flush and the relearning time is calculated by averaging the values obtained by "N" samples. Flush rate = (T1+T2+..Tn)/N Relearning rate = (T1+T2+..Tn)/N 4.4. MAC Flush due to remote Failure Objective: To record the time taken to flush the remote mac learned in DUT during remote link failure. Topology : Topology 1 Procedure: Send X frames with X different source and destination mac addresses to DUT from SHPE3 using traffic generator.Bring down the link between SHPE3 and traffic generator. Then measure the time taken to flush the DUT PBB-EVPN MAC address table.The remote MACs will be learned by Data plane, but the B-MAC will be learned by control plane.The DUT and MHPE2 are running SA mode. Measurement : Measure the time taken to flush X remote MACs from PBB-EVPN MAC table of DUT.Repeat these test and plot the data.The test is repeated for "N" times and the values are collected. The flush rate is calculated by averaging the values obtained by "N" samples. Flush rate = (T1+T2+..Tn)/N Jacob & Tiruveedhula Expires June 20, 2020 [Page 16] Internet-Draft EVPN Benchmarking Methodology December 2019 4.5. MAC aging Objective: To measure the mac aging time. Topology : Topology 1 Procedure: Send X frames with X different source and destination mac addresses to DUT from CE using traffic generator.Wait till X MAC address are learned in DUT PBB-EVPN MAC table.Then stop the traffic.Record the time taken to flush X MAC entries due to aging. The DUT and MHPE2 running in SA mode Measurement : Measure the time taken to flush X MAC addresses due to aging.Repeat these test and plot the data.The test is repeated for "N" times and the values are collected.The aging is calculated by averaging the values obtained by "N" samples. Aging time for X Macs in sec = (T1+T2+..Tn)/N 4.6. Remote MAC Aging. Objective: To measure the remote mac aging time. Topology : Topology 1 Procedure: Send X frames with X different source and destination mac addresses to DUT from SHPE3 using traffic generator.Stop the traffic at remote PE(SHPE3).Measure the time taken to remove these remote MACs from DUT PBB-EVPN MAC table.The DUT and MHPE2 are running in SA mode. Measurement : Measure the time taken to flush the X remote MACs from DUT PBB-EVPN MAC table due to aging .The test is repeated for "N" times and the Jacob & Tiruveedhula Expires June 20, 2020 [Page 17] Internet-Draft EVPN Benchmarking Methodology December 2019 values are collected. The aging is calculated by averaging the values obtained by "N" samples. Aging time for X Macs in sec = (T1+T2+..Tn)/N 4.7. Local and Remote MAC Learning Objective: To record the time taken to learn both local and remote macs. Topology : Topology 1 Procedure: Send X frames with X different source and destination mac addresses to DUT from SHPE3 using traffic generator.Send X frames with different source and destination mac addresses from traffic generator connected to CE.The source and destination mac addresses of flows must be complimentary to have unicast flows.Measure the time taken by the DUT to learn 2X in MAC table.DUT and MHPE2 are running in SA mode. Measurement : Measure the time taken to learn 2X MAC addresses table in DUT PBB- EVPN MAC table.The test is repeated for "N" times and the values are collected.The mac learning time is calculated by averaging the values obtained by "N" samples. Mac learning rate = (T1+T2+..Tn)/N 4.8. High Availability Objective: To record traffic loss during routing engine failover. Topology : Topology 1 Procedure: Send X frames to DUT with X different source and destination mac addresses from CE using the traffic generator.Send X frames from traffic generator to SHPE3 with X different source and destination Jacob & Tiruveedhula Expires June 20, 2020 [Page 18] Internet-Draft EVPN Benchmarking Methodology December 2019 mac addresses, so that 2X MAC address will be Learned in DUT. There is a bi directional traffic flow with X pps in each direction. Then do a routing engine fail-over. Measurement : The expectation of the test is 0 traffic loss with no change in the DF role.DUT should not withdraw any routes.But in cases where the DUT is not property synchronized between master and standby,due to that packet loss are observed. In that scenario the packet loss is measured.The test is repeated for "N" times and the values are collected.The packet loss is calculated by averaging the values obtained by "N" samples. Packet loss in sec with 2X mac addresses = (T1+T2+..Tn)/N 4.9. Scale Objective: To measure the scale limit of DUT for PBB-EVPN. Topology : Topology 1 Procedure: The DUT,MHPE2 and SHPE3 are scaled to "N" PBB-EVN instances.Clear BGP neighbors of the DUT. Once adjacency is established in the DUT. check the routes received from MHPE2 and SHPE3 for "N" PBB-EVPN instances in the DUT.Then increment the scale of N by 5% of N till the limit is reached.The limit is where the DUT cant learn any EVPN routes from peers. Measurement : There should not be any loss of route types 2,3 and 4 in DUT.DUT must relearn all type 2,3 and 4 from remote routers.The DUT must be subjected to various values of N to find the optimal scale limit. The scope of the test is find out the maximum evpn instance that a DUT can hold. 4.10. Scale Convergence Objective: To measure the convergence time of DUT when the DUT is scaled with EVPN instance along with traffic. Jacob & Tiruveedhula Expires June 20, 2020 [Page 19] Internet-Draft EVPN Benchmarking Methodology December 2019 Topology : Topology 1 Procedure: Scale N PBB-EVPN instances in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE using traffic generator with X different source and destination mac addresses for N PBB-EVPN instances. Send F frames from traffic generator to SHPE3 with X different source and destination mac addresses.There will be 2X MAC addresses will be learned in DUT PBB-EVPN MAC table.There is a bi directional traffic flow with F pps in each direction.Then clear the BGP neighbors in the DUT. Once the adjacency is restored in DUT. Measure the time taken to learn 2X MAC address in DUT MAC table. Measurement : The DUT must learn 2X MAC addresses.Measure the time taken to learn 2X MAC in DUT.The test is repeated for "N" times and the values are collected.The convergence time is calculated by averaging the values obtained by "N" samples. Convergence time for 2X in sec = (T1+T2+..Tn)/N 4.11. Soak Test Objective: To measure the stability of the DUT in a scaled environment with traffic. Topology : Topology 1 Procedure: Scale N PBB-EVPN instances in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE using traffic generator with X different source and destination mac addresses for N PBB-EVPN instances. Send F frames from traffic generator to SHPE3 with X different source and destination mac addresses.There will be 2X MAC addresses will be learned in DUT PBB- EVPN MAC table.There is a bi directional traffic flow with F pps in Each direction.The DUT must run with traffic for 24 hours, every hour check the memory leak, crashes. Measurement : Jacob & Tiruveedhula Expires June 20, 2020 [Page 20] Internet-Draft EVPN Benchmarking Methodology December 2019 Take the hourly reading of CPU, process memory. There should not be any leak, crashes, CPU spikes.The CPU spike is determined as the CPU usage which shoots at 40 to 50 percent of the average usage.The average value vary from device to device.Memory leak is determined by increase usage of the memory for PBB-EVPN process.The expectation is under steady state the memory usage for PBB-EVPN process should not increase. 5. Acknowledgments We would like to thank Fioccola Giuseppe of Telecom Italia reviewing our draft and commenting it.We would like to thank Sarah Banks for guiding and mentoring us. 6. IANA Considerations This memo includes no request to IANA. 7. Security Considerations The benchmarking tests described in this document are limited to the performance characterization of controllers in a lab environment with isolated networks. The benchmarking network topology will be an independent test setup and MUST NOT be connected to devices that may forward the test traffic into a production network or misroute traffic to the test management network. Further, benchmarking is performed on a "black-box" basis, relying solely on measurements observable external to the controller. Special capabilities SHOULD NOT exist in the controller specifically for benchmarking purposes. Any implications for network security arising from the controller SHOULD be identical in the lab and in production networks. 8. References 8.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, DOI 10.17487/RFC2544, March 1999, . Jacob & Tiruveedhula Expires June 20, 2020 [Page 21] Internet-Draft EVPN Benchmarking Methodology December 2019 [RFC2899] Ginoza, S., "Request for Comments Summary RFC Numbers 2800-2899", RFC 2899, DOI 10.17487/RFC2899, May 2001, . 8.2. Informative References [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015, . [RFC7623] Sajassi, A., Ed., Salam, S., Bitar, N., Isaac, A., and W. Henderickx, "Provider Backbone Bridging Combined with Ethernet VPN (PBB-EVPN)", RFC 7623, DOI 10.17487/RFC7623, September 2015, . Appendix A. Appendix Authors' Addresses Sudhin Jacob (editor) Juniper Networks Bangalore India Phone: +91 8061212543 Email: sjacob@juniper.net Kishore Tiruveedhula Juniper Networks 10 Technology Park Dr Westford, MA 01886 USA Phone: +1 9785898861 Email: kishoret@juniper.net Jacob & Tiruveedhula Expires June 20, 2020 [Page 22]