Internet Engineering Task Force S. Jacob, Ed. Internet-Draft V. Nagarajan Intended status: Standards Track Juniper Networks Expires: August 8, 2019 February 4, 2019 Benchmarking Methodology for EVPN Multicasting draft-vikjac-bmwg-evpnmultest-00 Abstract This document defines methodologies for benchmarking IGMP proxy performance over EVPN-VXLAN. IGMP proxy over EVPN is defined in draft-ietf-bess-evpn-igmp-mld-proxy-02, and is being deployed in data center networks. Specifically this document defines the methodologies for benchmarking IGMP proxy convergence, leave latency Scale,Core isolation, high availability and longevity. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on August 8, 2019. Copyright Notice Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of Jacob & Nagarajan Expires August 8, 2019 [Page 1] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3 2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.1. How long it takes to learn ( X1...Xn) IGMP join messages in DUT . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.2. How long it takes to clear the (*,G) entries in the DUT . 6 3.3. How long it takes the DUT to stop forwarding the traffic(Measuring the leave latency) . . . . . . . . . . 6 3.4. How long it takes to learn (X1...Xn) IGMP join messages for N vlans in DUT . . . . . . . . . . . . . . . . . . . 7 3.5. How long it takes to clear the (*,G) entries in the DUT for N vlans . . . . . . . . . . . . . . . . . . . . . . . 8 3.6. How long it takes the DUT to stop forwarding the traffic for N vlans(Measuring the leave latency) . . . . . . . . 8 3.7. How long it takes to learn (X1...Xn) IGMP join messages for N vlans in DUT working EVPN AA mode . . . . . . . . . 9 3.8. How long it takes to clear the (*,G) entries for N vlans in DUT working EVPN AA . . . . . . . . . . . . . . . . . 10 3.9. How long it takes the DUT operating in EVPN AA to stop forwarding the traffic for N vlans(Measuring the leave latency) . . . . . . . . . . . . . . . . . . . . . . . . 10 4. Link Flap . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.1. To Measure the multicast packet loss in EVPN AA scenario on a CE link failure . . . . . . . . . . . . . . . . . . 11 5. Scale Convergence . . . . . . . . . . . . . . . . . . . . . . 12 5.1. To measure the packet loss during the core link failure. 12 6. High Availability . . . . . . . . . . . . . . . . . . . . . . 12 6.1. To Record the whether there is traffic loss due to routing engine failover for redundancy test. . . . . . . 12 7. SOAK Test . . . . . . . . . . . . . . . . . . . . . . . . . . 13 7.1. To Measure the stability of the DUT with scale and traffic. . . . . . . . . . . . . . . . . . . . . . . . . 13 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 14 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 10. Security Considerations . . . . . . . . . . . . . . . . . . . 14 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 14 11.1. Normative References . . . . . . . . . . . . . . . . . . 14 11.2. Informative References . . . . . . . . . . . . . . . . . 14 Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 14 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 14 Jacob & Nagarajan Expires August 8, 2019 [Page 2] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 1. Introduction IGMP proxy over EVPN-VXLAN is defined in draft-ietf-bess-evpn-igmp- mld-proxy-02,and is being deployed in data center networks. Specifically this document defines the methodologies for benchmarking IGMP proxy convergence,leave latency Scale,Core isolation, high availability and longevity. 1.1. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 1.2. Terminologies Leaf A layer 2 or layer 3 capable device Spine layer 3 capable device which is used to inter connect leaves CE Customer Router/Devices/Switch. AA EVPN Terminologies AA All-Active. AC Attachment Circuit RT Router Tester. DUT Device under Test. Sub Interface Each physical Interfaces is subdivided in to Logical units. EVI EVPN Instances which will be running on sub interface or physical port of the provider Edge routers. DF Designated Forwarder. ESI Ethernet Segment Identifier. 2. Test Topology EVPN Overlay Network running on leaf1, leaf2 leaf3,spine1 and spine 2 : Topology Diagram Jacob & Nagarajan Expires August 8, 2019 [Page 3] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 RT +--------------+ +--------------+ RT | | | +-------------------+ +---------+ spine1 +----------------------------------+ | | | spine2 | | | | | | | +----+---------+---------+ +----+----+----+------------------+ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | RT +---------+-----+---------+---+-------+----++ +--+-----------+----+ +-------------+ | | | | leaf3 | ++ leaf1 | | leaf2 | | | | DUT | | | | | +----+----------+ +---+--------+----+ +--------+----------+ | | | | | | | | | | | | | | | | +-+-----------------+------+ | | | | +--+ RT ++ RT | CE | RT | +-------+ +-------------------+ CE connected to leaf1 and leaf2 in EVPN AA mode. Topology 1 Topology Diagram Figure 1 There are six routers in the topology. Leaf1,leaf2, leaf3,spine1,spine2 emulating a data center network. CE is a customer device connected to leaf1 and leaf2, it is configured with bridge domains in different vlans. The router tester is connected to CE,leaf1,leaf2,leaf3,spine1 and spine 2 to emulate multicast source and host generating igmp join/leave. Jacob & Nagarajan Expires August 8, 2019 [Page 4] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 All routers except CE are configured with EBGP for the underlay All router are configured with EVPN-VXLAN overlay All leaves and spine must be configured "N" EVPN-VXLAN instances depends up on the cases. Leaf1 and Leaf2 must be configured with ESI per vlan or ESI on IFD. Leaf1 and leaf2 are running Active Active mode of EVPN-VXLAN. CE is acting as bridge configured with vlans Depends up on the test multicast traffic/host will be emulated by RT The above configuration will serve as base configuration for all the test cases. 3. Test Cases The following tests are conducted to measure the learning rate,leave rate,leave latency of IGMP messages which propagates in leaf and spine. 3.1. How long it takes to learn ( X1...Xn) IGMP join messages in DUT Objective: To Record the time taken to learn X1...Xn igmp join generated by host/hosts. Topology : Topology 1 Procedure: Configure "N" evpn-vxlan in leaf1,leaf2,leaf3,spine1 and spine2.Leaf1 and leaf2 are connected to CE which are working in EVPN AA mode. Configure a vlan in RT which is present in leaf1 then send igmp join messages for groups X1... Xn from RT to this vlan present in leaf1. Measure the time taken to learn X1..Xn (*,G) entries in the DUT. Measurement : Measure the time taken to learn the X1....Xn groups creating (*,G) entries in the DUT. Jacob & Nagarajan Expires August 8, 2019 [Page 5] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The time is calculated by averaging the values obtained from "N" samples. Time taken by DUT to learn and create X1...Xn (*,G ) entries in DUT which is measured in sec = (T1+T2+..Tn/N) 3.2. How long it takes to clear the (*,G) entries in the DUT Objective: To Record the time taken to clear the X1... Xn (*,G) entries in DUT. Topology : Topology 1 Procedure: Configure "N" evpn-vxlan in leaf1,leaf2,leaf3,spine1 and spine2. Leaf1 and leaf2 are connected to CE which are working in EVPN AA mode. Configure a vlan in RT which is present in leaf1, then send igmp join messages for groups ranging from X1...Xn from RT to this vlan present in leaf1 Then stop these igmp join messages from RT. Measurement : Measure the time taken to flush these X1...Xn (*,G) entries in DUT. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The time is calculated by averaging the values obtained from "N" samples. Time taken by DUT to flush these X1...Xn (*,G) entries in sec = (T1+T2+..Tn/N) 3.3. How long it takes the DUT to stop forwarding the traffic(Measuring the leave latency) Objective: To Record the time taken by the DUT to stop forwarding the multicast traffic during the receipt of IGMP leave from RT. Topology : Topology 1 Procedure: Configure "N" evpn-vxlan in leaf1,leaf2,leaf3,spine1 and spine2. Leaf1 and leaf2 are connected to CE which are working in EVPN AA Jacob & Nagarajan Expires August 8, 2019 [Page 6] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 mode. Configure a vlan in RT which is present in leaf1, then send igmp join from RT for this vlan to leaf1 for groups ranging from "X1....Xn". Then send traffic to these groups from spine1. Traffic flows from spine1 to leaf1. Send IGMP leave messages for these groups from RT to leaf1. Measure the time taken by the DUT to stop these multicast traffic to RT. Measurement : Measure the time taken by DUT to stop the multicast traffic flowing towards RT. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The time is calculated by averaging the values obtained from "N" samples. Time taken by DUT to stop the traffic towards RT connected in leaf1 in sec = (T1+T2+..Tn/N) 3.4. How long it takes to learn (X1...Xn) IGMP join messages for N vlans in DUT Objective: To Record the time taken to learn X1...Xn IGMP join generated by host/hosts located in N vlans. Topology : Topology 1 Procedure: Configure "N" evpn-vxlan in leaf1,leaf2,leaf3,spine1 and spine2. Leaf1 and leaf2 are connected to CE which are working in EVPN AA mode. Configure N vlans in RT, these vlans must be present in leaf1, then send igmp join messages for the groups ranging from X1...Xn for these N vlans from RT. Measure the time taken to learn these X1..Xn (*,G) entries in the DUT for N vlans. Measurement : Measure the time taken to learn the X1....Xn groups creating (*,G) entries in the DUT for N vlans. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The time is calculated by averaging the values obtained from "N" samples. Jacob & Nagarajan Expires August 8, 2019 [Page 7] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 Time taken by DUT to learn and create X1...Xn (*,G ) entries for N vlans in DUT which is measured in sec = (T1+T2+..Tn/N) 3.5. How long it takes to clear the (*,G) entries in the DUT for N vlans Objective: To Record the time taken to clear the X1... Xn (*,G) entries in DUT for N vlans. Topology : Topology 1 Procedure: Configure "N" evpn-vxlan in leaf1,leaf2,leaf3,spine1 and spine2. Leaf1 and leaf2 are connected to CE which are working in AA mode. Configure N vlans in RT, these vlans must be present in leaf1, then send igmp join messages for groups ranging from X1...Xn for these N vlans from RT. Then stop these IGMP messages. Measurement : Measure the time taken to flush these X1...Xn (*,G) entries in DUT. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The time is calculated by averaging the values obtained from "N" samples. Time taken by DUT to flush these X1...Xn (*,G) entries in sec = (T1+T2+..Tn/N) 3.6. How long it takes the DUT to stop forwarding the traffic for N vlans(Measuring the leave latency) Objective: To Record the time taken by the DUT to stop forwarding the multicast traffic to N vlans during the receipt of IGMP leave messages from RT. Topology : Topology 1 Procedure: Configure "N" evpn-vxlan in leaf1,leaf2,leaf3,spine1 and spine2.Leaf1 and leaf2 are connected to CE which are working in EVPN AA mode. Configure N vlans in RT which are present in leaf1, then send igmp join messages from RT for groups ranging from X1...Xn to these vlans Jacob & Nagarajan Expires August 8, 2019 [Page 8] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 present in leaf1. Then send traffic to these groups from spine1. Traffic flows from spine1 to leaf1. Send the IGMP leave messages for these groups in all vlans. Measure the time taken by the DUT to stop the traffic for these group flowing towards RT. Measurement : Measure the time taken by DUT to stop the multicast traffic flowing towards RT. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The time is calculated by averaging the values obtained from "N" samples. Time taken by DUT to stop the traffic towards RT in sec = (T1+T2+..Tn/N) 3.7. How long it takes to learn (X1...Xn) IGMP join messages for N vlans in DUT working EVPN AA mode Objective: To Record the time taken to learn X1...Xn IGMP join generated by host/hosts located in N vlans in DUT operating in EVPN AA mode. Topology : Topology 1 Procedure: Configure "N" evpn-vxlan in leaf1,leaf2,leaf3,spine1 and spine2. Leaf1 and leaf2 are connected to CE which are working in EVPN AA mode. Configure N vlans in RT, these vlans must be present in leaf1,leaf2, then send igmp join messages for the groups ranging from X1...Xn for these N vlans from RT to CE connected to leaf1 and leaf2 working EVPN AA mode.Measure the time taken to learn these X1..Xn (*,G) entries in the DUT for N vlans. Measurement : Measure the time taken to learn the X1....Xn groups by creating (*,G) entries in the DUT for N vlans. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The time is calculated by averaging the values obtained from "N" samples. Time taken by DUT to learn and create X1...Xn (*,G ) entries for N vlans which is measured in sec = (T1+T2+..Tn/N) Jacob & Nagarajan Expires August 8, 2019 [Page 9] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 3.8. How long it takes to clear the (*,G) entries for N vlans in DUT working EVPN AA Objective: To Record the time taken to clear the X1... Xn (*,G) entries in DUT for N vlans. Topology : Topology 1 Procedure: Configure "N" evpn-vxlan in leaf1,leaf2,leaf3,spine1 and spine2. Leaf1 and leaf2 are connected to CE which are working in AA mode. Configure N vlans in RT, these vlans must be present in leaf1, then send igmp join messages for groups ranging from X1...Xn for these N vlans from RT to CE which is connected to leaf1 and leaf2 working in EVPN AA mode.Then stop these IGMP messages. Measurement : Measure the time taken to flush these X1...Xn (*,G) entries in DUT. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The time is calculated by averaging the values obtained from "N" samples. Time taken by DUT to flush these X1...Xn (*,G) entries in sec = (T1+T2+..Tn/N) 3.9. How long it takes the DUT operating in EVPN AA to stop forwarding the traffic for N vlans(Measuring the leave latency) Objective: To Record the time taken by the DUT to stop forwarding the multicast traffic to N vlans during the receipt of IGMP leave messages from RT. Topology : Topology 1 Procedure: Configure "N" evpn-vxlan in leaf1,leaf2,leaf3,spine1 and spine2.Leaf1 and leaf2 are connected to CE which are working in EVPN AA mode. Configure N vlans in RT which are present in leaf1, then send igmp join messages from RT connected to CE for groups ranging from X1...Xn to these vlans. The CE in turn forwards the igmp messages to leaf1 and leaf2 operating in EVPN AA mode. Then send traffic to these Jacob & Nagarajan Expires August 8, 2019 [Page 10] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 groups from spine1. Traffic flows from spine1 to CE. Send the IGMP leave messages for these groups in all vlans from RT connected to CE. Measure the time taken by the DUT to stop the traffic for these group flowing towards RT. Measurement : Measure the time taken by DUT to stop the multicast traffic flowing towards RT. Repeat these test and plot the data. The test is repeated for "N" times and the values are collected. The time is calculated by averaging the values obtained from "N" samples. Time taken by DUT to stop the traffic towards RT in sec = (T1+T2+..Tn/N) 4. Link Flap 4.1. To Measure the multicast packet loss in EVPN AA scenario on a CE link failure Objective: To measure the packet loss during the CE to DF link failure. Topology : Topology 1 Procedure: Configure "N" evpn-vxlan in leaf1,leaf2,leaf3,spine1 and spine2.Leaf1 and leaf2 are connected to CE which are working in EVPN AA mode. Configure N vlans in RT which are present in leaf1, then send igmp join messages from RT connected to CE for groups ranging from X1...Xn to these vlans. The CE in turn forwards the igmp messages to leaf1 and leaf2 operating in EVPN AA mode. Then send traffic to these groups from spine1. Traffic flows from spine1 to CE. Fail the DF-CE link. The NON DF now will act as DF and start forwarding the multicast traffic. Measurement : Measure the multicast packet loss during the link failure.Repeat the test "N" times and plot the data.The packet loss is calculated by averaging the values obtained from "N" samples. Packet loss in sec = (T1+T2+..Tn/N) Jacob & Nagarajan Expires August 8, 2019 [Page 11] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 5. Scale Convergence 5.1. To measure the packet loss during the core link failure. Objective: To Measure the convergence at a higher number of vlans and igmp joins. Topology : Topology 1 Procedure: Configure "N" evpn-vxlan in leaf1,leaf2,leaf3,spine1 and spine2.Leaf1 and leaf2 are connected to CE which are working in EVPN AA mode. Configure N vlans in RT which are present in leaf1, then send igmp join messages from RT connected to CE for groups ranging from X1...Xn to these vlans. The CE in turn forwards the igmp messages to leaf1 and leaf2 operating in EVPN AA mode. Then send traffic to these groups from spine1. Traffic flows from spine1 to CE. Fail the core link of DF. The NON DF now will act as DF and start forwarding the multicast traffic. The vlans and the multicast groups must be a higher value of N taken at random. Measurement : Measure the packet loss in seconds once the core link is restored.Repeat the test "N" times and plot the data.The packet loss is calculated by averaging the values obtained from "N" samples. Packet loss in sec = (T1+T2+..Tn/N) 6. High Availability 6.1. To Record the whether there is traffic loss due to routing engine failover for redundancy test. Objective: To record traffic loss during routing engine failover. Topology : Topology 3 Procedure: Jacob & Nagarajan Expires August 8, 2019 [Page 12] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 Configure "N" evpn-vxlan in leaf1,leaf2,leaf3,spine1 and spine2.Leaf1 and leaf2 are connected to CE which are working in EVPN AA mode. Configure N vlans in RT which are present in leaf1, then send igmp join messages from RT connected to CE for groups ranging from X1...Xn to these vlans. The CE in turn forwards the igmp messages to leaf1 and leaf2 operating in EVPN AA mode. Then send traffic to these groups from spine1. Traffic flows from spine1 to CE. Then perform a routing engine failure. Measurement : There should be 0 traffic loss which is the ideal case, No change in the DF role. DUT should not withdraw any routes.Repeat the test "N" times and plot the data.The packet loss is calculated by averaging the values obtained from "N" samples. Packet loss in sec = (T1+T2+..Tn/N) 7. SOAK Test This is measuring the performance of DUT running with scaled configuration with traffic over a peroid of time "T'". In each interval "t1" the parameters measured are CPU usage, memory usage, crashes. 7.1. To Measure the stability of the DUT with scale and traffic. Objective: To measure the stability of the DUT in a scaled environment with traffic. Topology : Topology 3 Procedure: Configure "N" evpn-vxlan in leaf1,leaf2,leaf3,spine1 and spine2.Leaf1 and leaf2 are connected to CE which are working in EVPN AA mode. Configure N vlans in RT which are present in leaf1, then send igmp join messages from RT connected to CE for groups ranging from X1...Xn to these vlans. The CE in turn forwards the igmp messages to leaf1 and leaf2 operating in EVPN AA mode. Then send traffic to these groups from spine1. Traffic flows from spine1 to CE. Measurement : Jacob & Nagarajan Expires August 8, 2019 [Page 13] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 Take the hourly reading of CPU, process memory. There should not be any leak, crashes, CPU spikes. 8. Acknowledgements We would like to thank Al and Sarah for the support. 9. IANA Considerations This memo includes no request to IANA. 10. Security Considerations There is no additional consideration from RFC 6192. 11. References 11.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, DOI 10.17487/RFC2544, March 1999, . [RFC2899] Ginoza, S., "Request for Comments Summary RFC Numbers 2800-2899", RFC 2899, DOI 10.17487/RFC2899, May 2001, . 11.2. Informative References [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015, . Appendix A. Appendix Authors' Addresses Jacob & Nagarajan Expires August 8, 2019 [Page 14] Internet-Draft EVPN-Multicasting Benchmarking Methodology February 2019 Sudhin Jacob (editor) Juniper Networks Bangalore, Karnataka 560103 India Phone: +91 8061212543 Email: sjacob@juniper.net Vikram Nagarajan Juniper Networks Bangalore, Karnataka 560103 India Phone: +91 8061212543 Email: vikramna@juniper.net Jacob & Nagarajan Expires August 8, 2019 [Page 15]