Network Working Group Vic Liu Internet Draft Lingli Deng Intended status: Informational Dapeng Liu China Mobile Expires: January 2015 July 5, 2014 Gap Analysis on Virtualized Network Test draft-liu-dclc-gap-virtual-test-00.txt Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. This document may not be modified, and derivative works of it may not be created, and it may not be published except as an Internet-Draft. This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. This document may not be modified, and derivative works of it may not be created, except to publish it as an RFC and to translate it into languages other than English. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents Vic Liu Expires January 5, 2015 [Page 1] Internet-Draft PS for VxLAN Performance July 2014 at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on January 5, 2009. Copyright Notice Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Abstract In virtual network, the VSwitch become an important element in the network. It is in charge for both forward management and virtual network function. We figure out that to evaluate the performance of VSwitch is not like the test on hardware (like using RFC2544). This draft introduce the gap of testing for virtual network performance. On chapter 2 we introduce the test practices which is the virtual test bad setup and benchmark test. On chapter three, we analysis the gap of virtual network performance test. Vic Liu Expires January 5, 2015 [Page 2] Internet-Draft PS for VxLAN Performance July 2014 Table of Contents 1. Introduction ................................................ 3 2. Virtual network test......................................... 3 2.1. Test Bed Setup ......................................... 3 2.2. Benchmark test on virtualized network .................. 6 3. Gap analysis for virtual network test ....................... 7 3.1. Test methodology ....................................... 7 3.2. Throughput ............................................. 7 3.3. Latency ................................................ 8 3.4. CPU .................................................... 8 3.5. Number of test.......................................... 9 4. Security Considerations..................................... 10 5. IANA Considerations ........................................ 10 6. Conclusions ................................................ 10 7. References ................................................. 10 7.1. Normative References................................... 10 7.2. Informative References................................. 10 8. Acknowledgments ............................................ 10 1. Introduction In virtual network, the VSwitch become an important element in the network. It is in charge for both forward management and virtual network function. We figure out that to evaluate the performance of VSwitch is not like the test on hardware (like using RFC2544). This draft introduce the gap of testing for virtual network performance. On chapter 2 we introduce the test practices which is the virtual test bad setup and benchmark test. On chapter three, we analysis the gap of virtual network performance test. 2. Virtual network test practices 2.1. Test Bed Setup The test bed is constituted by two physical server with 10GE NIC, a test center, a 10GE TOR switch for test traffic and a 1GE TOR switch for management. Vic Liu Expires January 5, 2015 [Page 3] Internet-Draft PS for VxLAN Performance July 2014 ---------------------- |Test Center PHY 10GE*2| ---------------------- || || ---------- =====| 10GE TOR |======= || ---------- || || || || || ------------------- ------------------- | -------------- | | -------------- | | |V-switch(VTEP)| | | |V-switch(VTEP)| | | -------------- | | -------------- | | | | | | | | | | ----- ----- | | ----- ----- | | |TCVM1| |TCVM2|| | |TCVM1| |TCVM2|| | ----- ----- | | ----- ----- | ------------------- ------------------- Server1 Server2 Vic Liu Expires January 5, 2015 [Page 4] Internet-Draft PS for VxLAN Performance July 2014 Two Dell server are R710XD (CPU: E5-2460) and R710 (CPU: E5-2430) with a pair of 10GE NIC. And in the server we allocate 2 vCPU and 8G memory to each Test Center Virtual Machine (TCVM). In traffic model A: We use a physical test center connect to each server to verify the benchmark of each server. ---------------------- |Test Center PHY 10GE*2| ---------------------- || || ------------------- | -------------- | | |V-switch(VTEP)| | | -------------- | | | | | | ----- ----- | | |TCVM1| |TCVM2|| | ----- ----- | ------------------- Server1 In traffic model B: We use the benchmark to test the performance of VxLAN. ---------- =====| 10GE TOR |======= || ---------- || || || || || ------------------- ------------------- | -------------- | | -------------- | | |V-switch(VTEP)| | | |V-switch(VTEP)| | | -------------- | | -------------- | | | | | | | | | | ----- ----- | | ----- ----- | | |TCVM1| |TCVM2|| | |TCVM1| |TCVM2|| | ----- ----- | | ----- ----- | ------------------- ------------------- Server1 Server2 Vic Liu Expires January 5, 2015 [Page 5] Internet-Draft PS for VxLAN Performance July 2014 2.2. Benchmark test on virtualized network The reason we need a benchmark test is we realized that the virtualized network is different from physical network device. We cannot use test methodology like RFC 2544. The performance is not linear growth with traffic we generate. It has an inflection point. We allocate the Test Center Virtual Machine (TCVM) with one v CPU and 4G memory to get the benchmark, we use traffic model A and get the result table below: Server 1: CPU E5-2430 ------------------------------------------------------------ | Byte| Rate(GE)| Server CPU MHZ |Server Mem| VM CPU| VM Mem| ------------------------------------------------------------ | 0 | 0 | 495 | 2817 | 370 | 495 | ------------------------------------------------------------ | 128 | 0.40 | 4704 | 2817 | 4541 | 495 | ------------------------------------------------------------ | 256 | 0.66 | 4830 | 2824 | 4519 | 495 | ------------------------------------------------------------ | 512 | 1.43 | 5161 | 2818 | 4870 | 495 | ------------------------------------------------------------ | 1024| 2.62 | 5131 | 2819 | 4782 | 495 | ------------------------------------------------------------ | 1518| 3.66 | 4957 | 2820 | 4585 | 495 | ------------------------------------------------------------ Server 2: CPU E5-2620 ------------------------------------------------------------ | Byte| Rate(GE)| Server CPU MHZ |Server Mem| VM CPU| VM Mem| ------------------------------------------------------------ | 0 | 0 | 188 | 2712 | 59 | 493 | ------------------------------------------------------------ | 128 | 0.46 | 4648 | 2703 | 4100 | 494 | ------------------------------------------------------------ | 256 | 0.70 | 4552 | 2704 | 4077 | 493 | ------------------------------------------------------------ | 512 | 1.50 | 4521 | 2703 | 3958 | 493 | ------------------------------------------------------------ | 1024| 3.10 | 4559 | 2702 | 3938 | 493 | ------------------------------------------------------------ | 1518| 5.46 | 5195 | 2704 | 4377 | 493 | ------------------------------------------------------------ Vic Liu Expires January 5, 2015 [Page 6] Internet-Draft PS for VxLAN Performance July 2014 3. Gap analysis for virtual network test 3.1. Test methodology There is a huge difference between testing on hardware network and virtual network. On virtual network, the Device under Test (DUT) is the VSwitch or hypervisor (or the protocol such as VxLAN encapsulation by VSwitch) while the test machine which is a VM connect to the DUT shares the same resources of the physical server. In virtual network, tester and the DUT (visual switches) are in one server (physically converged), so the CPU and MEM share the same resources. Theoretically, the tester's operation may has some influences on the DUT's performances. However, for the specialty of virtualization, this method is the only way to assess the truth of assessment method. 3.2. Throughput The throughput generate to test is also different from hardware switch (for example, RFC 2544). We realize that the throughput is very hard to reach the line rate by the TCVM. There is an inflection point as the CPU over loaded. For example, if we generate 1Gb traffic, it will 100% receive. But if we generate 10Gb traffic, it can only generate 530Mb because CPU is overloaded. Besides, the generate traffic is related to the packet length. For example, 128bit can only generate 0.4Gb traffic and 1518bit can generate 4Gb traffic. The test concept on traditional physical switches is not apply to virtual switch test. In traditional throughput, the capability of the switch can reach line rate at any bytes while virtual network cannot. Under the background of existing technology, when we mean to test the virtual switch's throughput, the concept of traditional physical switch will not be applicable. The traditional throughput indicates the switches' largest transmit capability, for certain selected bytes and selected cycle under zero-packet-lose conditions. But in virtual environment, the fluctuant of performance on virtual network will be much greater than dedicated physical devices. In the same time, because the DUT and the tester cannot be separated, which only proved the DUT realize same network performances under certain circumstances, it also means the DUT may achieve higher capability. Therefore, we change the throughout in virtual environment to actual throughput, hoping in future, as the improvement of technique, the actual throughput will approach the theoretical throughput gradually. Vic Liu Expires January 5, 2015 [Page 7] Internet-Draft PS for VxLAN Performance July 2014 Of course, under actual condition, this throughout have certain referential meanings. In most cases, common throughput application cannot compare with professional tester, so for virtual application and data center's deployment, the actual throughout already have great refinance value. 3.3. Latency Physical tester's time reference from its own clock or other time source, such as GPS, which can achieve the accuracy of 10ns. In virtual network circumstances, the virtual tester gets its reference time from Linux systems. But the clock on Linux of different server or VM can't synchronized accuracy due to current method. Although VM of some higher versions of CentOS or Fedora can achieve the accuracy of 1ms, if the network can provide better NTP connections, the result will be better. In the future, we may consider some other ways to have a better synchronization of the time to improve the accuracy of the test. 3.4. CPU The operation of DUT (VSwitch) can increase the CPU load of host server. While we find that the performance of the VSwitch and tester measure the index below: a. CPU type We test the VSwitch performance based on two servers which CPU are Intel E-2430 and Intel E-2620. At 1518 bytes, E-2620 can throughput 5.46Gb/s while E-2430 is 3.66Gb/s. And the E-2620 consume 5195 Mhz CPU and E2430 is 4957 Mhz. The better type of CPU can generate more traffic with almost same CPU consumption. b.vCPU allocation for tester Because of the tester is a VM, the allocation of vCPU for the tester may affect the performance. Besides, the vCPU can be allocated for VM like tester, but it can't allocated for the DUT (VSwitch). The reason is the VSwitch is established in the hypervisor which is a globe supervisory control for all the resource in the host server. Vic Liu Expires January 5, 2015 [Page 8] Internet-Draft PS for VxLAN Performance July 2014 The table below show the throughput performance of different vCPU allocation on one server. ---------------------------------- | Byte| 1*vCPU | 2*vCPU | 8*vCPU | ---------------------------------- | 0 | 0 | 0 | 0 | ---------------------------------- | 128 | 0.40 | 0.46 | 0.46 | ---------------------------------- | 256 | 0.66 | 0.84 | 0.84 | ---------------------------------- | 512 | 1.43 | 1.56 | 1.56 | ---------------------------------- | 1024| 2.62 | 2.88 | 2.88 | ---------------------------------- | 1518| 3.66 | 4.00 | 4.00 | ---------------------------------- 3.5. Number of test port We need test port to establish test flow. The test port is locate as a vNIC on the tester. As we know the tester is a VM which has resource limitation. We realized that we can add more vNIC or tester VM to measure the test performance. The table below show the performance of different test case: ------------------------------------------------------------------ | Byte| 1*n 1*t | 2*n 1*t | 4*n 1*t | 1*n 2*t | 2*n 2*t | 4*n 2*t | ------------------------------------------------------------------ | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ------------------------------------------------------------------ | 128 | 0.46 | 0.65 | 0.65 | 0.93 | 1.01 | 0.97 | ------------------------------------------------------------------ | 256 | 0.84 | 1.03 | 1.01 | 1.47 | 1.68 | 1.51 | ------------------------------------------------------------------ | 512 | 1.56 | 2.16 | 2.16 | 2.77 | 3.68 | 3.08 | ------------------------------------------------------------------ | 1024| 2.88 | 4.22 | 4.24 | 4.93 | 6.67 | 6.18 | ------------------------------------------------------------------ | 1518| 4.00 | 6.22 | 6.08 | 6.84 | 9.45 | 9.53 | ------------------------------------------------------------------ Vic Liu Expires January 5, 2015 [Page 9] Internet-Draft PS for VxLAN Performance July 2014 In the table, the n represent for number of vNIC and t represent for number of tester VM. There are 6 test case show that the throughput of one tester VM with 1/2/4 vNICs and two tester VM with 1/2/4 vNICs. We can see that as the number of test port increase the throughput also increased. This will affect the test result of DUT. 4. Security Considerations 5. IANA Considerations 6. Conclusions 7. References 7.1. Normative References [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [2] Crocker, D. and Overell, P.(Editors), "Augmented BNF for Syntax Specifications: ABNF", RFC 2234, Internet Mail Consortium and Demon Internet Ltd., November 1997. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2234] Crocker, D. and Overell, P.(Editors), "Augmented BNF for Syntax Specifications: ABNF", RFC 2234, Internet Mail Consortium and Demon Internet Ltd., November 1997. 7.2. Informative References [3] Faber, T., Touch, J. and W. Yue, "The TIME-WAIT state in TCP and Its Effect on Busy Servers", Proc. Infocom 1999 pp. 1573- 1583. [Fab1999] Faber, T., Touch, J. and W. Yue, "The TIME-WAIT state in TCP and Its Effect on Busy Servers", Proc. Infocom 1999 pp. 1573-1583. 8. Acknowledgments Vic Liu Expires January 5, 2015 [Page 10] Internet-Draft PS for VxLAN Performance July 2014 This document was prepared using 2-Word-v2.0.template.dot. Authors' Addresses Vic Liu China Mobile 32 Xuanwumen West Ave, Beijing, China Email: liuzhiheng@chinamobile.com Lingli Deng China Mobile 32 Xuanwumen West Ave, Beijing, China Email: denglingli@chinamobile.com Dapeng Liu China Mobile 32 Xuanwumen West Ave, Beijing, China Email: liudapeng@chinamobile.com Vic Liu Expires January 5, 2015 [Page 11]