Internet DRAFT - draft-liu-dclc-gap-virtual-test

draft-liu-dclc-gap-virtual-test



Network Working Group                                         Vic Liu
Internet Draft                                            Lingli Deng
Intended status: Informational                             Dapeng Liu
                                                          China Mobile
Expires: January 2015                                     July 5, 2014



                 Gap Analysis on Virtualized Network Test
                  draft-liu-dclc-gap-virtual-test-00.txt




Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79. This document may not be modified,
   and derivative works of it may not be created, and it may not be
   published except as an Internet-Draft.

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79. This document may not be modified,
   and derivative works of it may not be created, except to publish it
   as an RFC and to translate it into languages other than English.

   This document may contain material from IETF Documents or IETF
   Contributions published or made publicly available before November
   10, 2008. The person(s) controlling the copyright in some of this
   material may not have granted the IETF Trust the right to allow
   modifications of such material outside the IETF Standards Process.
   Without obtaining an adequate license from the person(s) controlling
   the copyright in such materials, this document may not be modified
   outside the IETF Standards Process, and derivative works of it may
   not be created outside the IETF Standards Process, except to format
   it for publication as an RFC or to translate it into languages other
   than English.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other documents



Vic Liu                Expires January 5, 2015                [Page 1]

Internet-Draft        PS for VxLAN Performance               July 2014


   at any time.  It is inappropriate to use Internet-Drafts as
   reference material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

   This Internet-Draft will expire on January 5, 2009.

Copyright Notice

   Copyright (c) 2014 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with
   respect to this document. Code Components extracted from this
   document must include Simplified BSD License text as described in
   Section 4.e of the Trust Legal Provisions and are provided without
   warranty as described in the Simplified BSD License.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with
   respect to this document.

Abstract

   In virtual network, the VSwitch become an important element in the
   network. It is in charge for both forward management and virtual
   network function. We figure out that to evaluate the performance of
   VSwitch is not like the test on hardware (like using RFC2544). This
   draft introduce the gap of testing for virtual network performance.
   On chapter 2 we introduce the test practices which is the virtual
   test bad setup and benchmark test. On chapter three, we analysis the
   gap of virtual network performance test.






Vic Liu                Expires January 5, 2015                [Page 2]

Internet-Draft        PS for VxLAN Performance               July 2014


Table of Contents


   1. Introduction ................................................ 3
   2. Virtual network test......................................... 3
      2.1. Test Bed Setup ......................................... 3
      2.2. Benchmark test on virtualized network .................. 6
   3. Gap analysis for virtual network test ....................... 7
      3.1. Test methodology ....................................... 7
      3.2. Throughput ............................................. 7
      3.3. Latency ................................................ 8
      3.4. CPU .................................................... 8
      3.5. Number of test.......................................... 9
   4. Security Considerations..................................... 10
   5. IANA Considerations ........................................ 10
   6. Conclusions ................................................ 10
   7. References ................................................. 10
      7.1. Normative References................................... 10
      7.2. Informative References................................. 10
   8. Acknowledgments ............................................ 10

1. Introduction

   In virtual network, the VSwitch become an important element in the
   network. It is in charge for both forward management and virtual
   network function. We figure out that to evaluate the performance of
   VSwitch is not like the test on hardware (like using RFC2544). This
   draft introduce the gap of testing for virtual network performance.
   On chapter 2 we introduce the test practices which is the virtual
   test bad setup and benchmark test. On chapter three, we analysis the
   gap of virtual network performance test.

2. Virtual network test practices

  2.1. Test Bed Setup

   The test bed is constituted by two physical server with 10GE NIC, a
   test center, a 10GE TOR switch for test traffic and a 1GE TOR switch
   for management.









Vic Liu                Expires January 5, 2015                [Page 3]

Internet-Draft        PS for VxLAN Performance               July 2014




                      ----------------------
                     |Test Center PHY 10GE*2|
                      ----------------------
                               ||
                               ||
                          ----------
                    =====| 10GE TOR |=======
                   ||     ----------       ||
                   ||                      ||
                   ||                      ||
          -------------------      -------------------
         |   --------------  |    |   --------------  |
         |  |V-switch(VTEP)| |    |  |V-switch(VTEP)| |
         |   --------------  |    |   --------------  |
         |      |       |    |    |      |       |    |
         |   -----     ----- |    |   -----     ----- |
         |  |TCVM1|   |TCVM2||    |  |TCVM1|   |TCVM2||
         |   -----     ----- |    |   -----     ----- |
          -------------------      -------------------
              Server1                   Server2


























Vic Liu                Expires January 5, 2015                [Page 4]

Internet-Draft        PS for VxLAN Performance               July 2014


   Two Dell server are R710XD (CPU: E5-2460) and R710 (CPU: E5-2430)
   with a pair of 10GE NIC. And in the server we allocate 2 vCPU and 8G
   memory to each Test Center Virtual Machine (TCVM).

   In traffic model A: We use a physical test center connect to each
   server to verify the benchmark of each server.

                         ----------------------
                        |Test Center PHY 10GE*2|
                         ----------------------
                                   ||
                                   ||
                           -------------------
                          |   --------------  |
                          |  |V-switch(VTEP)| |
                          |   --------------  |
                          |      |       |    |
                          |   -----     ----- |
                          |  |TCVM1|   |TCVM2||
                          |   -----     ----- |
                           -------------------
                               Server1

   In traffic model B: We use the benchmark to test the performance of
   VxLAN.

                          ----------
                    =====| 10GE TOR |=======
                   ||     ----------       ||
                   ||                      ||
                   ||                      ||
          -------------------      -------------------
         |   --------------  |    |   --------------  |
         |  |V-switch(VTEP)| |    |  |V-switch(VTEP)| |
         |   --------------  |    |   --------------  |
         |      |       |    |    |      |       |    |
         |   -----     ----- |    |   -----     ----- |
         |  |TCVM1|   |TCVM2||    |  |TCVM1|   |TCVM2||
         |   -----     ----- |    |   -----     ----- |
          -------------------      -------------------
              Server1                   Server2








Vic Liu                Expires January 5, 2015                [Page 5]

Internet-Draft        PS for VxLAN Performance               July 2014


  2.2. Benchmark test on virtualized network

   The reason we need a benchmark test is we realized that the
   virtualized network is different from physical network device. We
   cannot use test methodology like RFC 2544. The performance is not
   linear growth with traffic we generate. It has an inflection point.

   We allocate the Test Center Virtual Machine (TCVM) with one v CPU
   and 4G memory to get the benchmark, we use traffic model A and get
   the result table below:

   Server 1: CPU E5-2430

    ------------------------------------------------------------
   | Byte| Rate(GE)| Server CPU MHZ |Server Mem| VM CPU| VM Mem|
    ------------------------------------------------------------
   |  0  |    0    |       495       |   2817   |  370  |  495  |
    ------------------------------------------------------------
   | 128 |  0.40   |      4704       |   2817   | 4541  |  495  |
    ------------------------------------------------------------
   | 256 |  0.66   |      4830       |   2824   | 4519  |  495  |
    ------------------------------------------------------------
   | 512 |  1.43   |      5161       |   2818   | 4870  |  495  |
    ------------------------------------------------------------
   | 1024|  2.62   |      5131       |   2819   | 4782  |  495  |
    ------------------------------------------------------------
   | 1518|  3.66   |      4957       |   2820   | 4585  |  495  |
    ------------------------------------------------------------
   Server 2: CPU E5-2620

   ------------------------------------------------------------
   | Byte| Rate(GE)| Server CPU MHZ |Server Mem| VM CPU| VM Mem|
    ------------------------------------------------------------
   |  0  |    0    |       188       |   2712   |   59  |  493  |
    ------------------------------------------------------------
   | 128 |  0.46   |      4648       |   2703   | 4100  |  494  |
    ------------------------------------------------------------
   | 256 |  0.70   |      4552       |   2704   | 4077  |  493  |
    ------------------------------------------------------------
   | 512 |  1.50   |      4521       |   2703   | 3958  |  493  |
    ------------------------------------------------------------
   | 1024|  3.10   |      4559       |   2702   | 3938  |  493  |
    ------------------------------------------------------------
   | 1518|  5.46   |      5195       |   2704   | 4377  |  493  |
    ------------------------------------------------------------




Vic Liu                Expires January 5, 2015                [Page 6]

Internet-Draft        PS for VxLAN Performance               July 2014


 3. Gap analysis for virtual network test

  3.1. Test methodology

   There is a huge difference between testing on hardware network and
   virtual network. On virtual network, the Device under Test (DUT) is
   the VSwitch or hypervisor (or the protocol such as VxLAN
   encapsulation by VSwitch) while the test machine which is a VM
   connect to the DUT shares the same resources of the physical server.
   In virtual network, tester and the DUT (visual switches) are in one
   server (physically converged), so the CPU and MEM share the same
   resources. Theoretically, the tester's operation may has some
   influences on the DUT's performances. However, for the specialty of
   virtualization, this method is the only way to assess the truth of
   assessment method.

  3.2. Throughput

   The throughput generate to test is also different from hardware
   switch (for example, RFC 2544). We realize that the throughput is
   very hard to reach the line rate by the TCVM. There is an inflection
   point as the CPU over loaded. For example, if we generate 1Gb
   traffic, it will 100% receive. But if we generate 10Gb traffic, it
   can only generate 530Mb because CPU is overloaded.

   Besides, the generate traffic is related to the packet length. For
   example, 128bit can only generate 0.4Gb traffic and 1518bit can
   generate 4Gb traffic. The test concept on traditional physical
   switches is not apply to virtual switch test. In traditional
   throughput, the capability of the switch can reach line rate at any
   bytes while virtual network cannot.

   Under the background of existing technology, when we mean to test
   the virtual switch's throughput, the concept of traditional physical
   switch will not be applicable. The traditional throughput indicates
   the switches' largest transmit capability, for certain selected
   bytes and selected cycle under zero-packet-lose conditions. But in
   virtual environment, the fluctuant of performance on virtual network
   will be much greater than dedicated physical devices. In the same
   time, because the DUT and the tester cannot be separated, which only
   proved the DUT realize same network performances under certain
   circumstances, it also means the DUT may achieve higher capability.

   Therefore, we change the throughout in virtual environment to actual
   throughput, hoping in future, as the improvement of technique, the
   actual throughput will approach the theoretical throughput
   gradually.


Vic Liu                Expires January 5, 2015                [Page 7]

Internet-Draft        PS for VxLAN Performance               July 2014


   Of course, under actual condition, this throughout have certain
   referential meanings. In most cases, common throughput application
   cannot compare with professional tester, so for virtual application
   and data center's deployment, the actual throughout already have
   great refinance value.



  3.3. Latency

   Physical tester's time reference from its own clock or other time
   source, such as GPS, which can achieve the accuracy of 10ns. In
   virtual network circumstances, the virtual tester gets its reference
   time from Linux systems. But the clock on Linux of different server
   or VM can't synchronized accuracy due to current method. Although VM
   of some higher versions of CentOS or Fedora can achieve the accuracy
   of 1ms, if the network can provide better NTP connections, the
   result will be better.

   In the future, we may consider some other ways to have a better
   synchronization of the time to improve the accuracy of the test.

  3.4. CPU

The operation of DUT (VSwitch) can increase the CPU load of host
server. While we find that the performance of the VSwitch and tester
measure the index below:

   a. CPU type

   We test the VSwitch performance based on two servers which CPU are
   Intel E-2430 and Intel E-2620.

   At 1518 bytes, E-2620 can throughput 5.46Gb/s while E-2430 is
   3.66Gb/s. And the E-2620 consume 5195 Mhz CPU and E2430 is 4957
   Mhz. The better type of CPU can generate more traffic with almost
   same CPU consumption.

   b.vCPU allocation for tester

   Because of the tester is a VM, the allocation of vCPU for the tester
   may affect the performance. Besides, the vCPU can be allocated for
   VM like tester, but it can't allocated for the DUT (VSwitch). The
   reason is the VSwitch is established in the hypervisor which is a
   globe supervisory control for all the resource in the host server.




Vic Liu                Expires January 5, 2015                [Page 8]

Internet-Draft        PS for VxLAN Performance               July 2014


   The table below show the throughput performance of different vCPU
   allocation on one server.

                  ----------------------------------
                  | Byte| 1*vCPU | 2*vCPU | 8*vCPU |
                  ----------------------------------
                  |  0  |   0    |   0    |   0    |
                  ----------------------------------
                  | 128 |  0.40  |  0.46  |  0.46  |
                  ----------------------------------
                  | 256 |  0.66  |  0.84  |  0.84  |
                  ----------------------------------
                  | 512 |  1.43  |  1.56  |  1.56  |
                  ----------------------------------
                  | 1024|  2.62  |  2.88  |  2.88  |
                  ----------------------------------
                  | 1518|  3.66  |  4.00  |  4.00  |
                  ----------------------------------




  3.5. Number of test port

   We need test port to establish test flow. The test port is locate as
   a vNIC on the tester. As we know the tester is a VM which has
   resource limitation. We realized that we can add more vNIC or tester
   VM to measure the test performance.

   The table below show the performance of different test case:

   ------------------------------------------------------------------
   | Byte| 1*n 1*t | 2*n 1*t | 4*n 1*t | 1*n 2*t | 2*n 2*t | 4*n 2*t |
   ------------------------------------------------------------------
   |  0  |    0    |    0    |    0    |    0    |    0    |    0    |
   ------------------------------------------------------------------
   | 128 |   0.46  |   0.65  |   0.65  |   0.93  |   1.01  |   0.97  |
   ------------------------------------------------------------------
   | 256 |   0.84  |   1.03  |   1.01  |   1.47  |   1.68  |   1.51  |
   ------------------------------------------------------------------
   | 512 |   1.56  |   2.16  |   2.16  |   2.77  |   3.68  |   3.08  |
   ------------------------------------------------------------------
   | 1024|   2.88  |   4.22  |   4.24  |   4.93  |   6.67  |   6.18  |
   ------------------------------------------------------------------
   | 1518|   4.00  |   6.22  |   6.08  |   6.84  |   9.45  |   9.53  |
   ------------------------------------------------------------------



Vic Liu                Expires January 5, 2015                [Page 9]

Internet-Draft        PS for VxLAN Performance               July 2014


   In the table, the n represent for number of vNIC and t represent for number of tester VM.  There are 6 test case show that the throughput of one tester VM with 1/2/4 vNICs and two  tester VM with 1/2/4 vNICs. We can see that as the number of test port increase the throughput   also increased. This will affect the test result of DUT.



 4.  Security Considerations

 5. IANA Considerations

 6. Conclusions

 7. References



  7.1. Normative References

   [1]  Bradner, S., "Key words for use in RFCs to Indicate
         Requirement Levels", BCP 14, RFC 2119, March 1997.

   [2]  Crocker, D. and Overell, P.(Editors), "Augmented BNF for
         Syntax Specifications: ABNF", RFC 2234, Internet Mail
         Consortium and Demon Internet Ltd., November 1997.

   [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
             Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC2234] Crocker, D. and Overell, P.(Editors), "Augmented BNF for
             Syntax Specifications: ABNF", RFC 2234, Internet Mail
             Consortium and Demon Internet Ltd., November 1997.

  7.2. Informative References

   [3]  Faber, T., Touch, J. and W. Yue, "The TIME-WAIT state in TCP
         and Its Effect on Busy Servers", Proc. Infocom 1999 pp. 1573-
         1583.

   [Fab1999] Faber, T., Touch, J. and W. Yue, "The TIME-WAIT state in
             TCP and Its Effect on Busy Servers", Proc. Infocom 1999
             pp. 1573-1583.

 8. Acknowledgments





Vic Liu                Expires January 5, 2015               [Page 10]

Internet-Draft        PS for VxLAN Performance               July 2014


   This document was prepared using 2-Word-v2.0.template.dot.

Authors' Addresses

   Vic Liu
   China Mobile
   32 Xuanwumen West Ave, Beijing, China

   Email: liuzhiheng@chinamobile.com

   Lingli Deng
   China Mobile
   32 Xuanwumen West Ave, Beijing, China

   Email: denglingli@chinamobile.com

   Dapeng Liu
   China Mobile
   32 Xuanwumen West Ave, Beijing, China

   Email: liudapeng@chinamobile.com



























Vic Liu                Expires January 5, 2015               [Page 11]