Internet Engineering Task Force S. Tsuchiya, Ed. Internet-Draft M. Townsley Intended status: Informational Cisco Systems Expires: May 17, 2012 S. Ohkubo Sakura Internet November 14, 2011 IPv6 Rapid Deployment (6rd) in a Large Data Center draft-sakura-6rd-datacenter-03 Abstract IPv6 Rapid Deployment (6rd) as defined in RFC 5969 focuses on rapid deployment of IPv6 by an access service provider which has difficulty deploying native IPv6. This document describes how 6rd can be used to deliver IPv6 within a Large Data Center. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on May 17, 2012. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as Tsuchiya, et al. Expires May 17, 2012 [Page 1] Internet-Draft 6rd in an LDC November 2011 described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Network Architecture . . . . . . . . . . . . . . . . . . . . . 3 3. 6rd Availability in Server Operating Systems . . . . . . . . . 5 4. Deployment Consideration . . . . . . . . . . . . . . . . . . . 6 4.1. IPv4 compression address . . . . . . . . . . . . . . . . . 6 4.2. Configuration . . . . . . . . . . . . . . . . . . . . . . 6 4.3. MTU consideration . . . . . . . . . . . . . . . . . . . . 6 4.4. Performance . . . . . . . . . . . . . . . . . . . . . . . 7 5. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 7 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 7. Security Considerations . . . . . . . . . . . . . . . . . . . 7 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 7 8.1. Normative References . . . . . . . . . . . . . . . . . . . 7 8.2. Informative References . . . . . . . . . . . . . . . . . . 7 Appendix A. Additional Stuff . . . . . . . . . . . . . . . . . . 9 A.1. OS configuration . . . . . . . . . . . . . . . . . . . . . 9 A.1.1. Network Topology&Parameters . . . . . . . . . . . . . 9 A.1.2. configuration procedure . . . . . . . . . . . . . . . 10 A.2. OS Proportion on Sakura's VPS . . . . . . . . . . . . . . 13 A.3. Apache Bench performance test result . . . . . . . . . . . 13 A.3.1. Test Topology . . . . . . . . . . . . . . . . . . . . 13 A.3.2. Test method . . . . . . . . . . . . . . . . . . . . . 14 A.3.3. Hardware Configuration . . . . . . . . . . . . . . . . 14 A.3.4. Test Result . . . . . . . . . . . . . . . . . . . . . 14 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 14 Tsuchiya, et al. Expires May 17, 2012 [Page 2] Internet-Draft 6rd in an LDC November 2011 1. Introduction IPv6 Rapid Deployment (6rd) as defined in RFC 5969 focuses on rapid deployment of IPv6 by an access service provider which has difficulty deploying native IPv6. This document describes how one service provider in Japan, Sakura Interent, Inc., not for a large residential deployment, but for a large data center network. While the protocol mechanism of 6rd is unchanged, the deployment model varies a bit from the classical "residential home access provider" model. The motivation for using 6rd is very similar to that of the residential case where the service provider would like to offer IPv6 quickly to those users who want it, but without replacing equipment that currently does not support IPv6. This document is provided as information to the Internet community. 2. Network Architecture The case study presented here is based on the services provide by Sakura Internet Inc. Sakura Internet provides Internet services through Internet backbones and large data centers. Sakura offers four types of services: 1. Housing Service, which provides Collocation and Internet Access on 5 urban datacenters (4 in Tokyo, 1 in Osaka) 2. Hosting Service, which provides shared service on the servers. 3. Dedicated Server Service, which provides customer dedicated server with variable OSs. 4. Virtual Private Server Service (VPS), which provides guest operating system on the Kernel-based Virtual Machine (KVM). At the time of this writing, Sakura serves more than 200 Gpbs of traffic on its backbones, and around 50,000 dedicated servers, Virtual Private Servers, and collocated servers. Figure.1 describes server-based 6rd in datacenter's network architecture. Tsuchiya, et al. Expires May 17, 2012 [Page 3] Internet-Draft 6rd in an LDC November 2011 .--. _(. `) _( IPv6 `)_ ( Internet `) ( ` . ) ) `--(_______)---' | +----------+ | 6rd BR | +----------+ | +----------+ .--. |6rd CE on | _(. `) |guest OS | _( IPv4 `)_ +----------+ ( Backbone `) +----------+ | KVM |====( ` . ) )====|6rd CE on | +----------+ `--(_______)---' |dedicated | || |server | || +----------+ +----------+ |6rd CE on | |Housing GW| +----------+ | -------------IPv4/IPv6 dual stack | | [Server] [Server] Figure 1 Sakura has deployed commercial 6rd Border Relays, and relies on CE functionality in gateway routers or directly within the operating system of the servers. In the latter model, there is no need for a CE gateway as the 6rd function is implemented directly in the server operating system itself. -For Housing users, there are two options. Either the 6rd CE function is performed on the Gateway router itself, or the servers themselves can run 6rd directly. -For Hosting users, IPv6 service can start by deploying 6rd CE function on the server OS or guest OS on the KVM. Server administrators can start IPv6 service on demand themselves by using server-based 6rd. Tsuchiya, et al. Expires May 17, 2012 [Page 4] Internet-Draft 6rd in an LDC November 2011 There were some issues when Sakura considers IPv6 deployment on their backbone. 1.Some backbone Layer3 switches are too old. IPv6 Switching would be software switching even if IPv4 Switching in hardware. It needs replacement. 2.Some backbone Layer3 switches required software upgrade. IPv6 supports on hardware. But software upgraded is needed. In datacenter, there is different requirement on each server, even if the server connected to the same switch. Because the server administrator are completely different. Each server is providing different service to the different customers. So backbones maintenance time negotiation to the customer is very difficult. 3.Design Consideration There are some scalability issues within the large datacenter.Especially backbone Layer 3 switches has huge arp and IPv4 routing entries.When the switch would upgrade and enable IPv6,NDP entry will be added,many resource such as NVRAM and TCAM will be consumed. IPv6 datacenter also has security issues such as [draft-ietf-v6ops-v6nd-problems] and [RFC6104]. To provide native IPv6 service to the existing customer today rapidly, it needs cost,time,negotiation with customer and design consideration. This is the reason why Sakura decided to provide server-based 6rd to the existing customer. 3. 6rd Availability in Server Operating Systems In particular for the server-initiated case, Sakura relies on 6rd availability in Server operating systems. Linux kernel has started to support 6rd since 2.6.33. So if Linux based Operating Systems are using 2.6.33 and the later,it can provides server-based 6rd. FreeBSD and CentOS could not provide 6rd in default, but the patch exist. Tsuchiya, et al. Expires May 17, 2012 [Page 5] Internet-Draft 6rd in an LDC November 2011 +----------------------+---------------+----------------------------+ | Operating Systems | Linux Kernel | Description | +----------------------+---------------+----------------------------+ | Fedora14 and the | 2.6.35 and | Server-based 6rd ready | | later | above | | | Ubuntu 10.10 and the | 2.6.35 and | Server-based 6rd ready | | later | above | | | Debian6.0 | 2.6.32 | Kernel update needs | | CentOS5.6 | 2.6.18 | needs [CentOS | | | | patch1][CentOS patch2] | | FreeBSD8 | N/A | needs [BSD patch] | +----------------------+---------------+----------------------------+ 4. Deployment Consideration 4.1. IPv4 compression address 6rd protocol specification is defined on [RFC5969]. Section 4 of [RFC5969] describes o-bit which can compress 32 bit IPv4 address in the 6rd delegated prefix. Linux Kernel also supports this feature. So customer could get some IPv6 prefixes even if datacenter's prefix is /32. But [BSD patch] doesn't has the feature of aggregate IPv4 address, therefore datacenter provider has to prepare /32 IPv6 prefix at least in that case. In Sakura's case, 6rd prefix address using /32, and no compression IPv4 address.Thus the delegated 6rd address length is /64. It is enough address space for server-based 6rd. 4.2. Configuration Section 7.1 of [RFC5969] describes 6rd CE automatic configuration method such as DHCP,TR-69 and so on. But server-based 6rd does not needs automatic configuration because the server usually configure IPv4 address statically. 4.3. MTU consideration Section 9.1 of [RFC5969] describes about Maximum Transmission Unit(MTU) on 6rd tunnel. This guide also applicable for server-based 6rd. But datacenter's IPv4 network is well-managed and is known by the Tsuchiya, et al. Expires May 17, 2012 [Page 6] Internet-Draft 6rd in an LDC November 2011 server administrator. So 6rd CE's tunnel MTU could set be -20 byte from IPv4 MTU. If the 6rd CE would be TCP server such as WWW, TCP MSS(Maximum Segment Size will be calculated automatically from tunnel MTU. 4.4. Performance 6rd is tunnel technology, therefore the problem of performance is often taken up.It depends on server spec,but [ApacheBench] showed similar result with IPv4,IPv6 and 6rd.In Sakura internet server service,interface specifications are shared 100Mbps for VPS server,1Gbps for dedicated server. The result is enough to provide to customer. 5. Acknowledgements The authors thank Hiroki Sato and Masakazu Asama,who made BSD&CentOS patch. 6. IANA Considerations This document has no actions for IANA. 7. Security Considerations This document has no security considerations. 8. References 8.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC5969] Townsley, W. and O. Troan, "IPv6 Rapid Deployment on IPv4 Infrastructures (6rd) -- Protocol Specification", RFC 5969, August 2010. 8.2. Informative References [ApacheBench] ""ab - Apache HTTP server benchmarking tool"", . Tsuchiya, et al. Expires May 17, 2012 [Page 7] Internet-Draft 6rd in an LDC November 2011 [BSD patch] ""BSD patch"", . [CentOS] ""The Community ENTerprise Operating System"", . [CentOS patch1] ""CentOS Kernel patch"", . [CentOS patch2] ""CentOS iproute patch"", . [Debian] ""Debian -- The Universal Operating System"", . [Fedora] ""Fedora Project Homepage"", . [FreeBSD] ""The FreeBSD Project"", . [I-D.ietf-armd-problem-statement] Narten, T., "Problem Statement for ARMD", draft-ietf-armd-problem-statement-00 (work in progress), October 2011. [I-D.ietf-v6ops-v6nd-problems] Kumari, W., "Operational Neighbor Discovery Problems", draft-ietf-v6ops-v6nd-problems-00 (work in progress), October 2011. [Linux 2.6.33] ""sit: 6rd (IPv6 Rapid Deployment) Support."", . [RFC3849] Huston, G., Lord, A., and P. Smith, "IPv6 Address Prefix Reserved for Documentation", RFC 3849, July 2004. [RFC5569] Despres, R., "IPv6 Rapid Deployment on IPv4 Infrastructures (6rd)", RFC 5569, January 2010. [RFC5737] Arkko, J., Cotton, M., and L. Vegoda, "IPv4 Address Blocks Reserved for Documentation", RFC 5737, January 2010. [RFC5952] Kawamura, S. and M. Kawashima, "A Recommendation for IPv6 Address Text Representation", RFC 5952, August 2010. Tsuchiya, et al. Expires May 17, 2012 [Page 8] Internet-Draft 6rd in an LDC November 2011 [RFC6104] Chown, T. and S. Venaas, "Rogue IPv6 Router Advertisement Problem Statement", RFC 6104, February 2011. [Ubuntu] ""Ubuntu Homepage"", . Appendix A. Additional Stuff A.1. OS configuration A.1.1. Network Topology&Parameters Describes configuration of each on OS,for reference. .--. _(. `) _( IPv6 `)_ ( Internet `) ( ` . ) ) `--(_______)---' | +----------+ | 6rd BR | +----------+ | 203.0.113.1 .--. _(. `) _( IPv4 `)_ ( Backbone `) 6rd prefix:2001:db8::/32 ( ` . ) ) `--(_______)---' | -------------------------------------------- 203.0.113.0/24 |.10 |.11 |.12 |.13 |.14 [Ubuntu] [Fedora] [Debian] [CentOS] [FreeBSD] Figure 2 common parameter +-----------------+---------------+-------------+ | BR IPv4 address | 6rd prefix | IPv4MaskLen | +-----------------+---------------+-------------+ | 203.0.113.1 | 2001:db8::/32 | 0 | +-----------------+---------------+-------------+ Tsuchiya, et al. Expires May 17, 2012 [Page 9] Internet-Draft 6rd in an LDC November 2011 individual parameter +-----------+--------------+-------------------------+ | OS | IPv4 address | 6rd delegated prefix | +-----------+--------------+-------------------------+ | [Ubuntu] | 203.0.113.10 | 2001:db8:cb00:710a::/64 | | [Fedora] | 203.0.113.11 | 2001:db8:cb00:710b::/64 | | [Debian] | 203.0.113.12 | 2001:db8:cb00:710c::/64 | | [CentOS] | 203.0.113.13 | 2001:db8:cb00:710d::/64 | | [FreeBSD] | 203.0.113.14 | 2001:db8:cb00:710e::/64 | +-----------+--------------+-------------------------+ A.1.2. configuration procedure A.1.2.1. Ubuntu -modify "/etc/network/interfaces" # vi /etc/network/interfaces auto tun6rd iface tun6rd inet6 v4tunnel address 2001:db8:cb00:710a::1 netmask 32 local 203.0.113.10 endpoint any gateway ::203.0.113.1 ttl 64 up ip tunnel 6rd dev tun6rd 6rd-prefix 2001:db8::/32 up ip link set mtu 1280 dev tun6rd -reboot A.1.2.2. Fedora -make "/etc/sysconfig/network-scripts/ifcfg-sit1" # vi /etc/sysconfig/network-scripts/ifcfg-sit1 DEVICE=sit1 IPV6INIT=yes IPV6_MTU=1280 IPV6_DEFAULTGW=::203.0.113.1 IPV6TUNNELIPV4=any IPV6TUNNELIPV4LOCAL=203.0.113.11 IPV6ADDR=2001:db8:cb00:710b::1/32 -modify "/etc/rc.local" Tsuchiya, et al. Expires May 17, 2012 [Page 10] Internet-Draft 6rd in an LDC November 2011 # vi /etc/rc.local ip tunnel 6rd dev sit1 6rd-prefix 2001:db8::/32 -reboot A.1.2.3. Debian The latest version of Debian is 6.0.Debian6.0's kernel is 2.6.32.So it is required upgrade kernel. -modify "/etc/apt/sources.list" # vi /etc/apt/sources.list deb http://ftp.jp.debian.org/debian experimental main deb-src http://ftp.jp.debian.org/debian experimental main -upgrade kernel # apt-get update # apt-get -t experimental install linux-image-2.6.38-rc6-amd64 -reboot -modify "/etc/network/interfaces" # vi /etc/network/interfaces auto tun6rd iface tun6rd inet6 v4tunnel address 2001:db8:cb00:710c::1 netmask 32 local 203.0.113.12 endpoint any gateway ::203.0.113.1 ttl 64 up ip tunnel 6rd dev tun6rd 6rd-prefix 2001:db8::/32 up ip link set mtu 1280 dev tun6rd -reboot A.1.2.4. CentOS The latest version of CentOS is 5.5.CentOS5.5's kernel and iproute package does not supported 6rd.So it is required patch. -download package # wget http://enog.jp/~masakazu/6rd/kernel-2.6.18-238.9.1.el5.6rd.x86_64.rpm # wget http://enog.jp/~masakazu/6rd/iproute-2.6.18-11.6rd.x86_64.rpm Tsuchiya, et al. Expires May 17, 2012 [Page 11] Internet-Draft 6rd in an LDC November 2011 -install package # rpm -ivh kernel-2.6.18-238.9.1.el5.6rd.x86_64.rpm # rpm -Uvh iproute-2.6.18-11.6rd.x86_64.rpm -modify "/etc/yum.conf" # vi /etc/yum.conf exclude=kernel*,iproute -modify "/etc/sysconfig/network-scripts/ifcfg-sit1" # vi /etc/sysconfig/network-scripts/ifcfg-sit1 DEVICE=sit1 IPV6INIT=yes IPV6_MTU=1280 IPV6_DEFAULTGW=::203.0.113.1 IPV6TUNNELIPV4=any IPV6TUNNELIPV4LOCAL=203.0.113.13 IPV6ADDR=2001:db8:cb00:710d::1/32 modify "/etc/rc.local" # vi /etc/rc.local ip tunnel 6rd dev sit1 6rd-prefix 2001:db8::/32 -reboot A.1.2.5. FreeBSD FreeBSD does not support 6rd yet.But the patch exists. -download patch # cd /root # fetch http://people.allbsd.org/~hrs/FreeBSD/stf_6rd_20100923-1.diff -apply patch # cd /usr/src # patch -p0 < /root/stf_6rd_20100923-1.diff -kernel module compile and install # cd sys/modules/if_stf/ # make Tsuchiya, et al. Expires May 17, 2012 [Page 12] Internet-Draft 6rd in an LDC November 2011 # make install -install manual # cd /usr/src/share/man/ # make # make install -modify "/etc/rc.conf" # vi /etc/rc.conf ipv6_enable="YES" cloned_interfaces="stf0" ipv6_ifconfig_stf0="2001:db8:cb00:710e::1/32" ipv6_defaultrouter="2001:db8:cb00:7101::1" -reboot A.2. OS Proportion on Sakura's VPS The data of OS proportion on Sakura's VPS. All of OSs could server-based 6rd. +-------------------+---------------+ | Operating Systems | Proportion[%] | +-------------------+---------------+ | Ubuntu | 31 | | Fedora | 6 | | Debian | 13 | | CentOS | 39 | | FreeBSD | 11 | +-------------------+---------------+ A.3. Apache Bench performance test result A.3.1. Test Topology Measurement method of http performance using [ApacheBench]. Two servers connected to each other and excute [ApacheBench] from Client Tsuchiya, et al. Expires May 17, 2012 [Page 13] Internet-Draft 6rd in an LDC November 2011 Figure 3 A.3.2. Test method Two servers connected to each other. Configure transport IPv4,IPv6 and 6rd. Executes [ApacheBench] with 100 multiple and total 1000 requests from Client A.3.3. Hardware Configuration In this test cases,harware configuration is below. +-----------------------+----------------+---------------+-------+ | CPU | Hyper-V | OS | NIC | +-----------------------+----------------+---------------+-------+ | Intel Xeon3060 2.4GHz | VMware ESX 4.1 | Ubuntsu 10.10 | E1000 | +-----------------------+----------------+---------------+-------+ A.3.4. Test Result +----------------------------+----------+----------+----------+ | test item | IPv4 | IPv6 | 6rd | +----------------------------+----------+----------+----------+ | Complete requests | 1000 | 1000 | 1000 | | Failed requests | 0 | 0 | 0 | | Requests per second[#/sec] | 1.11 | 1.10 | 1.05 | | Time per request[ms] | 900.598 | 912.586 | 951.868 | | Transfer rate[Kbytes/sec] | 113702.5 | 112208.9 | 107578.2 | +----------------------------+----------+----------+----------+ Authors' Addresses Shishio Tsuchiya (editor) Cisco Systems Shinjuku Mitsui Building, 2-1-1, Nishi-Shinjuku Shinjuku-Ku, Tokyo 163-0409 Japan Phone: +81 3 6434 6543 Email: shtsuchi@cisco.com Tsuchiya, et al. Expires May 17, 2012 [Page 14] Internet-Draft 6rd in an LDC November 2011 Mark Townsley Cisco Systems L'Atlantis, 11, Rue Camille Desmoulins ISSY LES MOULINEAUX ILE DE FRANCE 92782 FRANCE Phone: +33 15 804 3483 Email: mark@townsley.net Shuichi Ohkubo Sakura Internet 33F Sumitomo fudosan Nishi shinjuku Bldg.,7-20-1 Nishi shinjuku Shinjuku-Ku, Tokyo 160-0023 Japan Phone: +81 3 5332 7070 Email: ohkubo@sakura.ad.jp Tsuchiya, et al. Expires May 17, 2012 [Page 15]