INTERNET-DRAFT Luyuan Fang Intended Status Rex Fernando Expires: August 25, 2013 Dhananjaya Rao Sami Boutros Cisco February 25, 2013 BGP IP VPN Data Center Interconnect draft-fang-l3vpn-data-center-interconnect-01 Abstract This document discusses solutions for inter-connection of BGP MPLS IP/VPN [RFC4364] and Data Center (DC) overlay networks. Two categories of inter-connections are discussed in this document. In the first category, Data Center overlay virtual network is built with BGP IP VPN technologies, the inter-connection of IP VPN in the Data Center to BGP IP VPN in the WAN enables end-to-end IP VPN connectivity. New Inter-AS solutions are required in certain scenarios, in addition to the existing Inter-AS Options (A, B, C) defined in [RFC4364]. In the second category, Data Centers overlay network uses non IP VPN technologies, the inter-connection of any overlay virtual network in the Data Center to BGP IP VPN in the WAN provides end user connectivity through stitching of different overlay technologies, the mapping of non IP VPN overlay to IP VPN need to be performed at the border Gateway of the two networks. The role of Software Defined Network (SDN) to assist the inter-connections is discussed. Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at Fang et al. Expires [Page 1] INTERNET DRAFT http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Copyright and License Notice Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1 Case 1: End-to-end BGP IP VPN cloud inter-connection . . . . 4 2.2 Case 2: Hybrid cloud inter-connection . . . . . . . . . . . 4 3. Architecture reference models . . . . . . . . . . . . . . . . . 5 3.1 BGP/MPLS IP VPN Inter-AS model . . . . . . . . . . . . . . . 5 3.2 BGP/MPLS IP VPN Gateway PE to DC vCE Model . . . . . . . . . 6 3.3 Hybrid inter-connection model . . . . . . . . . . . . . . . 7 4. Inter-connect IP VPN between DC and WAN . . . . . . . . . . . . 7 4.1 Existing Inter-AS options and DCI gap analysis . . . . . . . 7 4.1.1 Option A pros and cons . . . . . . . . . . . . . . . . . 7 4.1.2 Option B pros and cons . . . . . . . . . . . . . . . . . 8 4.1.3 Option C pros and cons . . . . . . . . . . . . . . . . . 9 4.1.4 Use of RTC . . . . . . . . . . . . . . . . . . . . . . . 9 4.2 Additional work discussion . . . . . . . . . . . . . . . . . 9 5. Inter-connect IP VPN and non-IP VPN overlay networks . . . . . 10 6. Security Considerations . . . . . . . . . . . . . . . . . . . . 10 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 11 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11 8.1 Normative References . . . . . . . . . . . . . . . . . . . 11 8.2 Informative References . . . . . . . . . . . . . . . . . . 11 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12 Fang et al. Expires [Page 2] INTERNET DRAFT 1 Introduction BGP/MPLS IP Virtual Private Networks (IP VPNs) [RFC4364] has been the most extensively deployed, and the most scalable Service Provider (SP) provisioned VPN solutions over the past decade. This document is centered around inter-connecting various Clouds/Data Centers to the BGP/MPLS IP VPNs. With the growth of cloud services, the needs for inter-connecting Data Centers and Enterprise BGP/MPLS IP VPNs in the Wide Area Network (WAN) become important. The interests are from multiple players: Service Providers who provide BGP/MPLS IP VPNs and may provide cloud service as well; cloud providers who provide cloud services but do not provide BGP/MPLS IP VPNs in the WAN; and the Enterprise users who use both BGP/MPLS IP VPNs services and cloud services. This document discusses use cases of the inter-connection of BGP/MPLS VPN to Data Centers, the general requirements, and the proposed solutions for the inter-connections. 1.1 Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. Term Definition ----------- -------------------------------------------------- AS Autonomous System ASBR Autonomous System Border Router BGP Border Gateway Protocol CE Customer Edge ED End device: where Guest OS, Host OS/Hypervisor, applications, VMs, and virtual router may reside GRE Generic Routing Encapsulation Hypervisor Virtual Machine Manager IaaS Infrastructure as a Service IRS Interface to Routing System LTE Long Term Evolution MP-BGP Multi-Protocol Border Gateway Protocol NVGRE Network Virtualization using GRE PCEF Policy Charging and Enforcement Function P Provider backbone router QoS Quality of Service RD Route Distinguisher RR Route Reflector RT Route Target Fang et al. Expires [Page 3] INTERNET DRAFT RTC RT Constraint SDN Software Defined Network ToR Top-of-Rack switch VI Virtual Interface vCE virtual Customer Edge Router VM Virtual Machine VN Virtual Network vPC virtual Private Cloud vPE virtual Provider Edge Router VPN Virtual Private Network vRR virtual Route Reflector1.2 Scope of the document VXLAN Virtual eXtensible Local Area Network WAN Wide Area Network 2. Use Cases 2.1 Case 1: End-to-end BGP IP VPN cloud inter-connection Many Service Providers have large deployment base of BGP/MPLS IP VPNs. They are interested in extending the same style IP VPN capabilities into their Data Centers to provide end-to-end native BGP IP VPN services to their enterprise customers. The advantage is BGP IP VPN provides better security, richer policy control and QoS support when comparing with transport through the public Internet. The technologies developed to extend IP VPN into Data Center servers or ToR are described is virtual Provider Edge (vPE) [I-D.fang-l3vpn- virtual-pe],[I-D.ietf-l3vpn-end-system]. Regardless if the WAN and DC are managed by the same administrative domain or not (other the former), inter-connecting the two VPN segments are needed. The interested parties are Service Providers and Enterprises. 2.2 Case 2: Hybrid cloud inter-connection Service Providers are interested in extending their cloud VPNs to provide opportunities for enterprise customers using the new services provided by other cloud providers. The cloud providers are interested to extend their customer base through the SP Enterprise IP VPN customers. The inter-connection between the SP BGP/MPLS IP VPNs and the cloud provider networks is needed to accomplished the goal. The inter-connection of different types of providers most likely not BGP/MPLS IP VPN inter-connections, as the cloud providers may use any type of technologies in their networks, virtualized or non- virtualized. The two inter-connecting networks are under the administrative domains of different commercial entities. The task can be more challenging than IP/MPLS IP VPN Inter-provider connections which had always been challenging not from technology point of view. Fang et al. Expires [Page 4] INTERNET DRAFT We now add the dimension to inter-connecting VPN to various different technologies. 3. Architecture reference models The architecture reference models described below are focusing on the inter-connection aspects. The intra DC implementation is not in discussion, but the intra DC technology has the direct impact to Inter-DC connection. Therefore, various models are illustrated. 3.1 BGP/MPLS IP VPN Inter-AS model The BGP/MPLS IP VPNs are implemented in both the WAN network and the Data Center. A customer VPN, for example VPNA in figure 1, consists of enterprise remote sites and VMs supporting applications in the Data Center. The IP VPN implementation is using vPE technology. The two segment of the VPNs are inter-connected through ASBRs facing each other in the respective networks. ,-----. ,-----. ( ') ( ') .--(. '.---. .-.(. '.---. ( ' ' +-----+ +-----+ ) ( IP/MPLS WAN |ASBR1|---|ASBR2| DC Network ) (. +-----+ +-----+ .) +-----+ ( .) ( ( +-----+ | PE1 |-.-' '-''--'' ''--' '-''-|vPE2 | .----.-.----. .----.-.----. |VRFA| |VRFB| |VRFA| |VRFB| '----' '----' '----' '----' / \ / \ \ +---+ +---+ .---. .---. .---. |CE1| |CE2| |VM1| |VM2| |VM3| +---+ +---+ '---' '---' '---' (VPNA) (VPNB) ( VPNA ) (VPNB) Figure 1. BGP/MPLS IP VPN Inter-Connection with ASBR in each network One boarding ASBR can be shared for the inter-connection of the two networks, especially if the WAN and DC belong to the same provider. Figure 2 illustrate this the shared ASBR model. Fang et al. Expires [Page 5] INTERNET DRAFT ,----.. ,-----. ( ') ( ') .--(. '.----. .-.(. '.---. ( ' ' +------+ ) ( IP/MPLS WAN | ASBR | DC Network ) (. +------+ .) +-----+ ( .) ( ( +-----+ | PE1 |-.-' '-''---'' ''--' '-''-|vPE2 | .----.-.----. .----.-.----. |VRFA| |VRFB| |VRFA| |VRFB| '----' '----' '----' '----' / \ / \ \ +---+ +---+ .---. .---. .---. |CE1| |CE2| |VM1| |VM2| |VM3| +---+ +---+ '---' '---' '---' (VPNA) (VPNB) ( VPNA ) (VPNB) Figure 2. BGP/MPLS IP VPN Inter-Connection with share ASBR 3.2 BGP/MPLS IP VPN Gateway PE to DC vCE Model A simple virtual CE (vCE) [I-D.fang-l3vpn-virtual-ce] model can be used to inter-connect client containers to the DC Gateway which function as PE. This model are used SPs to provide managed services, when scale can meet the service requirement. ,----.. ,-----. ( ') ( ') .--(. '.----. .----. '.----. ( ' ' +-----|VRFA| +----+ ( IP/MPLS WAN |GW/PE'----' DC Network |vCE4| (. +-----|VRFB| +----+ +-----+ ( )' '----'.( )-' | (VPNB) | PE1 |-.-' '-''- ' '--' '+----+ .---. .----.-.----. |vCE3| |VM3| |VRFA| |VRFB| (VPNA) +----+ '---' '----' '----' / \ / \ .---..---. +---+ +---+ |VM1||VM2| |CE1| |CE2| '---''---' +---+ +---+ (VPNA) (VPNB) Figure 3. BGP/MPLS IP VPN GW/PE to vCEs without BGP/MPLS IP VPN in the DC Fang et al. Expires [Page 6] INTERNET DRAFT 3.3 Hybrid inter-connection model The BGP/MPLS IP VPNs are implemented in the WAN network, and non BGP/MPLS IP VPN Overlay in DC. The connection of the two networks are outside of the technologies for Inter-AS connections for BGP IP VPNs. This model include many variations depending on the specific technologies used in the DC overlay. Figure 4 provides a general view of this inter-connecting model with ASBR on the MPLS WAN side, and the DC GW on the DC side. It is also viable to use one shared ASBR/GW for the inter-connection, especially if the WAN and the DC belong to the same provider. ,-----. ,-----. ( ') ( ') .--(. '.---. .-.(. '.---. ( ' ' +-----+ +-----+ ) ( IP/MPLS WAN |ASBR |---|DC GW| DC Network ) (. +-----+ +-----+ .) +-----+ ( .) ( ( +-----+ | PE1 |-.-' '-''--'' ''--' '-''-| NVE | .----.-.----. +-----+ |VRFA| |VRFB| / \ \ '----' '----' .---. .---. .---. / \ |VM1| |VM2| |VM3| +---+ +---+ .---. .---. .---. |CE1| |CE2| ( TenantA) (TenantB) +---+ +---+ (VPNA) (VPNB) Figure 4. BGP/MPLS IP VPN Inter-Connection with non BGP/MPLS IP VPN Overlay in DC 4. Inter-connect IP VPN between DC and WAN 4.1 Existing Inter-AS options and DCI gap analysis The inter-AS options described in [RFC4364] can be used for DC inter- connection. Option A, B, and C must be supported. 4.1.1 Option A pros and cons In Option A: back-to-back VRF. The PE-ASBR in one AS performs MPLS or IP VPN decapsulation and transmits packets to the peer PE-ASBR in the adjacent autonomous system. The peer PE-ASBR performs MPLS or IP VPN encapsulation on the customer IPv4/IPv6 packets received, and transmits the packet through the IPv4 backbone of the autonomous system. VPN service providers exchange routes across a back-to-back VRF connection. Each VRF instance represents a separate VPN client, Fang et al. Expires [Page 7] INTERNET DRAFT and it is configured on a separate PE-ASBR interface, allowing a PE- ASBR to communicate with its peer PE-ASBR as if the peer was a CE router. Pros: It is the most secure option among options A, B, and C. And it is the simplest model from operation perspective. Each PE-ASBR is treating the other as a CE. Cons: Scaling limitation, because per Inter-AS VPN VRF and interface are needed on the PE-ASBR. Option A has been used in Inter-Provider inter-connections because the security consideration and clear operational demarcation. DCI considerationa: It is a simple way to connect DC and WAN if both sides are small scale. Scale will be the major concern for DC inter- connect if large scale support is needed. Even if the DC scale is small, there are major concerns on receiving relevant routes (potentially large number) from the WAN side. 4.1.2 Option B pros and cons In Option B: EBGP redistribution of labeled VPN-IPv4/IPv6 routes between the neighboring ASes. ASes exchange VPN routing information (routes and labels) to establish connections. To control connections between autonomous systems, the PE routers and EBGP border edge routers maintain a label forwarding information base (LFIB). The LFIB manages the labels and routes that the PE routers and EBGP border edge routers receive during the exchange of VPN information. The autonomous systems use the following guidelines to exchange VPN routing information: Routing information: The destination network, the next hop field associated with the distributing router, a local MPLS label; An RD: route distinguisher; ASBRs are configured to change the next hop (next-hop-self) when sending VPN-IPv4 NLRIs to the IBGP neighbors, the ASBRs must allocate a new label when they forward the NLRI to the IBGP neighbors. Pros: It provides better scale than option A does, as it removes the needs of per Inter-AS VPN VRF and interface on the ASBR. Cons: vanilla version of Option B is considered less secure in comparison with Option A, due to dynamic routing information exchange is involved. The ASBR scaling may still be issues because ASBR must maintain all VPN routes. Option B is commonly used within single provider or for inter- provider connections. Fang et al. Expires [Page 8] INTERNET DRAFT DCI consideration: Option B is one viable option to be used in DC inter-connection. But it has the same scale concerns as other options on potential large number of routes exchange between WAN and DC. 4.1.3 Option C pros and cons In option C: Multihop eBGP Redistribution of Labeled VPN-IPv4 Routes Between Source and Destination ASs, with eBGP Redistribution of Labeled IPv4 Routes from AS to Neighboring AS. The ASBRs need only to exchange host routes (/32 or /128) to the PE routers involved in the VPN, with the labels needed to get there. A Label Switch Path (LSP) is built from the ingress PE router in one AS to the egress PE in the other AS (using loopback addresses). VPN traffic uses this LSP to reach the other AS. From data plane is perspective, the ASBRs act as P routers, with no knowledge about the VPNs concerned. Between ASBRs the VPN traffic looks like traffic between P routers: each data packet is pre-pended with the VPN label and then with an egress-PE label. Option C can be further scaled by using route reflectors (RRs) in each AS. Pros:It is the most scalable option among all. ASBR is no longer a bottle neck for VPN routes scaling as in Option B. Cons: Major security issues as IGP reachability need to be exchanged between the neighboring ASes. Option C has seen used within a single SP for inter-AS connections. Using RR for VPN routes exchange is the common approach. DCI consideration: Option C should not be used for any DCI which is between two different providers. In addition, even though ASBR is off the burden of scaling VPN routes, VRFs, VPN interfaces. The VPN routes are still exchanged between the two ASes. 4.1.4 Use of RTC RT constraint [RFC4684] function must be used to only distribute the IP VPN routes of a VPN from one AS to another under the condition that they both support that VPN in each of the AS. This is one most important function for scaling the solution. But all IP VPN routes are exchanged between the two ASes (e.g. WAN and DC) as long as they have support the same VPNs. The potential IP VPN routes distribution can still be very substantial in large WAN and DC deployment. 4.2 Additional work discussion Fang et al. Expires [Page 9] INTERNET DRAFT Focus is very large scale DCI. To be added. 5. Inter-connect IP VPN and non-IP VPN overlay networks As one significant instance of the hybrid use-case described in section 2.2, a DC may support a multi-tenant virtualized service network using IP based DC overlay encapsulations such as VXLAN [I- D.mahalingam-dutt-dcops-vxlan] or NVGRE [I-D.sridharan- virtualization-nvgre]. Different deployment models may be used within the DC depending on the DC provider's functional and operational requirements. When an IP DC overlay is terminated at the DC gGateway router and traffic directed into an MPLS IP VPN. The DC Gateway router performs MPLS encapsulation towards the WAN and IP overlay based forwarding within the DC. In this case, the inter-connection mechanisms between the DC and the WAN may fall into two categories: 1. VRF Termination The overlay based virtual network terminates into a BGP IP VPN VRF at the DC-WAN Gateway router. Both the internal routes of the DC as well as the external routes received from the WAN router can be installed in the VRF forwarding table at the DC gateway router. The DC gateway will perform an IP lookup and forward traffic after doing the appropriate MPLS or IP encapsulation. The DC gateway router will peer with the WAN router using one of the existing inter-AS mechanisms described above. The DC Gateway functions as an IP-VPN ASBR with local VRFs, for example packets still undergo an IP forwarding lookup. 2. DC-VN and IP VPN Inter-working In this case, the DC Gateway router performs a direct translation between VN-IDs and IP VPN labels while switching packets between the DC and WAN interfaces without performing an IP lookup. The forwarding table at the DC Gateway router is set up to do a VN-ID or label lookup and derive the output label or VN-ID. The DC Gateway Router acts as an Inter-AS Option B ASBR peering with other ASBRs. 6. Security Considerations To be added. Fang et al. Expires [Page 10] INTERNET DRAFT 7. IANA Considerations None. 8. References 8.1 Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack Encoding", RFC 3032, January 2001. [RFC4023] Worster, T., Rekhter, Y., and E. Rosen, Ed., "Encapsulating MPLS in IP or Generic Routing Encapsulation (GRE)", RFC 4023, March 2005. [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A Border Gateway Protocol 4 (BGP-4)", RFC 4271, January 2006. [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private Networks (VPNs)", RFC 4364, February 2006. [RFC4684] Marques, P., Bonica, R., Fang, L., Martini, L., Raszuk, R., Patel, K., and J. Guichard, "Constrained Route Distribution for Border Gateway Protocol/MultiProtocol Label Switching (BGP/MPLS) Internet Protocol (IP) Virtual Private Networks (VPNs)", RFC 4684, November 2006. [RFC5036] Andersson, L., Ed., Minei, I., Ed., and B. Thomas, Ed., "LDP Specification", RFC 5036, October 2007. 8.2 Informative References [I-D.ietf-l3vpn-end-system] Marques, P., Fang, L., Pan, P., Shukla, A., Napierala, M., "BGP-signaled end-system IP/VPNs", draft-ietf-l3vpn-end-system-00, October 2012. [I-D.fang-l3vpn-virtual-pe] Fang, L., Ward, D., Fernando, R., Fang et al. Expires [Page 11] INTERNET DRAFT Napierala, M., Bitar, N., Rao, D., Rijsman, B., So, N., "BGP IP VPN Virtual PE", draft-fang-l3vpn-virtual-pe-00, Feb. 2013. [I-D.fang-l3vpn-virtual-ce] Fang, L., Evans, J., Ward, D., Fernando, R., Mullooly, J., So, N., Bitar., N., Napierala, M., "BGP IP VPN Virtual PE", draft-fang-l3vpn-virtual-ce-01, Feb. 2013. [I-D.fang-l3vpn-end-system-req] Napierala, M., and Fang, L., "Requirements for Extending BGP/MPLS VPNs to End-Systems", draft-fang-l3vpn-end-system-requirements-00, Oct. 2012. [I-D.ward-irs-framework] Atlas, A., Nadeau, T., Ward. D., "Interface to the Routing System Framework", draft-ward-irs- framework-00, July 2012. [I-D.rfernando-irs-fw-req] Fernando, R., Medved, J., Ward, D., Atlas, A., Rijsman, B., "IRS Framework Requirements", draft- rfernando-irs-framework-requirement-00, Oct. 2012. [I-D.mahalingam-dutt-dcops-vxlan]: Mahalingam, M, Dutt, D.., et al., "A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks" draft-mahalingam-dutt-dcops-vxlan- 03, Feb. 2013. [I-D.sridharan-virtualization-nvgre]: SridharanNetwork, M., et al., "Virtualization using Generic Routing Encapsulation", draft-sridharan-virtualization-nvgre-02.txt, Feb. 2013. Authors' Addresses Luyuan Fang Cisco 111 Wood Ave. South Iselin, NJ 08830 Email: lufang@cisco.com Rex Fernando Cisco 170 W Tasman Dr San Jose, CA Email: rex@cisco.com Dhananjaya Rao Cisco 170 W Tasman Dr Fang et al. Expires [Page 12] INTERNET DRAFT San Jose, CA Email: dhrao@cisco.com Sami Boutros Cisco 170 W Tasman Dr San Jose, CA Email: dhrao@cisco.com Fang et al. Expires [Page 13]