Network Working Group XiaoLan. Wan Internet-Draft XiaoPeng. Yang Intended status: Standards Track HangZhou H3C Expires: June 9, 2012 Vishwas. Manral Alvaro. Retana HP December 7, 2011 Extending TRILL over WAN draft-xl-trill-over-wan-00.txt Abstract TRILL is a key technology for large-scale layer 2 networking within data center, most enterprises have multiple data centers in different physical sites, TRILL over WAN(ToW) provides a scalable and simple solution that interconnect multiple TRILL networks to form a single TRILL domain using the currently deployed enterprise or service provider networks. This document provides an overview of this solution. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on June 9, 2012. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents Wan, et al. Expires June 9, 2012 [Page 1] Internet-Draft Extending TRILL over WAN December 2011 carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 3. Control Plane . . . . . . . . . . . . . . . . . . . . . . . . 5 3.1. Provider Control Plane . . . . . . . . . . . . . . . . . . 5 3.2. Overlay Control Plane . . . . . . . . . . . . . . . . . . 5 4. Data Plane . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4.1. Encapsulation . . . . . . . . . . . . . . . . . . . . . . 5 4.2. Forwarding Process . . . . . . . . . . . . . . . . . . . . 8 4.2.1. Forwarding from an Internal Link to the Overlay . . . 8 4.2.2. Forwarding from the Overlay Link to an Internal Link . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.2.3. Multicast Packet Flows . . . . . . . . . . . . . . . . 8 4.2.4. Broadcast Packet Flows . . . . . . . . . . . . . . . . 8 4.2.5. Mac Address Learning . . . . . . . . . . . . . . . . . 8 4.2.6. Multi-homing . . . . . . . . . . . . . . . . . . . . . 8 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 6. Security Considerations . . . . . . . . . . . . . . . . . . . 10 6.1. Failure Separation . . . . . . . . . . . . . . . . . . . . 10 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 10 8. Normative References . . . . . . . . . . . . . . . . . . . . . 10 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 11 Wan, et al. Expires June 9, 2012 [Page 2] Internet-Draft Extending TRILL over WAN December 2011 1. Overview TRILL over WAN is a technology for supporting L2 VPNs over an L2/L3 infrastructure. It provides an "over-the-top" method of doing virtualization among several sites where the routing and forwarding state is maintained at the network edge, but not within the site or in the core. TRILL over WAN can reside in a small number of device at the edge between TRILL sites and the core, we call these devices "Joint Devices" which perform typical transit Rbridges functions(nickname- based forwarding) and perform overlay functions on their core facing interfaces. TRILL traffic which requires traversing the WAN to reach its destination, is prepended with an IP header. As shown in figure1, if a destination RBridge(Nickname) is reachable via Joint Device S2(with a core facing IP address of IPB), other Joint Devices forwarding traffic to such Rbridge will add on IP header with a destination IP address of IPB and forward the traffic into the core. The core will forward traffic based on IP address IPB, once the traffic makes it to Joint Device S2 it will be stripped of the overlay IP header and it will be forwarded into the site in the same way a regular RBridge would forward a packet. Broadcast or multicast traffic is encapsulated with a multicast header and follows a similar process. +---+ +---+IPA --------- IPB +---+ +---+ |S10|-----|S1 |----/ IP Core \----|S2 |-----|S20| +---+ ^ +---+ \ Network / +---+ ^ +---+ | --------- | | | | +------------+ | | | DA=IPB | | | +------------+ | | | SA=IPA | | +------------+ +------------+ +------------+ |Outer Eth. | |Outer Eth. | |Outer Eth. | |Header | |Header | |Header | +------------+ +------------+ +------------+ |TRILL Header| |TRILL Header| |TRILL Header| +------------+ +------------+ +------------+ |Inner Eth. | |Inner Eth. | |Inner Eth. | |Frame | |Frame | |Frame | +------------+ +------------+ +------------+ Figure 1: Traffic encapsulation Wan, et al. Expires June 9, 2012 [Page 3] Internet-Draft Extending TRILL over WAN December 2011 The key piece that TRILL over WAN adds is the state to map a given egress Nickname to an IP address of the Joint Device behind which that egress Nickname is located. TRILL over WAN forwarding is a function of mapping a destination Nickname to a Joint Device IP address in the overlay network. To achieve this, a control plane is required to exchange the reachability information among different Joint Devices. After Joint Device discovery and adjacency setup, the virtual links to neighbor Joint Devices will be treated as TRILL interface, and the TRILL's control plane runs over these virtual links. Through these steps, all the Rbridges in multiple site forms a single TRILL domain. Each Rbridge (including Joint Device) calculates its TRILL forwarding table independently. Figure 2 shows what the resulting forwarding tables look like in a simple example. +---+ L11+---+IPA --------- IPB +---+ L21+---+ |S10|----|S1 |----/ IP Core \----| S2|----|S20| +---+ +---+ \ Network / +---+ +---+ --------- S1 S2 +----------------------+ +-------------------+ |Nickname |Interface | |Nickname |Interface| +----------------------+ +-------------------+ | S10 | L11 | | S10 | IPA | +----------------------+ +-------------------+ | S20 | IPB | | S20 | L21 | +----------------------+ +-------------------+ | S1 | self | | S1 | IPA | +----------------------+ +-------------------+ | S2 | IPB | | S2 | self | +----------------------+ +-------------------+ Figure 2: Forwarding Tables TRILL over WAN supports multi-homing for sites where one or more of the Joint Devices connected to core network. It can support active- active multi-homing capability and loop elimination by nature of TRILL. No need to add other extra mechanism. Wan, et al. Expires June 9, 2012 [Page 4] Internet-Draft Extending TRILL over WAN December 2011 2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. Some ideas of this specification is being discussed on the EAI mailing list. See https://www1.ietf.org/mailman/listinfo/ima for information about subscribing. The list's archive is at http://www1.ietf.org/mail-archive/web/ima/index.html. 3. Control Plane 3.1. Provider Control Plane The provider control plane enables unicast reachability among the Joint Devices and also provides the multicast group than makes Joint Devices adjacent from the overlay control plane perspective. It also provides the multicast trees in the core that will be used for optimal forwarding of the TRILL data traffic. 3.2. Overlay Control Plane The overlay control plane provides auto-discovery of the Joint Devices that are members of an overlay VPN. The TRILL control plane traffic between Joint Devices of different sites will be carried over the virtual link. Detailed to be added. 4. Data Plane 4.1. Encapsulation The encapsulation format is TRILL frame encapsulated in UDP inside of IPv4 or IPv6. The format of the UDP IPv4 encapsulation is as follows: Wan, et al. Expires June 9, 2012 [Page 5] Internet-Draft Extending TRILL over WAN December 2011 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| IHL |Type of Service| Total Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identification |Flags| Fragment Offset | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time to Live | Protocol = 17 | Header Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source-site TRILL Joint Device IP Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination-site TRILL Joint Device (or multicast) Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port = xxxx | Dest Port = TBD | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | UDP length | UDP Checksum = 0 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |R|R|R|R|I|R|R|R| Overlay ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Instance ID | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Outer Ethernet Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | TRILL Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Inner Ethernet Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Ethernet Payload | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The format of the UDP IPv6 encapsulation is as follows: Wan, et al. Expires June 9, 2012 [Page 6] Internet-Draft Extending TRILL over WAN December 2011 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| Traffic Class | Flow Label | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Payload Length | Next Header=17| Hop Limit | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + + | | + Source-site TRILL Joint Device IPv6 Address + | | + + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + + | | + Destination-site TRILL Joint Device (or multicast) Address + | | + + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port = xxxx | Dest Port = TBD | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | UDP Length | UDP Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |R|R|R|R|I|R|R|R| Overlay ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Instance ID | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Outer Ethernet Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | TRILL Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Inner Ethernet Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Ethernet Payload | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Wan, et al. Expires June 9, 2012 [Page 7] Internet-Draft Extending TRILL over WAN December 2011 4.2. Forwarding Process 4.2.1. Forwarding from an Internal Link to the Overlay The forwarding within a site is normal TRILL forwarding, so here only describes the forwarding from an Internal link to the Overlay Link, or vice versa. A Joint Device is a transit Rbridge from TRILL point of view. When a TRILL packet is received from the internal interface, egress Nickname is used to lookup the Nickname table which will yield a next-hop IP address entry pointing to a remote Joint Device. Then the packet is encapsulated with UDP/IP header and sent over the overlay interface to destination Joint Device at Layer-3 as a regular IP packet. 4.2.2. Forwarding from the Overlay Link to an Internal Link When a packet is received on the overlay interface, it will be IP decapsulated to reveal the inner TRILL(including the outer MAC) header for forwarding. The egress Nickname will used for forwarding, the forwarding action is same as a transit RBridge. 4.2.3. Multicast Packet Flows To be added. 4.2.4. Broadcast Packet Flows To be added. 4.2.5. Mac Address Learning The TRILL edge devices learn remote MAC addresses(including the MAC addresses in other data centers) in data plane by hardware. In most cases, the Joint device is like a transit RBridge, and doesn't learn end host's MAC addresses. From DCI(Data Center Interconnect) perspective, the Joint Device is DCI device at the same time, so TRILL over WAN can relieve the pressure of MAC addresses table capability in DCI device. 4.2.6. Multi-homing In the situation of multi-homing shown as Figure 3, all the Joint Devices can be active by the nature of TRILL. Figure 4 shows what the resulting forwarding tables would look like in the multi-homing example. Wan, et al. Expires June 9, 2012 [Page 8] Internet-Draft Extending TRILL over WAN December 2011 +---+ L1 +---+ IPA -------- IPD +---+ +---+ |S10|----| S1|-------- / \ ---------|S4 |----|S21| +---+ +---+ / \ +---+ +---+ \/L2 | IP Core | \/ /\ | Networ | /\ +---+ +---+ IPB \ / IPC +---+ +---+ |S11|----|S2 |-------- \ / ---------|S3 |----|S20| +---+ +---+ -------- +---+ +---+ Figure 3: Multi-homing Scenario S1 +----------------------+ |Nickname |Interface | +----------------------+ | S1 | self | +----------------------+ | S2 | - | +----------------------+ | S3 | IPC | +----------------------+ | S4 | IPD | +----------------------+ | S10 | L1 | +----------------------+ | S11 | L2 | +----------------------+ | S20 | IPC/IPD | +----------------------+ | S21 | IPC/IPD | +----------------------+ Figure 4: Forwarding Table of S1 In S1 device, the traffic destinated to S10 and S21 have two next hops, IPC and IPD. In forwarding process, hashing of TRILL packet inner information will be used to determine which next hop IP address to use. Thus, the ingress traffic will be load balanced between multiple Joint Devices within a site. Wan, et al. Expires June 9, 2012 [Page 9] Internet-Draft Extending TRILL over WAN December 2011 5. IANA Considerations IANA need to allocate the following UDP Ports for the TRILL IS-IS and Data channels: UDP Port Protocol TBD TRILL IS-IS Channel TBD TRILL Data Channel 6. Security Considerations 6.1. Failure Separation To be added. 7. Acknowledgements Many ideas are from the discussion in the list ima@ietf.org. 8. Normative References [I-D.hasmit-otv] Grover, H., Rao, D., Farinacci, D., and V. Moreno, "Overlay Transport Virtualization", draft-hasmit-otv-03 (work in progress), July 2011. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC6325] Perlman, R., Eastlake, D., Dutt, D., Gai, S., and A. Ghanwani, "Routing Bridges (RBridges): Base Protocol Specification", RFC 6325, July 2011. [RFC6326] Eastlake, D., Banerjee, A., Dutt, D., Perlman, R., and A. Ghanwani, "Transparent Interconnection of Lots of Links (TRILL) Use of IS-IS", RFC 6326, July 2011. [RFC6327] Eastlake, D., Perlman, R., Ghanwani, A., Dutt, D., and V. Manral, "Routing Bridges (RBridges): Adjacency", RFC 6327, July 2011. Wan, et al. Expires June 9, 2012 [Page 10] Internet-Draft Extending TRILL over WAN December 2011 Authors' Addresses XiaoLan Wan HangZhou H3C No.2 ChuangYe Road, HaiDian District Beijing P.R. China Phone: +86 10 82774971 Email: wxlan@h3c.com XiaoPeng Yang HangZhou H3C No.2 ChuangYe Road, HaiDian District Beijing P.R. China Phone: +86 10 82774963 Email: yxp@h3c.com Vishwas.Manral HP 19111 Pruneridge Ave. Cupertino, CA USA Email: vishwas.manral@hp.com Alvaro.Retana HP 2610 Wycliff Road Raleigh, NC USA Email: alvaro.retana@hp.com Wan, et al. Expires June 9, 2012 [Page 11]