Internet Draft Pascal Menezes Expiration Date: May 2002 Ting Cai Don Flynn Terabeam Networks Yakov Rekhter Kireeti Kompella Juniper Networks Loa Andersson Ufors Marc Lasserre Riverstone Sunil Khandekar Timetra Networks Hamid Ould-Brahim Nortel Networks Giles Heron PacketExchange Ltd. Hans Johnsen Tropic Networks, Inc. Inter-City MAN Services Using MPLS draft-menezes-inter-city-man-mpls-00.txt Status of this Memo This document is an Internet Draft and is in full conformance with all provisions of Section 10 of RFC 2026. Internet Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet Drafts. Internet Drafts are draft documents that are valid for a maximum of six months and may be updated, replaced, or made obsolete by other documents at any time. It is inappropriate to use Internet Drafts as reference material or to cite them other than as "works in progress." The list of current Internet Drafts can be accessed at: http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet Draft Shadow Directories can be accessed at: http://www.ietf.org/shadow.html. Abstract This document describes a network service model for high-speed Metropolitan Area Network (MAN) service providers to deliver economical services between cities. It reduces the cost of services Internet Draft - Expires May 2002 1 Internet Draft Inter-City MAN Services Using MPLS November 2001 for MAN providers by using a distance-insensitive IP Network Service Provider (NSP) as a WAN partner for inter-city services. It simplifies MAN operation and improves the scalability of a traditional standard overlay model by allowing the MAN provider to peer to the NSP for both Internet transit and inter-city MAN services (e.g., TLS). This network service model allows an NSP to offer hierarchical MPLS services to downstream providers while providing scalability and automation for both the NSP and MAN provider. While this document refers to a solution for MAN providers, any downstream provider that needs hierarchical MPLS services from an NSP can use this service. Table of Contents 1 Specification of Requirements..................................2 2 Introduction...................................................2 3 Inter-City MAN Peering Model and Framework.....................5 3.1 Inter-City MAN LSP Hierarchical Model........................6 3.2 Class of Service Mapping.....................................7 4 Technical Details..............................................8 5 Example of Operations..........................................9 6 Security Considerations.......................................10 7 Reference.....................................................10 8 AuthorsÆ Addresses............................................10 1 Specification of Requirements The key words "MUST," "MUST NOT," "REQUIRED," "SHALL," "SHALL NOT," "SHOULD," "SHOULD NOT," "RECOMMENDED," "MAY," and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [1]. 2 Introduction MAN service providers deliver high-speed network connectivity to businesses in a metropolitan area. Many businesses have multiple office locations in one city or in several cities scattered in different geographical regions. With more capacity available in the NSPs that span multiple geographical regions and lower costs for long-distance IP services, there is an opportunity for MAN service providers to extend enterprise networks for these businesses by providing seamless connectivity regardless of whether the communication is intra-city or inter-city. To provide such services, one possibility for a MAN service provider is to lease circuit networks, such as Frame Relay or ATM from some Frame Relay or ATM WAN service provider(s) and then use these circuits to carry inter-city traffic. But it is costly to lease large-circuit bandwidth over long distances, and it is complex to operate both MAN and WAN networks from end to end. Menezes Internet Draft - Expires May 2002 2 Internet Draft Inter-City MAN Services Using MPLS November 2001 To reduce costs and complexity, one might consider IP NSPs for inter-city connectivity. NSP IP network services are typically distance-insensitive. To traffic engineer the network and to provide QoS, VPN, and other services, a possible approach for the MAN provider is to use the overlay model (see Figure 1) where explicitly routed MPLS Label Switched Paths (LSPs)are established end to end to form a full-mesh virtual network on top of the underlying network provided by the NSP. However, this approach raises scalability concerns because it requires N^2 square number of LSPs(where N is the number of cities being connected), with routers at the end of these LSPs having routing peering with each other; and it demands configuration changes at all cities when a new city is added. Also, the MAN service provider would need to design and operate routing that spans not just individual cities but the wide area as well. CE (MAN) / | \ / | \ / | \ / PE (NSP) \ / | \ / | \ / | \ / | \ / | \ CE (MAN) /---PE (NSP)------------------- PE (NSP)--- \ CE(MAN) \ | / \ | / \ | / \ | / \ | / \ | / \ PE (NSP) / \ | / \ | / \ | / CE (MAN) Figure 1. Overlay Model We propose a peering model (see Figure 2) that improves the scalability of the overlay model, reduces the operating cost of MAN providers, and pushes the complexity of operating an inter-city WAN backbone to NSP partners. In the peering model, a MAN router (CE) that connects the MAN to a WAN has routing peering not with MAN routers in other cities but with a WAN router (PE). As a result of this peering, the MAN router receives routing information for other Menezes Internet Draft - Expires May 2002 3 Internet Draft Inter-City MAN Services Using MPLS November 2001 cities, even if the router doesnÆt have routing peering with MAN routers in these cities. In addition, the NSP creates a collection of ôsink trees,ö where each tree is rooted at a particular city with the rest of the cities forming leaves of that tree. By using the hierarchical VPN approach described in [2], the NSP could support multiple MAN providers. Moreover, by using the concept of MPLS label stacking, each MAN provider could establish LSPs and interconnect routers in different cities without introducing additional routing/forwarding overhead on the NSP. Finally, the MAN provider could use MPLS label stacking to further reduce routing/forwarding overhead within each MAN area. Menezes Internet Draft - Expires May 2002 4 Internet Draft Inter-City MAN Services Using MPLS November 2001 CE (MAN) | | | PE (NSP) / | \ / | \ / | \ / | \ / | \ CE (MAN)---PE(NSP)_/---------------------- \ PE (NSP)--- CE(MAN) \ | / \ | / \ | / \ | / \ | / \ | / PE (NSP) | | | CE (MAN) Figure 2. Peering Model 3 Inter-City MAN Peering Model and Framework To accommodate these requirements, this document proposes that a VPN hierarchical model be used, a model in which the lower hierarchy is operated by the NSP and the higher hierarchy is made up of inter- city MAN services operated by the MAN provider. The generation of these lower hierarchical connections is the responsibility of the NSP and are coordinated with the MAN provider via an automated procedure (which is the purpose of this document). This document focuses on MPLS as the technology for hierarchical capability, the use of RFC 2547bis [2] as the mechanism that the NSP would use internally to offer VPN services to the MAN provider, and the use of BGP [3] to exchange both the routing and label information between the NSP and the MAN provider. An automated procedure, which is based on the mechanisms described in [2] and [3], automates the NSPÆs generation of inter-city LSPs that can be used by the MAN provider. Other mechanisms and standards that can be used to accommodate these same requirements are for future study. Menezes Internet Draft - Expires May 2002 5 Internet Draft Inter-City MAN Services Using MPLS November 2001 ( MAN 1 ) ( NSP Network ) ( MAN 2 ) ( ) ( ) ( ) ( ) ( )( ) ( PE1 - MAN - CE1 --- PE1 û P1 û- P2 û PE2 --- CE2 û MAN û PE2 ) ( BB ASBR ) ( )( ASBR BB ) ( ) ( ) ( ) ( ) ( ) ( ) <------><----------------><------> Automated Announcement NSP Service Automated Announcement Figure 3. Inter-City MAN Peering Model In Figure 3, the NSP PE routers that connect to the MAN at each city form peering relationships with the MAN routers (e.g., PE1 forms routing peering with CE1) (this is a function of the NSPÆs VPN services using RFC 2547bis [2]). While the MAN provider may be under the same administrative control for each city, we assume that the MAN in each city forms a separate IGP domain. The MAN CE router(s) in a given city exchanges only with the PE router(s) of the NSP routes and labels for the destinations that originate from that city. When a city is added to the network, the CE router automatically advertises to the NSP PE its routes for the destinations in that city, as well as labels for these routes, by using BGP [3]. The NSP PE routers in turn distribute the routes to their PE peers through BGP, and each PE peer further delivers the routes to other CE routers. Other citiesÆ CE routers that are in the same virtual private network as the new CE router will receive announcement of the new CE router and, therefore, have an LSP connection back to this new CE router and to other destinations reachable through this router (these are the destinations in the city to which the router belongs). Thus, LSPs among CE routers are automatically established, and these LSPs could be used by the inter-city MAN services. It is important to note that these inter-city LSPs that are generated by the NSP are hierarchical in nature, in the sense that they could be used to carry all kinds of traffic, including another higher-level LSP (which could be signaled by RSVP-TE or LDP) or IP traffic or both. Moreover, the procedures/protocols for establishing the LSPs at one level of the hierarchy (e.g., inter- city service LSPs) need not be the same as the procedures/protocols for establishing the LSPs at other levels of the hierarchy (e.g., inter-city LSPs). 3.1 Inter-City MAN LSP Hierarchical Model It is important to note the scalability achieved through hierarchical LSPs. For example, as shown in Figure 4, Ps of NSP network only need to store labels to reach its PEs. These PEs only Menezes Internet Draft - Expires May 2002 6 Internet Draft Inter-City MAN Services Using MPLS November 2001 need to know how to reach the corresponding CEs that it services. It is not necessary for the NSPÆs PEs to understand or care about the inter-city service LSPs of the MAN providers. The PEs in the NSPÆs network do not see any label stack other than the inter-city LSP. Therefore, when a MAN provider creates an inter-city service from one city to another across the NSP, the NSP has no state information for that LSP (this is assuming that the inter-city service LSP is created by using RSVP-TE). Therefore, an NSP for a given PE can accommodate many MAN customers in a scalable fashion. Moreover, an inter-city service LSP could be used by a MAN provider to offer services, such as Layer 2 VPNs, to multiple customers, thus further improving the scalability of the approach. ( MAN 1 ) ( NSP Network ) ( MAN 2 ) ( ) ( ) ( ) ( ) ( )( ) ( PE1 - MAN - CE1 --- PE1 û P1 û- P2 û PE2 --- CE2 û MAN û PE2 ) ( BB ASBR ) ( )( ASBR BB ) ( ) ( ) ( ) ( ) ( ) ( ) <------NSP LSP-----> <------------Inter-City LSP --------> <-----------------------Inter-City Service LSP -----------------> Figure 4. Hierarchical LSP Model 3.2 Class of Service Mapping An important aspect of this service is for the NSP to offer CoS capabilities so that SLAs within a MAN providerÆs city also can be honored for inter-city services. To achieve this requirement, the markings of any higher-level LSP (or IP packet) SHOULD be propagated to lower-layer LSPs, including traffic-engineered LSPs that are within the NSPÆs network. This functionality offers great scalability because as each packet (IP or MPLS) enters the NSPs network, it carries with it a marking that indicates which CoS PHB [diff serv] it needs. Therefore, the propagation of the marking from higher-layer services to lower-layer LSPs is completely Menezes Internet Draft - Expires May 2002 7 Internet Draft Inter-City MAN Services Using MPLS November 2001 feasible as long as a MAN provider enters into a bilateral CoS agreement with the NSP by specifying the behavior of each CoS marking. Future work will describe how a hard QoS model can also be achieved using a similar process. ( MAN 1 ) ( NSP Network ) ( MAN 2 ) ( ) ( ) ( ) ( ) ( )( ) ( PE1 - MAN - CE1 --- PE1 û P1 û- P2 û PE2 --- CE2 û MAN û PE2 ) ( BB ASBR ) ( )( ASBR BB ) ( ) ( ) ( ) ( ) ( ) ( ) <------------------> NSP LSP EXP Marking <----------------------------------> Inter-City LSP EXP Marking <---------------------------------------------------------------> Inter-City Service LSP EXP Marking 4 Technical Details As we mentioned above, the inter-city LSPs are created by following the procedures described in Section 9 of [2]. In addition, this document assumes that BGP is used as the protocol for exchanging routing and label information between the PEs of the WAN provider and the CE-ASBRs of the MAN provider (as specified in [3]). Note that when a CE-ASBR advertises a route to a WANÆs PE, the label associated with this route should be Implicit NULL. This document assumes that the inter-city service LSPs are established by using RSVP-TE. The RSVP-TE setups for these LSPs are carried by the NSP (including the ingress and egress PEs of the NSP) as plain data; neither the ingress nor the egress PE create any RSVP and/or labeling states for these setups. Doing this avoids the need for the NSP to maintain state on a per inter-city service LSP basis. The setups create RSVP and label state on the routers within MAN, including the routers (CEs) that peer with the NSP routers but not on any of the NSPÆs routers. With respect to how one would construct the Explicit Route Object (ERO) carried by the setups, there are several options. One option is for the MAN service provider to use some off-line tool to compute the EROs for the inter-city service LSPs. Another alternative is to compute the ERO on a per MAN basis. Using the example shown in Figure 4, with this alternative, PE1 in MAN1 would compute the ERO Menezes Internet Draft - Expires May 2002 8 Internet Draft Inter-City MAN Services Using MPLS November 2001 for the segment of the path through MAN1, and CE2-ASBR would compute the ERO for the segment of the path through MAN2. For the purpose of handling the ERO carried in the setups, one alternative is to assume that the MAN routers that peer with the NSP routers are treated as being one hop away from each other. Using the example shown in Figure 4, for the purpose of handling the ERO carried by the RSVP-TE setup for an inter-city service LSP from PE1 in MAN1 to PE2 in MAN2, CE1-ASBR and CE2-ASBR are treated as being one hop from each other. Another alternative for handling the ERO is to require that the originator of an inter-city service LSP specify the hop over the NSP network as a loose one. Again, using the example shown in Figure 4, PE1 in MAN1 would specify CE2-ASBR as a loose hop in the ERO. 5 Example of Operations Consider the following topology AS1 (MAN) | AS2 (WAN) | AS3 (MAN) R1---R2---R3---R4--....--R5---R6---R7---R8 where AS2 provides WAN VPN services to a MAN service provider composed of AS1 and AS3. The routing protocol between R3 and R4 is eBGP. The same is true for R5 and R6. R6 advertises to R5 (via eBGP) a single address prefix ("AS3 prefix") that covers all the destinations in AS3 with itself as a next hop. R6 advertises the "AS3 prefix" to R5 with an Implicit NULL label. R5 advertises this prefix to R4 with label L2 and next-hop R5. R4 advertises the prefix to R3 with label L1 and next-hop R4. With this in mind, the following illustrates MPLS forwarding associated with the inter-city LSP (LSP from R3 to R6). LSR dest action R3 "AS3" push L1 R4 L1 swap L1 with L2; push R5 L2 pop L2, forward to R6 For the inter-city service LSP (R1-R8), the RSVP hops are R1, R2, R3, R6, R7, and R8. R{2,3,6,7} send RSVP labels M{1,2,3,4}, respectively, to the previous hop. With this in mind, the following illustrates MPLS forwarding associated with the inter-city service LSP (LSP from R1 to R8). LSR dest action R1 R8 push M1 Menezes Internet Draft - Expires May 2002 9 Internet Draft Inter-City MAN Services Using MPLS November 2001 R2 M1 swap M1 with M2 R3 M2 swap M2 with M3; push L1 R4 L1 swap L1 with L2; push R5 L2 pop L2, forward to R6 (top label is now M3) R6 M3 swap with M4 R7 M4 PHP/explicit null R8 - - 6 Security Considerations This document does not affect the underlying security issues of MPLS and does not introduce any security considerations that are not already part of the existing usage for LSPs. 7 Reference [1] Bradner, S. "Key Words for Use in RFCs to Indicate Requirement Levels," BCP 14, RFC 2119, March 1997. [2] Rosen, E., et al. "BGP/MPLS VPNs," draft-ietf-ppvpn-rfc2547bis- 00.txt, July 2001. [3] Rekhter, Y., and E. Rosen. "Carrying Label Information in BGP-4," RFC3107, May 2001. [4] "RSVP-TE: Extensions to RSVP for LSP Tunnels," draft-ietf-mpls- rsvp-lsp-tunnel-08.txt. Work in progress. [5] Kompella,K., and Y. Rekhter. "LSP Hierarchy with MPLS TE," draft-ietf-mpls-lsp-hierarchy-02.txt, February 2001. 8 AuthorsÆ Addresses Pascal Menezes Terabeam 14833 NE 87th St. Phone: (206) 686-2001 Redmond, WA, USA Email: Pascal.Menezes@Terabeam.com Ting Cai Terabeam 14833 NE 87th St. Phone: (206) 255-6220 Redmond, WA, USA Email: Ting.Cai@terabeam.com Don Flynn Terabeam 14833 NE 87th St. Phone: (206) 321-4724 Redmond, WA, USA Email: Don.Flynn@Terabeam.com Yakov Rekhter Phone: (408) 745-2000 Juniper Networks Email: yakov@juniper.net Menezes Internet Draft - Expires May 2002 10 Internet Draft Inter-City MAN Services Using MPLS November 2001 Loa Andersson Utfors Research, Architecture and Future Lab (URAX) Utfors AB R…sundav„gen 12 Box 525, 169 29 Solna, Phone: +46 8 5270 2000 Sweden Email: loa.andersson@utfors.se Marc Lasserre Riverstone Networks 5200 Great America Pkwy Phone: (408) 878-6550 Santa Clara, CA 95054 Email: marc@riverstonenet.com Sunil Khandekar Timetra Networks 274 Ferguson Drive Phone: (650) 237-5100 Mountain View, CA 94043 Email: sunil@timetra.com Hamid Ould-Brahim Nortel Networks P O Box 3511 Station C Phone: (613) 765-3418 Ottawa ON K1Y 4H7 Canada Email: hbrahim@nortelnetworks.com Giles Heron PacketExchange Ltd. The Truman Brewery 91 Brick Lane LONDON E1 6QL Phone: +44 207 377 4130 United Kingdom Email: giles@packetexchange.net Kireeti Kompella Juniper Networks 1194 N. Mathilda Ave. Phone: (408) 745-2000 Sunnyvale, CA 94089 Email: kireeti@juniper.net Hans Johnsen Tropic Networks, Inc. 135 Michael Cowpland Dr, Suite 200 Kanata, Ontario K2M 2E9 Phone: (613) 270-5399 Canada Email: hjohnsen@tropicnetworks.com Menezes Internet Draft - Expires May 2002 11