Internet Draft draft-xu-bgp-gmpls-00.txt Expiration Date: November 2001 Yangguang Xu (Lucent) June 2001 A BGP/GMPLS Solution for Inter-Domain Optical Networking Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 [RFC-2026], except that the right to produce derivative works is not granted. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. 1 Abstract This document describes a method by which customer can request circuit connection through multiple providers. The target environment is the transport network backbone, where customers are ISPs, CLECs, ILECs etc. and providers are IXCs, carriers' carrier etc. BGP is used to disseminate route and calculate the circuit path over the transport backbone networks, and GMPLS is used as signaling for the connection operations. The primary goal is to support customer network Network Engineering function to adjust its layer 1 network topology in a multi-provider environment, which is typical in today's transport backbone. This method is very flexible and scalable. It support various applications including optical VPN. It is essential to achieve true end-to-end automatic switched network, which will dramatically reduce the network operation cost. Yangguang Xu [Page 1] draft-xu-bgp-gmpls-00.txt Nov. 2001 2 Acronym AG: Access Group AP: Access Point NE: Network Element BNE: Border NE FA: Forwarding Adjacency PFA: Potential Forwarding Adjacency NNI: Network Network Interface UNI: User Network Interface NNI: Network Network Interface OXC: Optical Cross Connect IDP: Initial Domain Part ASTN: Automatic Switched Transport Network ASN: Autonomous System Number 3 Introduction 3.1 Goal - Integrated Control Plane There are many ways to layer the network. For "switching" networks, we can layer them according to network' "switching" granularity. An IP router's switching granularity is packet; a High Order (LO) SONET circuit switch's switching granularity is STS-1; a non-OEO optical switch's switching granularity is wavelength. These different types of NEs are at different network layers. They form client/server relation between each other. PDM (packet/cell) ____________________________________________________________ TDM LO (DS-1, VT1.5 etc.) | | | ______________________________V_______ | | TDM HO (DS-3, STS-1 etc.) | | | __________________________________V______V____ | WDM | | ______________________________________V_____________V___ SDM | __________________________________________V_________________ The purpose of layering the network is to "divide & conquer". Our final goal is to achieve the true end-to-end automatic switched network, not only layer 3 networking or layer 1 networking, but networking of different layers and different administrative domains. In another word, we are looking for a integrated control plane across networks. Yangguang Xu [Page 2] draft-xu-bgp-gmpls-00.txt Nov. 2001 GMPLS has brought the concept of extending MPLS signaling protocols and other IP control protocols and applying them in non-packet switched networks. It enables the transport network into automatic switched transport network. This new capacity of the transport network has profound impact on conventional data networking. For example, it changes the data network's basic assumption that the transport network consists of fixed pipes. In the new paradigm, the transport network can be conceived as a configurable back plane. Packet switches can dynamically adjust their layer-1 network topology according to end user traffic for optimized overall network performance. There are two functional areas we need further study to achieve an integrated control plane: 1. "How" to automatically change the network topology? This document focuses on the solution in the inter-domain environment. It follows the direction GMPLS and proposes to use a common set of IP based protocols for different types of networks. 2. "When" and "where" to change the network topology? This needs network engineering as well as conventional traffic engineering. Network engineering, in a short sentence, is to put the bandwidth where the traffic is [ASTN-LN]. It is the glue for control planes integration of different layers. Details will be covered by [NE-FRWK]. 3.2 Scope and Target Environment The target application environment of this draft is the core transport network. In this environment, we can separate networks into providers and customers. customers are networks that request circuit services from provider networks. customer NE can be IP routers, ATM switches or SONET/SDH switches. Providers are networks that provide layer 1 transport service. A network can be both a customer network and a provider network simultaneously. For example, a SONET/SDH network can be provider network to an IP network; meanwhile, it's a customer to an OTN. In the core transport network, providers are transport backbone providers and carriers' carrier. customers are ISPs, CLECs, ILECs and large enterprises. They are all large organizations. This draft assumes customer/provider either has a public AS number or public initial domain part (IDP). It also assumes that providers run IP based control plane with IP based addressing scheme and public AS numbers. Current GMPLS drafts mainly focus on intra-area scope. However, in the core network, it's more of a multiple administrative domain environment. Yangguang Xu [Page 3] draft-xu-bgp-gmpls-00.txt Nov. 2001 A customer may request a circuit connection across multiple providers. It is equally important and urgent to standardize the inter-domain interfaces for the core transport network. This draft specifies protocol extensions of BGP-4 and GMPLS signaling for this interface. This solution doesn't disrupt conventional IP networking or hurt its scalability. BGP and GMPLS signaling run on circuit switches. They are not used for packet forwarding, instead: 1. BGP serves as routing protocols to build up layer 1 network inter-domain topology database (similar to TE database) and calculates circuit path between customer NEs. 2. GMPLS is extended to create the circuit connections across multiple providers. This document doesn't repeat current BGP and GMPLS. It focuses on extensions and modifications. 4 A Simple Example First - Optical VPN Below is one of the applications that enabled by the solution proposed by this draft. It's one type of optical VPN services. In this example, a customer has multiple locations and leases access points to different providers, which provides circuit transport service between any two access points. The purpose of this example is to highlight the procedures and the functional areas that BGP and GMPLS can be applied and need to be extended. | | | | | | +--+ +--+ | +--+ +--+ | +--+ +--+ | +--+ +--+ |A1|-///-|A2|-+-|X1|-///-|X2|-+-|Y1|-///-|Y2|-+-|A3|--|A4| +--+ +--+ | +--+ +--+ | +--+ +--+ | +--+ +--+ customer A | \ | | | | customer A Location 1 | \ | /// | | Location 2 | \----\ | | | +-------------- --------------+ \ | | | | +--+ +--+ +--+ +--+ | +--+ +--+ | +--+ +--+-|A5|--|A6| |B1|-///-|B2|-+-|X3|-///-|X4|-+-|Y3| | +--+ +--+ +--+ +--+ | +--+ +--+ | +--+ | | | | | | +--+ | | | +-----------+-|A7|----+ | | | +--+ customer B | Provider X | Provider Y | customer A | | | Location 3 Inter-domain Inter-domain Inter-domain Interface Interface Interface Yangguang Xu [Page 4] draft-xu-bgp-gmpls-00.txt Nov. 2001 4.1 Network Topology In this example, customer A has three locations: 1, 2 and 3. Provider X and Y are optical transport backbone providers. X1, X2, X3, X4, Y1, Y2, Y3 are OXCs. * customer A location 1 connects to Provider X through A2 * customer A location 2 connects to Provider Y through A3 * customer A location 3 connects to Provider Y through A5 * customer A location 3 also connects to Provider Y through A7 B is another customer connects to Provider X through X3 4.2 Operation Scenario - Time Sequence 1. A2 registers service to X1; A3 registers service to Y2; A5 registers service to Y2; A7 registers service to Y3; 2. Xs disseminate AG routes(section 5.1.3) information through I-BGP Ys disseminate AG routes(section 5.1.3) information through I-BGP 3. Y2 and Y3 disseminate A3 information to A5 and A7 respectively because they belong to the same customer. For the same reason, Y2 disseminate A5 and A7's information to A3. 4. X and Y exchange AG route through E-BGP according to business agreement. 5. In case that X and Y exchange all customer A related information. With the same procedure as step 2, customer A location 1, 2, 3 know each other and their potential access points. Customer B may know nothing about customer A if the business agreement between customer A and provider X prohibits X to do so. 6. While disseminating the AG information, X and Y maintain all AG routes information, including BGP Next Hop. Also they calculate each route's degree of preference. 7. If A2 decides to establish optical path to A7, A2 initiates GMPLS signaling request with ERO = A2, A7 (loose) 8. Each BGP speaker decides which BGP Next Hop to choose for the connection when it receives the GMPLS connection request. 9. GMPLS signaling propagates all the way to A7. Meanwhile, intra-domain connection operations are triggered at each provider and eventually the connection is set up. Yangguang Xu [Page 5] draft-xu-bgp-gmpls-00.txt Nov. 2001 5 Inter-domain Routing The routing protocol at inter-domain interface should provide network controls to deliver right information to the right place. Information should be enough but not too much. The control should be configurable according to policies, service agreement and contract. 5.1 Network Topology View Customer and provider NEs have different views of the network topology. They don't have and don't need to have the same network view. There are sensitive information that a provider doesn't want to disclose to its customers or other providers. Meanwhile, in order to support the network scalability, it's critical to distribute network functions and only have the function performer equipped with the required information. 5.1.1 IP Customer Network Topology View 1. Conventional Routes to IP peers These are conventional routes in current IGP and EGP 2. Forwarding Adjacency FA is introduced by [LSP-HIER]. FAs are the neighbors that were dynamically connected through the provider network. The connection can be torn down if necessary. The first two pieces of information are used for packet forwarding. These information can/should be disseminated in the conventional way through existing connections. Transport network treats these information as normally user traffic. (Another way to say, transport network is unaware of these information) 3. Potential FAs PFAs are access points to other locations of the same customer network or other customer networks. They are routers that can be connected through underlying transport network, but are not connected yet. PFAs are typically customer border NEs. This information can be disseminated through provider network BGP or directory service/yellow page like mechanisms. 4. Accessible IP Routes through the PFAs This information can be used for IP network TE function to determine the source and destination of the layer 1 network connections. This information may be disseminated through established customer network connections, dedicated customer network control plane network or provider Yangguang Xu [Page 6] draft-xu-bgp-gmpls-00.txt Nov. 2001 network's control plane. As this information may be huge and proprietary, the last choice may need service contract between customer and provider. 5. Filtered/Abstracted Topology Information of the Provider Network This information is leaked from provider to customer according to their business agreement. It can be used by customer networks to specify path constrains within provider's network. With these five pieces routing information, the Network Engineering [NE-FRMK] function at the customer network could decide when and where to set up or tear down FAs for more efficient network resource utilization according to user traffic pattern, requirements etc.. It then requests the service from the transport networks accordingly. One of the PFA's applications is enabling the virtual terabit router. It's especially useful for hybrid NEs with both IP switch fabric and circuit switch fabric. When interconnected together, they can aggregate traffic and choose appropriate circuit shortcuts by dynamically adjusting their circuit connections through circuit switch fabric in order to avoid unnecessary packet switching and minimize traffic port consumption for NE interconnection. 5.1.2 Circuit Switched Customer Network Topology View 1. PFAs 2. Filtered/Abstracted Topology Information of a Provider's Network 3. Customer Network Routes that are Accessible through the PFAs. 5.1.3 Circuit Switched Provider Network Topology View There are two types of routes: 1. Routes that initiated from a provider, and 2. Routes that initiated from a customer. Routes that initiated from a provider are only used for intra-domain connections. They are sensitive topology information to the provider and are not distributed across domain interfaces. Indeed, providers only need to know information that is enough to create link connections for their customers. So a provider network only cares about its customer networks' link termination points, locally and remotely. These connection termination points are called Access Group (AG) in this document. AG is a set of Access Points (AP) - customer NE ports (both physical and logical), that share the same physical and logical attributes. Each individual access point in this access group is treated equivalently for connection operations. Yangguang Xu [Page 7] draft-xu-bgp-gmpls-00.txt Nov. 2001 So a provider only needs to know: 1. Local Customers. 2. Routes to Remote Customers. 5.2 BGP Extensions In this draft, BGP is used to create link connections between customer NEs. The customer link connection across over multiple provider networks. Even most of BGP and its extensions specified by various RFC and IDs can be re-used for the inter-domain circuit connection operations, their applications, behavior and some procedures are changed. For circuit connection operations, BGP can be applied for three areas: (1) conventional routing functions for circuit connection operation (topology information dissemination control and path selection), (2) distributed directory service, and (3) dissemination of abstracted provider network topology to customer networks. * Conventional Routing Functions These functions include topology dissemination control and path selection. Access Group (AG) routes are disseminated by BGP for circuit path selection across multiple providers. Different from the distributed directory service mentioned below, only aggregated AG information is needed for inter-domain routing purpose. In inter-domain routing, hop by hop path selection is desirable for various reasons. The BGP path selection chooses the right "BGP next hop" from multiple possibilities for a path creation request. * Distributed Directory Service: BGP can also be extended to disseminate customer PFA information. PFAs provide a NE (and its AS) with the information of possible access points to other customer networks or network locations of the same customer. Individual PFA information can be obtained either through "push" via BGP or "pull" via a directory service/yellow page. Because PFA information is used for directory service, this type of information shouldn't be aggregated. Thus BGP has to exchange all individual piece of PFA information. PFA information dissemination needs strict control. A customer may only want to disseminate its information to some other customers. As a policy routing protocol, BGP could provide control as to who to disseminate what information. Yangguang Xu [Page 8] draft-xu-bgp-gmpls-00.txt Nov. 2001 PFA routes can also be used for routing purpose. They are simply non-aggregated AG routes. To provider, they are AGs with NO-AGGREGATION (section 5.2.3) attribute attached to them. * Disseminating abstracted provider network topology information A customer and a provider may have agreement that allow the provider to leak certain topology information of the provider network to the customer for some applications, e.g. the customer could specify certain routing constrains for it connection requests. BGP can also be extended to disseminate this type of information. This extension is not covered in this document. 5.2.1 Access Group Address Family A backbone transport network can serve different types of customers. For example, an OTN can serve as server network to SONET/SDH, PDH, ATM or IP networks. For the same type of customer networks, each customer network may have its own address space, which means that a given address may denote different systems in different customer networks. Furthermore, each customer or provider may have its addressing preference. So we need is a new address family that can accommodate different networks. An AG may only have one unique address, while each AP can be identified with a locally unique interface ID. Otherwise, if each AP has an address, the AP address is the aggregation of these individual AP addresses. inter-domain routing is only interested in the AG address. Details such as which AP to select can be resolved locally. AG address is an external address and should be global unique. It may be different from the customer address that is used for internal routing. AG address can be generated in two approaches: 1. Extending customer NE's internal address * If AG internal address is already globally unique, then it is presented as it is. * If AG internal address is not globally unique, it should be pre-attached with a public AS number assigned by IANA. 2. Using provider assigned address * If the assigned customer AG address is global unique, then it is presented as it is. * If the assigned customer AG address is not global unique, it should be pre-attached with a public AS number assigned by IANA Each choice has pros and cons. * Extending AG internal address makes mapping between internal address to external address easy. But this address scheme is hard to aggregate Yangguang Xu [Page 9] draft-xu-bgp-gmpls-00.txt Nov. 2001 and may have scalability problem. Meanwhile, customer information may be sensitive to providers. AG route using this type of address structure may disclose a provider's customer information. * Using provider assigned address requires mapping between internal address to external address. Multi-homed customer NE may have more than one address. However, this mechanism provides providers better control, easy address aggregation and customer information hiding. For both approaches, the same new address family is introduced. This new address type enables customers/providers of different address schemes share a common addressing structure for inter-domain circuit connection operations. The new address has an overall length of 22 bytes. It consist two fields: * Address Type * Address Value +---------------+------------------------------------------------+ | Type(2 bytes) | Value(20 bytes) | +---------------+------------------------------------------------+ Address Type: 2 bytes * IPv4 * IPv6 * NSAP * Private IPv4 * Private IPv6 Value: 20 bytes The value field is formatted and interpreted according to Address Type field. The 20 bytes is patted with "0" if not fully occupied. * If Address Type = IPv4, value is a public IPv4 address. * If Address Type = IPv6, value is a public IPv6 address. * If Address Type = NSAP, value is a public NSAP address. * If Address Type = Private IPv4, the value has 4-byte "Organization Distinguisher" (OD) before the private IPv4 address. An OD is a 2/4 bytes Autonomous System number. These numbers must be assigned by IANA and are publicly known. * If Address Type = Private IPv6, the value has 4-byte OD before the private IPv6 address. An OD is a 2/4 byte ASN. 5.2.2 Access Group NLRI MP-REACH-NLRI [BGP-MP] can be used to support this new type of address family. Yangguang Xu [Page 10] draft-xu-bgp-gmpls-00.txt Nov. 2001 * AFI needs a new type for the new address family. (IANA consideration). When receiving this new type of NLRI, the BGP speaker should understand that the NLRI is not used for packet forwarding. Instead, it should follow the procedures defined in this spec. for inter-domain circuit operations. * Network Address of Next Hop has the same format of the new address family defined above. It's the network address of the next circuit switch on the path to the destination system * NLRIs are the AGs or AG prefixes that are accessible through provider networks. 5.2.3 New Path Attributes and Extended Community Attribute * AG-TYPE (type code TBD) This is an optional transit attribute that defines the AG type. It's encoded in four bytes as following. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Encod. Type | Sig Bit Map | G-PID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Encoding Type and Signal Bit Map together indicate the layer 1 attribute. G-PID indicates the upper layer protocol ID. Signal Bit Map is interpreted according to the encoding type field. Details are TBD. This attribute is used to control the AG route dissemination. A provider disseminates an AG route to a customer only if the customer's interface is compatible with remote the AG route. Meanwhile, A BGP speaker disseminates the AG route to external BGP speakers only if it can support the connection request of the AG-TYPE. * NO-AGGREGATION (type code TBD) This is an optional transit attribute that defines an AG route that shouldn't be aggregated and MUST be disseminated individually. This attribute is defined for distribute directory service. * DISCLOSE-SET (type code TBD) This is an optional transit attribute that defines which customer networks this route should be disseminated. The customer network is identified either by AS number networks using IP address or initial domain part (IDP) for networks using NSAP based addressing structure. Yangguang Xu [Page 11] draft-xu-bgp-gmpls-00.txt Nov. 2001 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Number of customer Networks (1 byte) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | First customer Network (in TLV format) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Last customer Network (in TLV format) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ If number of customer networks byte is all "1". This route should be "pushed" to all customer networks. * EXTENDED-LINK-BANDWIDTH This attribute is extended from Link Bandwidth Community [BGP-EXTCOMM]. It follows operation rules defined in [BGP-EXTCOMM]. It carries the TE attributes of the links that connects two provider networks. This attribute can be used for the calculation of degree of preference and tie breaking for path selection purpose. This attribute contains the same information as defined by [OSPF-GMPLS] and [ISIS-GMPLS]. They are TotalBW, AvailableBW, MinProvisionableBW and MaxProvisionableBW. * AG-ORIGINATOR This attribute is an optional transit attribute that indicates the network ID of the route originator. This attribute is used for provider assigned addressing scheme only. It's encoded as a TLV. Type field indicates the address type. Length is the network ID length. Value is the network ID. 5.2.4 AG Route Dissemination Control at Customer/Provider Boundary Route distribution at Customer/Provider network boundary is controlled by policies according to business agreement between the customer and the provider. Customer information is known through neighbor discovery, customer registration and service discovery procedures. When a AG route is imported from a customer network, it is associated with one or more attributes, which are carried in BGP attributes of the route according to the business agreement between the provider and customer. Or the customer device, when it distributes these routes to the provider BNE, Yangguang Xu [Page 12] draft-xu-bgp-gmpls-00.txt Nov. 2001 might specify one or more route attributes for each AG route. The latter method grants control the customer. If this method is used, it may still be desirable to have the provider BNE eliminate any route attributes that, according to its own configuration, are not allowed, and/or to add in some route attributes according to its own configuration. When a provider border NE receives an AG route through I-BGP, it disseminates it to a customer if: 1. AG-TYPE of the route match the customer interface, and 2. The route originator and the customer are of the same network according to the network ID (AS number or IDP), or 3. The customer's network ID matches one of the network ID in the AG route's DISCLOSE-SET attribute, or 4. The route's Export Targets match the provider BNE's Import Targets. [BGP/MPLS-VPN] If a provider decides to disseminate an AG route to a customer, it may choose to truncate the AG route's AS-PATH attribute if it doesn't want the customer to know the information or doesn't want the customer to specify ER object/TLV. However, this MUST not be allowed between providers. 5.2.5 Route Dissemination Control at Provider/Provider Boundary There are many considerations as what to disseminate to other providers. For example, some customer information may be sensitive to operators, or a provider want to enforce certain "transit policy". BGP route attributes and community attributes can help operator to control where and what to disseminate. When a provider border NE receives a route through I-BGP, it puts the route into the Adj-RIBs-Out if: 1. The BGP speaker has the capacity and capability to perform the connection as specified by AG-TYPE, and 2. The "Number of customer Networks" field in DISCLOSE-SET is all "1"s, or 3. The route's Import and Export Targets match. A network may aggregate some AG routes. * If AG address uses provider assigned address, a provider can aggregate all its customer information into one AG route. This would dramatically reduced BGP traffic and also hide its sensitive customer information. * If AG address uses extension to customer internal address, AG routes of different customers are very hard to aggregate and indeed should not be aggregated. If a customer has different network locations, the address aggregation of different locations shouldn't overlap. Or a BNE may choose the wrong next hop because of the incomplete information. Yangguang Xu [Page 13] draft-xu-bgp-gmpls-00.txt Nov. 2001 5.2.6 AG Route Dissemination via I-BGP The default BGP behavior is that a router does not change the next hop attribute for a route learned via an E-BGP session when it advertises the route via I-BGP. For the circuit connection operations, this behavior is changed. An BNE puts its address as the BGP Next Hop when disseminating the AG route to other BNEs through I-BGP. This address is encoded as an AG address because [BGP-MP] requires that the next hop address be in the same address family as the NLRI. This change simplifies the protocol by avoiding the unnecessary IGB/BGP interaction (section 5.2.8). However, route reflector [BGP-REF] and other BGP mechanisms need to be adjusted for this change. 5.2.7 Circuit Connection Route Selection Process Different from IP network, BGP disseminates all possible routes. A major part of the BGP routing protocol is the algorithm used to select which, if any, of the possible paths toward an AG route should be selected and established. For the inter-domain circuit connection, path selection is typically done hop by hop (here the hop is in AS granularity) because a provider may not be willing to follow its customer or other providers' path selection result. Each AS makes the route selection for its own benefit, so the end-to-end path selection may not be optimized. The input to this route selection process is the set of routes to the same AG that have been learned and accepted by the local system and a set of routing constrains from customers or other providers. The output is the selected BGP next hop and some modified constrains (e.g. ER object/TLV with loose hop and abstract node). Path selection involves several steps: 1. Calculation of the local degree of preference. This step can be done whenever the local BGP speaker receives an UPDATE message from a peer located in a neighboring AS that advertises a new route, a replacement route, or a withdraw route. It also should be done whenever BGP Next Hop's TE information has significant change (a change over certain threshold). The BGP Next Hop's TE information can be obtained from IGP and EXTENDED-LINK-BANDWIDTH community attribute. For each new route, the local BGP speaker shall determine a degree of preference. The degree of preference is a local decision that may invoke TE, economic, policy, service and business related analysis and decision-making. 2. Route selection. This step is invoked on completion of step 1 and the Yangguang Xu [Page 14] draft-xu-bgp-gmpls-00.txt Nov. 2001 receiving of GMPLS connection setup request. It is a constrain-based route selection procedure. The selection is based on the local degree of preference. All constrains coming with the path request, IGP TE information and EXTENDED-LINK-BANDWIDTH are used as qualifiers. In case there are ties in the route selection, local system can use the tie-breaking rules to decide which one of the next hop to select. The tie-breaking sequence basically follow what's being specified [BGP-4]. In short, they are: (1) The route with the highest LOCAL-PREF is selected (2) The route with the shortest AS-PATH is selected (3) The route with the lowest MULTI-EXIT-DISCRIMINTOR value is selected. (4) The route with the minimum cost to Next Hop is selected. Here, the cost should take the IGP TE information and EXTENDED-LINK-BANDWIDTH into consideration (5) Routes learned from E-BGP are preferred (6) If all routes were learned via I-BGP, the route that was learned from the I-BGP neighbor with the lowest BGP ID is selected. 3. Connection creation and GMPLS signaling relay. Details are covered in the inter-domain GMPLS signaling section. 5.2.8 BGP/IGP Interaction Different from IP network, IGP in circuit network only deals with circuit connections within the domain. BGP and IGP don't need to exchange topology information for the circuit connection operations. BGP route selection procedure described above chooses intra-domain sub-connection ingress and egress points. CSPF and IGP link state database are then used to compute the exact path within the domain. 5.2.9 Scalability Enhancement A BGP speaker distributes AG routes to any other internal BGP speaker by means of an I-BGP connection between them. Alternatively, each can have an I-BGP connection to a route reflector [BGP-REF] for scalability. BGP speakers can also support cooperative route filtering capability. They may distribute filters to other BGP speakers to block certain information that it doesn't care/want [BGP-ORF]. Yangguang Xu [Page 15] draft-xu-bgp-gmpls-00.txt Nov. 2001 6 Inter-Domain Signaling In packet networks, LSPs are created either because of topology changes (control-driven) or data traffic flow (data-driven). In layer 1 transport network, circuit LSPs are service-driven. They are created either because of customer requests or customer network Network Engineering decisions. Signalings at different interfaces share the same signaling message types: create/response, delete/response, modify/response, query/response and notification. Two other interfaces: UNI [OIF-UNI] and intra-area interface [GMPLS-SIG] have been studied. This section focuses on the uniqueness of inter-domain signaling. Inter-domain Signaling is used between network and network boundary. It is different from UNI and intra-area interface in their information content and processing procedures: * Comparing to intra-area signaling, inter-domain signaling needs to deal with business related issues at network boundaries, such as service, policy and security, which are not problems for intra-area interface. * Comparing to UNI, inter-domain signaling deals with network to network interface. In this sense, networks are treated more like peers. A network may know much topology information and has network level considerations. So inter-domain signaling may specify routing constrains that are unavailable to UNI. In deed, UNI signaling can be treated as a special case of inter-domain signaling. If the ERO is simplified to source customer address and destination customer address marked as loose hop, UNI signaling should behave the same as inter-domain signaling. 6.1 Inter-Domain Extension to GMPLS Detailed formats of the objects defined below are protocol dependent. 6.1.1 Routing Constrain Related Objects/TLVs * Explicit Route Explicit Route in inter-domain routing is in AS granularity. Different from intra-domain ER, it is only treated as a suggestion in inter-domain case. In inter-domain routing, a customer may either specify the strict AS path or only specify the source, next hop and destination. The destination can be either a specific AG (a PFA) or a network ID. If a customer or provider specifies the entire ER according to a route's AS-PATH attribute distributed through BGP, it is only treated as a preference/suggestion to the next AS. Yangguang Xu [Page 16] draft-xu-bgp-gmpls-00.txt Nov. 2001 Otherwise, a customer or provider only specifies the strict next AS and loose destination based on its own BGP route selection result. If a customer network wants to connect to another network and doesn't care about which access point to attach, it can just specify an network ID (AS number or IDP) as the destination. This was not allowed in MPLS signaling but should be allowed for GMPLS. For example, if an ISP wants an OC-12c connection to an Internet Backbone Provider, which has multiple AGs. The ISP only specifies the destination AS number to the layer 1 transport network provider, who then finds the best AG to the Internet Backbone Provider and creates the connection. * AS Record Route If a provider chooses to decline the ER chose by previous AS or customer. It may overwrite the entire ER. This may create loops in connection set up. The AS Record Route is used to prevent this situation in inter-domain connection set up. When an AS ingress border NE calculates the whole path or next AS, it only chooses a next hop or entire ER that are different from what's already in AS Record Route. An AS Record Route consists a series of variable-length sub-objects, which are 2 bytes or 4 bytes AS numbers or IDPs. AS Record Route is only used for connection creation. Providers may choose not to propagate this information to its customers. * Avoid AS List A customer or a provider may want to avoid going through a certain provider because of some business considerations. It can use the Avoid AS List to specify which AS to avoid in its connection request. Avoid AS List also enables a customer to create AS diversified paths if they exist. The customer can set up a path first and then request another path by putting the first connection's AS Record Route into the Avoid AS List of the second request. The Avoid AS List MUST be met. Otherwise, error should return with indication that the Avoid AS List can't be met. * Avoid Connection ID and Diversity Options A customer may want to specify diversified paths in other granularities. These two parameters serve the purpose. The diversity option is the same as what's been defined in [OIF-UNI]. It includes link level, node level and SRLG level diversification. Connection ID is AG address + local interface ID. It is globally unique and should be maintained by all nodes the path goes through. Yangguang Xu [Page 17] draft-xu-bgp-gmpls-00.txt Nov. 2001 6.1.2 Security/Policy Related Objects/TLVs When signaling messages passing across AS boundaries, they should include security and/or policy information. Each AS needs to validate the information in order to guarantee the integrity of network operation. In the inter-domain case, two sets of security/policy objects need to be presented. One set is for neighbor to neighbor. The other set is for customer to customer. The later set should be transparent to providers. 6.1.3 Service/Contract Related Objects/TLVs * Service Type This attribute indicates a class of service as specified in the service layer agreement between networks. A provider may specify a range of different classes of service with predefined characteristics (e.g. survivability). Service and contract related objects are negotiated between neighboring networks. They normally have local significant. Customer networks that are connected through transport networks are also neighboring networks. (Connections between customer NEs are link connections to customer networks and network connections to provider networks). Same as security/policy objects, when across network boundary, they should be replaced with the right set. 6.2 Connection Operation Procedures Connection operation in circuit network actually involves two logical steps: 1. link connection between customer NEs, and 2. network connections through provider networks. For many reasons, the requested customer network may not want to accept the connection to the requesting customer network. In the inter-domain case, it's desirable to verify the link connection request before requesting the network connection. Out-of-band signaling is designed to meet this purpose. Inter-domain connection setup is a three-stage process. | | | +--+ +--+ | +--+ +--+ | +--+ +--+ | +--+ +--+ |A1|-///-|A2|-+-|X1|-///-|X2|-+-|Y1|-///-|Y2|-+-|B1|-///-|B2| +--+ +--+ | +--+ +--+ | +--+ +--+ | +--+ +--+ | | | Yangguang Xu [Page 18] draft-xu-bgp-gmpls-00.txt Nov. 2001 1. Link Connection Verification Stage This stage makes sure both customer networks are willing to set up the connection and have enough capacity for the connection. In the example above, A and B are customers. X and Y are providers. A2 initiates the request, reserves the local resource and then forwards the request to X1. X1 selects the path and forwards the message to its BGP next hop X2. X1 doesn't invoke intra-domain signaling, but it may consult its IGP database to make sure the intra-domain path to X2 is available. The procedure repeated across different providers until the message reaches B1. If B1 is ready for the connection, it sets up the local connection and reverses the request to Y2. The connection setup process goes to the next stage. 2. Network Connection Setup Stage In this stage, the inter-domain signaling reverses the first request message. At provider networks, the message triggers intra-domain signaling for sub-network connection setup. At this stage, an intermediate provider shouldn't change the previous path selection result of the first stage. 3. Acknowledge Stage In this stage, both the network connection and link connection are set up. An acknowledgement is sent from A2 to B1. 6.3 LSP Protection Switching and Restoration For a LSP over multiple ASes, restoration could be done either end-to-end (customer-to-customer) or within the AS where the failure happens. The exact mechanism depends on business agreements of the involved parties. Details and implications to signaling protocols need further study. 7 Security Security is critical where different business domains interact. When service request across business domains, encryption and authentication mechanisms are required at the service interfaces. This document has discussed some issues. An overall security architecture for inter-domain circuit operation needs to be addressed further and deserves an independent document. Yangguang Xu [Page 19] draft-xu-bgp-gmpls-00.txt Nov. 2001 8 Acknowledgements The authors want to thank Charles Zhou, Raghu Srinivasan, Nabil Bitar and Roshan Rao for their reviews and comments. 9 Authors' Address Yangguang Xu 21-2A41, 1600 Osgood St. Lucent Technologies, Inc. N. Andover, MA 01845 Email: xuyg@lucent.com 10 Reference [BGP-4] Y. Rekhter and T. Li, "A Border Gateway Protocol 4 (BGP-4)", RFC 1771, March, 1995 [BGP-COMM] R. Chandra, P. Traina, and T. Li, "BGP Communities Attribute", RFC 1997, August 1996 [BGP-MP] Bates, Chandra, Katz, and Rekhter, "Multiprotocol Extensions for BGP4", February 1998, RFC 2283 [BGP-EXTCOMM] Ramachandra, Tappan, "BGP Extended Communities Attribute", February 2000, work in progress [BGP-ORF] Chen, Rekhter, "Cooperative Route Filtering Capability for BGP-4", March 2000, work in progress [GMPLS-AR] Mannie, et. al., "GMPLS Architecture", March, 2001, Work in progress [MPLS-VPN] Rosen, Rekhter, et. al., "MPLS/BGP VPN", June, 2000, Work in progress [GMPLS-SIG] P. Ashwood, et. al., "Generalized MPLS Signaling Functional Spec.", April 2001, work in progress [OIF-UNI] Many, OIF2000-125.4, "OIF UNI Functional Spec.", April 2001, work in progress [NE-FRMK] Y. Xu, "Internet Network Engineering Framework", July, 2001, Work in progress Yangguang Xu [Page 20] draft-xu-bgp-gmpls-00.txt Nov. 2001 [OSPF-GMPLS] K. Kompella, et. al., "OSPF Extension for GMPLS", April, 2001, work in progress [ASTN-LN] M. Vissesse, "ASTN Layer Network Architecture" [BGP-CONF] Traina, P., "Limited Autonomous System Confederations for BGP", RFC 1965, June 1996. [BGP-REF] Bates, T. and R. Chandra, "BGP Route Reflection An alternative to full mesh IBGP", RFC 2796, June 1996. Yangguang Xu [Page 21]