Network Working Group Eric C. Rosen Internet Draft Cisco Systems, Inc. Expiration Date: August 2002 February 2002 Applicability Statement for VPNs Based on rfc2547bis draft-rosen-ppvpn-as2547-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This document provides an Applicability Statement for the VPN solution described in [2547bis] and other documents listed in the References section. Rosen [Page 1] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 Table of Contents 1 Introduction ......................................... 2 2 SP Provisioning Model ................................ 4 3 Supported Topologies and Traffic Types ............... 6 4 Isolated Exchange of Data and Routing Information .... 7 5 Access Control and Authentication .................... 8 6 Security ............................................. 9 6.1 Protection of User Data .............................. 9 6.2 SP Security Measures ................................. 10 7 Addressing ........................................... 11 8 Interoperability and Interworking .................... 12 9 Network Access ....................................... 12 9.1 Physical/Link Layer Topology ......................... 12 9.2 Temporary Access ..................................... 12 9.3 Access Connectivity .................................. 13 10 Service Access ....................................... 13 10.1 Internet Access ...................................... 14 10.2 Hosting, ASP, Other Services ......................... 14 11 SP Routing ........................................... 15 12 Migration Impact ..................................... 15 13 Scalability .......................................... 16 14 QoS, SLA ............................................. 18 15 Management ........................................... 19 15.1 Customer Management of a VPN ......................... 19 16 Acknowledgments ...................................... 19 17 References ........................................... 19 1. Introduction This document provides an Applicability Statement for the VPN solution described in [2547bis] and other documents listed in the References section. We refer to these as "2547-style VPNs". It does so by comparing the characteristics of 2547-style VPNs against the requirements specified in [PPVPN-REQs]. Any VPN service is provided by a Service Provider to a Customer. 2547-style VPNs are intended for the situation in which: Rosen [Page 2] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 - The customer does not want to manage a routed backbone; the customer may be using routing within his sites, but wishes to outsource the inter-site routing to the SP. - The customer wants the SP to make the backbone and its routing completely transparent. If the customer has a routed infrastructure at his sites, he does not want his site routing algorithms to need to be aware of any part of the SP backbone network, other than the PE routers to which the sites are attached. In particular, the customer does not want his routers to need to be aware of either the native structure of the SP backbone, or of an overlay topology of tunnels through the SP backbone. - The customer is willing to send IP packets exclusively on this VPN. - The Service Provider has an IP backbone, possibly MPLS-enabled. - The Service Provider wants to provide a service which meets the customer requirements above. - The Service Provider does not want to maintain a distinct overlay topology of tunnels for each VPN customer. The basic principle is to model each VPN as a self-contained "internet", where each site makes one or two access connections to a SP, sends the SP its routing information, and then relies on the SP to distribute routing information to and from the other sites in that same VPN. The service differs from Internet service, however, in that the SP strictly controls the distribution of this routing information so that routes from within a VPN are not sent outside the VPN, unless that is explicitly authorized by the customer. In fact, even within the VPN, the distribution of routes may be controlled by the SP so as to meet some policy of the customer's. The routers at a given customer site need not be routing peers of the routers at other customer sites, and indeed need not know anything about the internal structure of other customer sites. In fact, different routing protocols may run at the different sites, with each site using whatever protocol is most appropriate for that particular site. In some cases, it is considered desirable for routers at different sites to know about each other (e.g., to support "IGP backdoors"). To support such cases, when a route is distributed from one site to another, the corresponding routing metric is distributed with it. If a customer runs OSPF within his sites and needs this functionality, Rosen [Page 3] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 the optional procedures of [2547-OSPF] should be used. If EBGP is used on the PE/CE access links, the SP and the customer do not peer in any IGP. 2547-style VPNs are optimized for the situation in which a customer (an enterprise) expects a service provider to operate and maintain the customer's "backbone" (i.e., the customer's inter-site routing). As such, the service provider becomes a "business partner" of the enterprise. The technical mechanisms accommodate the case in which a number of closely cooperating SPs can jointly offer the VPN service to a customer, in that the BGP-based route distribution mechanisms can operate between different SPs. If a set of SPs have sufficient agreements with respect to QoS, SLA, etc., then the customer's VPN could have sites attached to different SPs from that set. 2547-style VPNs are NOT an optimum solution for the customer who wants to purchase local connectivity from whatever ISP happens to offer the best price at a given site, nor for the customer who wants his VPN backbone to consist of tunnels running over the public Internet. [2547bis] specifies the inter-AS mechanisms that allow a single VPN to have sites attached to different SPs.. However, the design center is not an environment where a given VPN is spread among a very large number (e.g., hundreds) of SPs. However, in cases where remote offices, individual telecommuters, etc., must use the public Internet to access the VPN, it is possible to "tunnel" the remote traffic to a PE router, and the PE router will treat the traffic as if it had arrived over an interface connected to the PE. Remote PPP connections can be tunneled via L2TP to a PE router; IPsec tunnels can also be used to tunnel traffic to a PE router across the public Internet. Of course when the public Internet is used, issues such as QoS and SLAs must be carefully considered. 2. SP Provisioning Model If a particular VPN attaches to a particular PE router, the SP must configure that PE router with a VRF ("Virtual Routing and Forwarding table"), a routing table that is specific to the specified VPN. (This is known as a VFI, "VPN Forwarding Instance", in the language of [PPVPN-REQ] and [PPVPN-FRMWRK].) Each interface or sub-interface at that PE which attaches to a site in the specified VPN (i.e., each local access link of that VPN) must be configured so as to be associated with that VRF. Each such interface may be unnumbered, or may be assigned an address from within the VPN's address space. In Rosen [Page 4] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 general, a routing algorithm needs to be run on each of these links (though static routing can be used instead). The routing algorithm can be EBGP, or an IGP such as RIP or OSPF. (If OSPF is used, the procedures of [2547-OSPF] must be implemented.) In an IGP is run on the access links, the IGP MUST be a separate IGP instance, different than the IGP instance running among the backbone routers, and different than the IGP instance running on the access links of any other VPN. Static routing is also allowed. The VRF is populated automatically with routes distributed from locally attached CE routers via whatever routing algorithm is run on the PE/CE links. It is also populated automatically with routes distributed from other VRFs via BGP. Standard routing decision processes are used to automatically select the proper routes. Static configuration of routes in the VRF is optional. Each PE router must run BGP, and must be pre-configured with the identities of a small set of BGP Route Reflectors, with which it is to peer via IBGP. (In lieu of using Route Reflectors, one could configure each PE with the identities of all the other PEs, and set up a full mesh of IBGP connections. But the use of Route Reflectors is necessary to achieve adequate scalability. See section 4.3.3 of [2547bis] for a more complete discussion of the use of Route Reflectors, and related scalability mechanisms such as Outbound Route Filtering.) Each VRF must be configured with three parameters: - A Route Distinguisher. This is a globally unique 8-byte value. Each VRF may have a unique RD, or there may be a single unique RD for an entire VPN. When BGP is used to distribute VPN routing information across the SP backbone, this value is prepended to the VPN's IPv4 address prefixes, creating a new address family, the VPN-IPv4 address family. Thus even when two VPNs have overlapping IPv4 address spaces, they have unique VPN-IPv4 address spaces. - One or more Export Route Targets. An RT is a globally unique 8- byte value which BGP carries, as the Extended Communities Route Target attribute, along with routes that are exported form the VRF. - One or more Import Route Targets. This RT is used to select routes to be imported from other VRFs into this VRF. In the simplest cases and most common cases, the Export RT, Import RT, and RD can be identical, and all VRFs in the same VPN will Rosen [Page 5] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 distribute routes to each other (a typical intranet). In more complex cases, they can be set differently, allowing a very fine degree of control over the distribution of routes among VRFs. This can be used to create extranets, or to enforce various customer policies. In complicated cases, particular Export RTs can be assigned to particular routes using router management mechanisms. Adding a new site to a VPN is a matter of attaching the site's CE router to a PE router, configuring the interface, and, if a VRF for that VPN already exists in the PE router, associating that interface with the VRF. If a VRF for that VPN does not already exist in the PE, then one must be configured as specified above. Changes to the configuration of a PE are automatically reflected via BGP to the other PEs. The RTs and RDs are made unique by being structured as an SP identifier followed by a number which is assigned by the identified SP. SPs may be identified by their AS numbers, or by a registered IP address owned by that SP. Although RTs are encoded as BGP Extended Communities, the encoding itself distinguishes them from any other kind of BGP Extended Community. 3. Supported Topologies and Traffic Types The scheme is optimized for full inter-site connectivity, in the sense that this is what the simplest configurations provide. However, the SP has full control, through the mechanism of Route Targets, of the distribution of routing information among the set of VRFs. This enables the SP to provide hub-and-spoke or partial mesh connectivity as well as full mesh connectivity. Note that, strictly speaking, the scheme does not create a topology, as it does not create layer 2 connections among the sites. It does however allow for control over the IP connectivity among the sites. It is also possible to constrain the distribution of routing in arbitrary ways, e.g., so that data from site A to site B must travel through a third site C. (In fact, if it is desired to do so, this level of control can be specified at the granularity of a single route.) It is possible for some of the routes from a particular customer site A to be distributed to one set of remote sites, while other routes from site A are distributed to a different set of remote sites. This is done with the Route Target mechanism previously described. Rosen [Page 6] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 Unicast IP traffic is fully supported. Customer IP packets are passed transparently. Multicast IP traffic is optionally supported, if the SP provides the optional mechanisms of [2547-MCAST]. There are however scaling implications to the use of these mechanisms. Discussion of these implications is deferred. Non-IP traffic is not supported. If support for non-IP traffic is necessary, either the SP must additionally provide a layer 2 tunneling service, or the customer must use IP tunneling. 4. Isolated Exchange of Data and Routing Information The Route Target mechanism is used to control the distribution of routing information, so that routes from one VPN do not get sent to another. VPN routes are treated by BGP as a different address family than general Internet routes. Routes from a VRF do not get leaked to the Internet unless the VRF has been explicitly configured to allow it (and this is NOT the default). The way in which a particular VPN is divided into sites, or the topology of any particular VPN site, are hidden from the Internet. (Of course, if a particular site can received Internet traffic, and if it responds to traceroute probes from the Internet, then any user of the Internet can learn something about the site topology. The fact that the site is in a VPN does not make this any easier or any harder.) Similarly, Internet routes do not get leaked into the VPN, unless a VRF of that VPN is explicitly configured to import the Internet routes. Proper configuration is essential to maintaining the isolation. In particular, each access link must be associated with the proper VRF for that access link, and each VRF must be configured with the proper set of RTs. A number of means for exchanging reachability information between the PE and CE devices are supported: static routing, EBGP, and RIP are supported by the procedures of [2547bis]. If the procedures of [2547-OSPF] are implemented, OSPF may be used; if OSPF is used, and any VPN access link is an area 0 link, the procedures of [2547-OSPF- Area0] must be used as well. If OSPF is used between two VPN sites which are in the same OSPF area, and if it is desired for routes over the VPN backbone to be preferred to the OSPF intra-site routes, then the "sham link" procedures of [2547-OSPF] must be used. Rosen [Page 7] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 The routing protocols used among the customer routers are not in any way restricted by the VPN scheme, as whatever IGP is used within the VPN, the PE/CE access links may run EBGP, or may otherwise be in a different routing domain than the site's internal links. BGP is used for passing routing information among SPs. BGP may be authenticated by use of the TCP MD5 option, or by operating through an IPsec tunnel. Data traveling between two customer sites is encapsulated while in transit through the backbone. The encapsulation contains sufficient information to ensure that the packet is sent to the proper PE router, and then, in conjunction with the VRF and related information at that PE, to the proper CE routers. If two VPNs attach to the same PE, there is strict separation of forwarding at that PE, as well as strict separation of the routing information. Isolation of traffic is similar to that provided by classical VPNs (FR-based and ATM-based). As in classical VPNs, the customer must rely on the SP to properly configure the backbone network to ensure proper isolation, and to maintain the security of his communications gear. 5. Access Control and Authentication No particular means of PE/CE authentication is specified for 2547- style VPNs. PE/CE mutual authentication may be done via any mechanism supported by the routing protocol in which the CE and PE are peers (e.g., use of the TCP MD5 authentication when the CE/PE protocol is BGP), or by any other mechanism that may be desired. Currently there is no specified method by which a CE may be kept out of the VPN if it is not authorized by a customer-managed authentication mechanism. This is for further study. No particular means of access control is specified for 2547-style VPNs. It is possible to cause extranet traffic to be directed through a firewall, if that is desired. 2547-style VPNs are compatible with any ACL features that are supported on the PE routers. It is possible for various sorts of "tunnel interfaces" to be associated with a VRF. In this case, whatever authentication is natively used in the establishment of the tunnel interface may be used. For example, an IPsec tunnel can be used as an "access link" Rosen [Page 8] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 to attach a remote user or site to a VRF. The authentication procedure in this case is part of IPsec, not part of the VPN scheme. Where L2TP is used, each PPP session carried in an L2TP tunnel can be associated with a VRF. The SP's AAA server can be used to determine the VPN to which the PPP session belongs, and then the customer's AAA server can be given the opportunity to authenticate that session as well. 6. Security 6.1. Protection of User Data No particular means of ensuring user data security is specified for 2547-style VPNs. The optional procedures of [2547-IPsec] may be used to provide authentication and/or encryption of user data as it travels from the ingress PE to the egress PE. However, the data is exposed at those two PEs, as well as on the PE/CE access links. The customer may provide his own user data security by using IPsec tunnels that terminate within the customer sites. Such tunnels are transparent to the VPN scheme. Schemes which discover the remote tunnel endpoints automatically and then set up the tunnels automatically as needed are the best fit with this VPN technology. Note that there is no requirement in general that IPsec tunnels between customer sites terminate at CE routers. The use of end-to-end transport mode IPsec by the customer is also transparent to the VPN scheme. In fact, the VPN scheme is compatible with any use of security by the customer, as long as a cleartext IP header is passed from CE to PE. When data must cross the Internet to reach the ingress PE router, IPsec tunnels between the enduser and the PE router can be used; the PE router must then associate each IPsec tunnel with the proper VRF. This association would have to be based on user-specific information provided by the IKE exchange, such as a VPN-id. If data is going from one SP network to another, and must cross the public Internet to get between those two networks, IPsec tunnels can be used to secure the data. This would require bilateral agreement between the two SPs. BGP connections can also be passed through an IPsec tunnel if this is deemed necessary, in order to protect user data, by a pair of SPs. QoS/SLA factors would have to be carefully considered in this case. Rosen [Page 9] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 6.2. SP Security Measures In order to prevent traffic from illegitimately entering a VPN, the SP must take various security measures. The PE and P routers must themselves be secure against breakins (either from someone physically present or from the Internet), and neither P nor PE routers should form routing adjacencies to other P or PE routers without benefit of some kind of security. This may be authentication in the IGP, or physical security. The CE/PE access link should be secured in some manner, though it may be deemed the responsibility of the customer to ensure that the CE is not replaced by an illegitimate router. If the CE/PE access link is a tunnel over the Internet, then of course some sort of authentication protocol should always be used. LDP sessions and BGP sessions between PE and/or P routers should be authenticated. This can be done via the TCP MD5 option, or by use of IPsec. If the SP is providing the VPN service over an MPLS backbone, it should not accept MPLS packets from its external interfaces (i.e. interfaces to CE devices or to other providers' networks) unless the top label of the packet was legitimately distributed to the system from which the packet is being received. If the packet's incoming interface leads to a different SP (rather than to a customer), an appropriate trust relationship must also be present, including the trust that the other SP also provides appropriate security measures. If the SP is providing the VPN service by using an IP (rather than an MPLS) encapsulation, or if it accepts IP-encapsulated VPN packets from other SPs, it should apply filtering at its borders so that it does not accept from other SPs or from customers any IP packets which are addressed to the PE routers, unless appropriate trust relationships are in place. Cryptographic authentication of the encapsulated data packets is certainly advantageous when there are multiple SPs providing a single VPN. Rosen [Page 10] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 7. Addressing Overlapping customer addresses are supported. There is no requirement that such addresses be in conformance with RFC 1918. There is no requirement that customer VPN addresses be distinct from addresses in the SP network. Any set of addresses used in the VPN can be supported, irrespective of how they are assigned, how well they aggregate, whether they are public or private. However, the set of addresses which are reachable from a given site must be unique. Network address translation for packets leaving/entering a VPN is possible, and is transparent to the VPN scheme. There is nothing in the architecture to preclude the mechanisms from being extended to support IPv6, provided that the appropriate IPv6- capable routing algorithms are in place. I.e., PE/CE routing must support IPv6, and the PE-PE BGP must support the labeled IPv6 address family. The latter has not been specified, but its specification is obvious from the specification of the labeled IPv4 address family. The IGP used in the SP backbone need not be IPv6 capable in order to support customer IPv6 networks. In theory, the same could be said of other network layers, but in practice a customer who has non-IP traffic to carry must expect to carry it either in site-to-site IP tunnels, or using some additional service (such as a layer 2 service) from the SP. Layer 2 addresses and identifiers are never carried across the SP backbone. No restrictions are placed on the customer's addressing schemes or policies. Note though that the SP may place restrictions on the number of routes from a given customer site, or may charge differentially depending on the number of such routes, and such restrictions may have implications for the customer's addressing scheme. In particular, addressing schemes which facilitate route aggregation on a per-site basis will result in the most efficient use of the SP's resources, and this may be reflected in SP charging policies. Rosen [Page 11] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 8. Interoperability and Interworking Interoperability should be ensured by proper implementation of the published standards. Direct PE-PE interworking over the SP backbone with other VPN solutions is not supported. As all the different types of L3VPNs are IP networks, they can of course interwork in the same way that any two IP networks can interwork. For example, a single site can contain a CE router of one VPN scheme and a CE router of another VPN scheme, and these CE routers could be IGP peers, or they might even be the same CE router. This would result in the redistribution of routes from one type of VPN to the other, providing the necessary interworking. 9. Network Access 9.1. Physical/Link Layer Topology The architecture and protocols do not restrict the link layer or the physical layer in any manner. 9.2. Temporary Access Temporary access via PPP is possible, using industry standard PPP- based authentication mechanisms. For example: - A dial-up user (or other PPP user) is authenticated by the PE, using the SP's AAA server, based on a login string or on the number dialed. - The SP's AAA server returns a VPN-id to PE. - The PE assigns the user to a VRF, based on that VPN-id. - The user is then authenticated by a AAA server within the VPN (i.e., managed by the customer rather than by the SP). This AAA server would typically be addressed through the VRF (i.e., may be in VPN's private address space). - The user gets disconnected if either authentication step is unsuccessful. IPsec access to a VRF is also possible. In this case, the security association is between the enduser and the SP. Rosen [Page 12] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 In these ways, a user can access a 2547-style VPN via the public Internet. There is no explicit support for mobility, other than what is stated above. 9.3. Access Connectivity Homing of a CE to two PEs is fully supported, whether the two PEs are on the same SP network or not. If a CE is connected to two PEs in the same SP network, both connections will be used to carry traffic, as the routing in the backbone of the traffic to the CE is determined by the BGP decision process of the ingress PE for that traffic. Thus traffic from different ingress PEs to a particular CE may be sent over different PE-CE links, depending on the backbone routing itself. (By default, a particular ingress PE would choose the egress PE which is closest to it according to the backbone network's IGP routing metric.) Splitting of the traffic from a single ingress PE can be done, if the ingress PE router has the "IBGP load sharing feature". If a CE is connected to two PEs in different networks, both connections will be used to carry traffic, but traffic from a single ingress PE will in general not be split, as this would require an "EBGP load sharing" feature. At the current time, EBGP load sharing is only done in certain specific scenarios. BGP contains a multitude of knobs which allow a SP to control the traffic sent on one PE-CE link as opposed to the other. One can also make use of the Link Bandwidth extended community to control how traffic is distributed among multiple egress PE-CE links. The VPN scheme is of course compatible with the use of traffic engineering techniques (RSVP-TE based or otherwise) in the backbone network. 10. Service Access Rosen [Page 13] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 10.1. Internet Access Internet access and VPN access are possible from the same site. This is even possible over the same interface, as long as the VPN's internal addresses are distinct from the addresses of the systems which must be reached via the Internet. This requires only that Internet routes as well as VPN routes be imported into the VRF associated with that interface. This may be as simple as putting a default route to the Internet into that VRF. The "route to the Internet" which is in a particular VRF need not lead directly to the Internet, it may lead to a firewall or other security device at another site of the VPN. The VPN customer can cause this to happen simply by exporting a default route from the site with the firewall. Generally, a site with a firewall will use a different virtual interface for Internet access than for VPN access, since the firewall needs to distinguish the "clean interface" from the "dirty interface". In such a configuration, the customer would export his routes to the Internet via the firewall's dirty interface, but would export the same routes to the VPN via the clean interface. Thus all traffic from the Internet would come through the dirty interface, then though the firewall, and possibly go to another VPN site though the clean interface. This also allows any necessary NAT functionality to be done in the firewall. 10.2. Hosting, ASP, Other Services Any externally provided service can be accessed from the VPN, provided that it can be addressed with an address that is not otherwise in use within the VPN. Access can be firewalled or non- firewalled. If the client accessing the service does not have a globally unique IP address, and a single server provides a service to multiple VPNs, NAT will have to be applied to the client's packets before they reach the server. This can be done at a customer site, or by a VRF-specific NAT function in a PE router. Rosen [Page 14] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 11. SP Routing Routing through the backbone is independent of the VPN scheme, and is unaffected by the presence or absence of VPNs. The only impact is that the backbone routing must carry routes to the PE routers. The VPN routes themselves are carried in BGP as a distinct address family, different than the address family that is used to carry "ordinary" IP routes. These routes are passed from PE router to Route Reflector to PE router, and are never seen by the P routers. The Route Reflectors that carry the VPN routes can be entirely separate from the Route Reflectors that carry the "ordinary" IP routes. The fact that two PE routers support a common VPN does not require those PE routers to form an IGP routing adjacency between themselves. The number of adjacencies in the backbone IGP is independent of and unrelated to the number of VPNs supported by any set of PE routers. No VPN-specific protection and restoration mechanisms are needed; these are general routing considerations, and the VPN scheme is compatible with any protection and restoration mechanisms that may be available. The SP does not manage the customer's IGP in any way, and routes are never leaked between the SP's IGP and any customer's IGP. If the PE/CE protocol is EBGP, the SP and the customer do not ever participate in a common IGP. 12. Migration Impact Generally, this means replacement of an existing legacy backbone with VPN backbone. The general migration mechanism would be to hook up the sites one at a time to the VPN backbone, and to start giving the routes via the VPN backbone preference to routes via the legacy backbone. Details depend on the legacy backbone's IGP. In general, one would have to manipulate the IGP metrics to provide the proper route preference. If the legacy backbone routing protocol is OSPF, then migration is best done with OSPF as the PE/CE protocol and the PE supporting the [2547-OSPF] procedures, OR with BGP as the PE/CE protocol, and the CE supporting the BGP/OSPF interaction specified in [2547-OSPF]. With other legacy backbone routing protocols, the proper metrics must be set at the point (PE or CE) where the BGP routes from the SP network are being redistributed into the legacy IGP. Rosen [Page 15] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 13. Scalability There is no upper limit on the number of VPNs per SP network, as there is no one box in the SP network which needs to know of all VPNs. Knowledge of a particular VPN is confined to the PE routers that attach to sites in that VPN, and to the BGP Route Reflectors which receive routing data from those PEs; other systems maintain no state at all for the VPN. Note though that there is no need for any one Route Reflector to know of all VPNs. The SP network IGP does need to carry routes for all of the PE routers, so the total number of PE routers is limited by the number of routes which the IGP can carry. There is no upper limit on the number of site interfaces per VPN, as state for a particular interface is maintained only at the PE router to which that interface attaches. The number of site interfaces per VPN at a given PE router is limited only by the number of interfaces which that PE router can support. The number of routes per VPN is constrained only by the number of routes that can be supported in BGP, the number of routes which can be maintained in the PEs that attach to that VPN, and the number of routes which can be maintained in the BGP Route Reflectors which hold the routes of that VPN. The major constraint in considering scalability is the number of routes which a given PE can support. In general, a given PE can support as many VPNs as it has interfaces (including virtual interfaces or "sub-interfaces", not just physical interfaces), but it is constrained in the total number of routes it can handle. The number of routes a given PE must handle depends on the particular set of VPNs it attaches to, and the number of routes in each such VPN, and the number of "non-VPN" Internet routes that it must also handle. The SP may need to engage in significant planning to ensure that these limits are not often reached. If these limits are reached, it may be necessary either to replace the PE with one of larger capacity, or to reorganize the way in which access links lead from CEs to PEs, in order to better concentrate the set of access links from sites that are in the same VPN. Rehoming a site to a different PE may not involve actual rewiring; if the access technology is switched, this is a matter of provisioning, but may still be a significant undertaking. If it is necessary to have down time while performing the rehoming, the customer is impacted as well. Rehoming can also be done "virtually", by creating a layer 2 tunnel from a CE's "old" PE to its "new" PE. Rosen [Page 16] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 An important consideration to remember is that one may have any number of INDEPENDENT BGP systems carrying VPN routes. This is unlike the case of the Internet, where the Internet BGP system must carry all the Internet routes. The difference stems from the fact that all Internet addresses much be reachable from each other, but a given VPN address is only supposed to be reachable from other addresses in the same VPN. Scalability is also affected by the rate of changes in the reachability advertisements from CE to PE, as changes reported by a CE to its attached PE may be propagated to the other PEs. BGP mechanisms to control the rate of reported changes should be used by the SP. Another constraint on the number of VPNs which can be supported by a particular PE router is based on the number of routing instances which the PE router can support. If the PE/CE routing is static, or is done by BGP, the number of routing protocol instances in a PE device does not depend on the number of CEs supported by the PE device. In the case of BGP, a single BGP protocol instance can support all CEs that exchange routing information using BGP. If the CE/PE router is done via RIP or OSPF, then the PE must maintain one RIP or OSPF instance per VRF. Inter-AS scenarios constructed according to option (b) of section 10 of [2547bis] require BGP "border routers" to hold the routes for a set of VPNs. If two SPs share in a small number of VPNs, a single border router between them may suffice. As the number of shared VPNs increases, additional border routers may be needed to handle the increased number of routes. Again, no single border router would handle all the routes from all the VPNs, so an increase in the number of VPNs can always be supported by adding more border routers. Inter-AS scenarios constructed according to option (c) of section 10 of [2547bis] eliminates the need for border routers to contain VPN routes (thus improving scalability in that dimension), but at the cost of requiring that each AS have a route to the PEs in the others. (Inter-AS scenarios constructed according to option (a) of section 10 of [2547bis] do not scale well.) The solution is intended to simplify CE and site operations, by hiding the structure of the rest of the VPN from a site, and by hiding the structure of the backbone. Thus CEs need have only a single sub-interface to the backbone, CEs at one site need not even be aware of the existence of CEs at another, and CEs at one site need not be routing peers of CEs at another. CEs are never routing peers of P routers. This helps to scale the customer's network. Rosen [Page 17] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 It is also intended to simplify the SP's VPN provisioning, so that potentially the SP will have to do little more than say which sites belong to which VPNs. However, as the system scales up, planning is needed to determine which PEs should home which VPNs, and which BGP RRs should take which VPNs' routing information. P routers maintain NO per-VPN state at all; the only requirement on them is to maintain routes to the PE routers. However, certain VPN multicast schemes require per-multicast-group state in the P routers, summed over all VPNs. Others require only no state in the P routers at all, but will result in sending more unnecessary traffic. The complete set of tradeoffs for multicast is not that well understood yet. Note that as the scaling of a particular PE is primarily a matter of the total number of routes which it must maintain, scalability is facilitated if the addresses are assigned in a way that permits them to be aggregated. (I.e., if the customers have a sensible addressing plan.) 14. QoS, SLA The provision of appropriate QoS capabilities may require any combination of the following: - QoS in the access network. - Admission control (shaping, policing) on the access links. - Traffic engineering in the backbone. - Intserv/diffserv classification by the PE, for traffic arriving from the CE. Once the PE classifies the user packets, this classification needs to be preserved in the encapsulation (MPLS or IP) used to send the packet across the backbone. - DSCP mapping. - DSCP transparency. - Random Early Discard in the backbone. None of these features are VPN-specific. The ability to support them depends on whether the features are available on the edge and core platforms, rather than on any particular VPN scheme. Rosen [Page 18] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 2547-style VPNs can support either the "hose model" or the "pipe model", though the hose model is a more natural fit and is what would tend to be used by default; the addition of pipes would tend to be done on a more limited basis. 15. Management 15.1 SP Management TBD 15.1. Customer Management of a VPN CE devices may be managed from within the VPN, transparently to the SP. The SP may manage CE devices from a central site, provided that a route to the central site is exported into the CE's VPN, and the central site is in a VPN into which the routes to the managed CE devices have been imported. This is a form of extranet. If the central site is managing CE devices from several VPNs, those CE devices must have mutually unique addresses. Note that this does not enable the CE devices from different VPNs to reach each other. Customer access to a PE device is totally at the discretion of the SP, but is not required by the solution. The PE device is a routing peer of a CE device, and can be pinged, etc. 16. Acknowledgments Many thanks to Jeremy De Clercq, Luyuan Fang, Dave McDysan, Ananth Nagarajan, Yakov Rekhter, and Muneyoshi Suzuki, for their comments, criticisms, and help in preparing this document. 17. References [2547bis] "BGP/MPLS VPNs", Rosen, Rekhter, et. al., draft-ietf- ppvpn-rfc2547bis-01.txt, January 2002 [2547-GRE-IP] "Use of PE-PE GRE or IP in RFC2547 VPNs", Rekhter, Rosen, draft-ietf-ppvpn-gre-ip-2547-01.txt, February 2002 [2547-OSPF] "OSPF as the PE/CE Protocol in BGP/MPLS VPNs", draft- Rosen [Page 19] Internet Draft draft-rosen-ppvpn-as2547-00.txt February 2002 rosen-vpns-ospf-bgp-mpls-04.txt, Rosen, Psenak, Pillay-Esnault, January 2002 [2547-OSPF-Area0] "OSPF Area 0 PE/CE Links in BGP/MPLS VPNs", Rosen, Psenak, Pillay-Esnault, draft-rosen-ppvpn-ospf2547-area0-00.txt, January 2002 [2547-IPsec] "Use of PE-PE IPsec in RFC2547 VPNs", Rosen, De Clercq, Paridaens, T'Joens, Sargor, draft-ietf-ppvpn-ipsec-2547-01.txt, February 2002 [2547-MCAST] "Multicast in MPLS/BGP VPNs", Rosen, Cai, Tappan, Wijsnands, Rekhter, Farinacci, draft-rosen-vpn-mcast-02.txt, July 2001 [PPVPN-FRMWRK] "A Framework for Provider Provisioned Virtual Private Networks", Callon, Suzuki, De Clercq, Gleeson, Malis, Muthukrishnan, Rosen, Sargor, Yu, draft-ietf-ppvpn-framework-03.txt, January 2002 [PPVPN-REQS] "Service requirements for Provider Provisioned Virtual Private Networks", Carugi, McDysan, Fang, Johansson, Nagarajan, Sumimoto, Wilder, draft-ietf-ppvpn-requirements-03.txt, November 2001 Rosen [Page 20]