Fabio Chiussi Lucent Technologies Jeremy De Clercq Alcatel Sudhakar Ganti Tropic Networks Wing Cheong Lau Lucent Technologies Biswajit Nandy Tropic Networks Provider Provisioned VPN WG Nabil Seddigh Internet Draft Sven Van den Bosch Alcatel Document: draft-chiussi-ppvpn-qos-framework-01.txt Expires: September 2003 March 2003 Framework for QoS in Provider-Provisioned VPNs Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Expires September 2003 1 PPVPN QoS framework March 2003 Abstract This document defines the QoS framework for Layer 2 and Layer 3 provider-provisioned VPNs. The scope of this document includes MPLS, IPSec, GRE, and L2TP VPNs. This document focuses on the QoS aspects that are specific of PPVPNs, and discusses issues pertaining to Service Level Agreements, provisioning, resource allocation, signaling, and protection/restoration. Both non-hierarchical and hierarchical VPNÆs are addressed. Table of Contents Status of this Memo................................................1 Abstract...........................................................2 Conventions used in this document..................................3 1. Introduction...................................................3 2. Background.....................................................5 2.1. PPVPN Architectures of Interest.............................5 2.1.1. Provider Provisioned PE-based L3 VPNs.....................5 2.1.2. Provider Provisioned CE-based VPNs........................6 2.1.3. Provider Provisioned MPLS-Based L2 VPNs...................7 2.2. VPN Tunneling Mechanisms of Interest........................7 2.3. QoS Building Blocks........................................10 3. Service Models................................................11 3.1. QoS Frameworks in PPVPNÆs..................................11 3.1.1. Scope....................................................11 3.1.2. Reference models.........................................11 3.2. VPN Endpoints (Service Access Point) Identification........17 3.3. Hard and Soft QoS Guarantees, Aggregated Models and Non- Aggregated Models.................................................18 3.4. QoS OF the VPN, QoS WITHIN the VPN.........................20 4. QoS Guarantees, Service Level Agreements, and Management in PPVPNs............................................................20 4.1. QoS Guarantees.............................................20 4.2. VPN Service Level Specification (SLS)......................22 4.2.1. SLS structure............................................22 4.2.2. Layer 3 SLS template.....................................22 4.2.3. Layer 2 SLS template.....................................25 4.2.4. SLS example..............................................25 4.3. Management of QoS VPN......................................25 5. Other QoS issues in PPVPNÆs...................................25 5.1. Learning issues............................................26 5.2. Marking issues.............................................26 5.3. Garbage collection, resource de-allocation/recovery........26 5.4. Specific QoS issues........................................26 5.4.1. QoS in RFC 2547 VPNÆs....................................26 Expires September 2003 2 PPVPN QoS framework March 2003 5.4.2. QoS in VR VPNÆs..........................................26 5.4.3. QoS in L2 LDP-based VPNÆs................................26 5.4.4. QoS in L2 BGP-based VPNÆs................................26 5.4.5. QoS in IPSec VPNÆs.......................................27 5.4.6. QoS in L2TP VPNÆs........................................27 5.4.7. QoS in GRE VPnÆs.........................................27 6. Traffic Engineering, QoS Routing, Protection/Restoration and their Relation with PPVPN QoS.....................................27 6.1. Traffic Engineering and QoS Routing........................27 6.2. Protection and Restoration.................................28 6.2.1. Review of protection schemes of interest.................28 6.2.2. Network-specific aspects.................................28 6.2.3. PPVPN-specific aspects...................................29 6.2.4. Required signaling extensions û building blocks..........29 6.2.5. Traffic engineering considerations in protection.........30 7. QoS Signaling in PPVPNs.......................................30 7.1. Resource Allocation........................................30 7.2. Resource Reservation tasks for PPVPNs......................31 7.3. "Partial" QoS Signaling Approaches, Existing Proposals.....32 7.4. Towards Full QoS signaling support for PPVPNs..............33 7.4.1. Signaling of VPN endpoint identifiers....................33 7.4.2. Coordinating VC and Outer Tunnel QoS Signaling...........34 8. Future Work...................................................35 9. Security Considerations.......................................35 References........................................................35 Author's Addresses................................................39 Full Copyright Statement..........................................39 Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [1]. 1. Introduction The provision of Quality of Service (QoS) is an intrinsic part of emerging services centered on provider-provisioned VPNs (PPVPNs). The standardization of issues related to QoS has been the focus of several working groups within IETF. However, the provision of QoS in the context of PPVPNs introduces a number of QoS issues that are specific of PPVPNs, and have not been addressed by existing work on QoS. Furthermore, some of the specific PPVPN solutions that have recently emerged have their own QoS aspects that need to be defined. For these reasons, there is a need for a standardization effort on Expires September 2003 3 PPVPN QoS framework March 2003 QoS that builds on what defined by other working groups, but is explicitly targeted at PPVPNs. The scope of this document is the provision of QoS within the PPVPN cloud only. In other words, we focus on the provision of QoS from one VPN endpoint to another. Clearly, in order to provide end-to-end QoS, the QoS issues specific to the portion of the network from the end user to the edge of the PPVPN cloud also have to be addressed. Such issues are outside of the scope of this document, since they have been addressed by other working groups and are not specific of PPVPNs. This document discusses QoS aspects pertaining to Layer 3 VPNs, Layer 2 VPNs, and CE-based VPNs (material on CE-based VPNÆs will be added in future version of this document). The scope of this document encompasses different technologies that are used for PPVPNs; in particular, MPLS-based VPNs, IPSec VPNs, GRE VPNs, and L2TP VPNs are discussed. Prior to addressing specific VPN solutions, a very general issue that needs to be defined is which QoS models and QoS frameworks are relevant for PPVPNs. These models and frameworks can become quite complex, especially when hierarchical VPNÆs are considered. As a consequence, a wide range of variations is possible in the notion of QoS that a PPVPN service may offer to the end customers. These variations, ranging from simple to very elaborate, are specified through corresponding Service Level Agreements, which need to be defined. In general, in the context of PPVPNs, two levels of QoS are relevant: QoS of the VPN and QoS within the VPN. In fact, two scenarios are possible, and are likely to coexist in the same network. A first possibility is that a given VPN may be assigned a specific level of QoS. In this configuration, a user with traffic of different QoS requirements would require more than one VPN, one for each of the traffic types (of course, a similar scenario would be one where the provider sets up different VPNs to handle the different types of traffic, but does so transparently to the user). An alternative scenario is one where a given VPN may carry traffic of different characteristics, which requires different QoS treatment within the VPN. In this case, a user with traffic of different QoS requirements only needs a single VPN. In this latter case, signaling mechanisms capable to set up separate resources within the VPN need to be defined. The possible solutions to support either of these scenarios may introduce a number of subtle issues, including learning across the various levels of QoS. Once the relevant QoS frameworks are established, a first, most obvious QoS issue that is specific of PPVPNs stems from the structure of outer tunnel/inner tunnel that is characteristic of most emerging VPN solutions. There is a need to allocate QoS resources in the network for both outer and inner tunnels, possibly incrementally. QoS provides compelling reasons for supporting multiple outer tunnels Expires September 2003 4 PPVPN QoS framework March 2003 between PEs, which introduces new requirements in the binding of inner and outer tunnels. Accordingly, a number of signaling extensions to existing protocols are needed to support the provision of QoS in the VPN context. Similarly, in MPLS VPN solutions, the presence of two labels requires the marking of QoS information (policing and/or class of service information) at both levels. Finally, once QoS is considered, auto-discovery of the VPN endpoints and automatic provisioning of the VPN service become challenging, and need to be specified. The multipoint nature of many VPN services introduces a level of complexity in terms of QoS that has not yet been exhaustively addressed in other contexts. Although PPVPNs are not the only multipoint service requiring QoS, they are a prominent example that can be used to drive the standardization effort for many aspects of multipoint QoS. 2. Background 2.1.PPVPN Architectures of Interest The PPVPN framework documents [VPN-L3FW][VPN-L2FW] distinguish different types of VPNs: Provider Provisioned PE-based L3 VPNs, Provider Provisioned CE-based VPNs, and Provider Provisioned L2 VPNs. The PPVPN WG explicitly aims at defining mechanisms to provide VPN services over an IP or MPLS network. This means that different tunneling technologies can be used to tunnel the VPN traffic over the shared SP backbone network. Examples of such tunneling technologies are MPLS, IP-in-IP, IPsec, L2TP, and GRE. Both the type of VPN (the type of service offered to the customer) and the tunneling technology used in the SP backbone network can have implications with regards to the offering of QoS. This section gives some background information on each of these concepts, which we use in the rest of this document. It is assumed that the reader is familiar with the terminology used in [VPN-term], [VPN-L3FW], [VPN-L2FW]. The term "Virtual Private Network" (VPN) refers to the communication between a set of sites, making use of a shared network infrastructure. For the members of the VPN, the network ôfeelsö like a private network, although the communication may occur over a shared, public infrastructure. 2.1.1. Provider Provisioned PE-based L3 VPNs A layer 3 PE-based VPN is one in which the Service Provider (SP) takes part in IP level forwarding based on the customer network's IP Expires September 2003 5 PPVPN QoS framework March 2003 address space. In general, the customer network is likely to make use of private and/or non-unique IP addresses. This implies that the PE devices that are directly attached to the customer understand the IP address space as used in the customer network and keep it separate from the possibly overlapping address spaces used by other customers. These PE devices typically implement Virtual Routing and Forwarding instances (VFI) for VPN context separation. Two approaches for providing PP PE-based L3 VPNs are being considered within the PPVPN working group. The "BGP/MPLS VPN" approach [rfc2547bis] uses MPLS for packet forwarding in the SP's backbone network, and uses BGP between PEs for VPN membership auto-discovery and for inter-site VPN reachability information distribution. The PE routers implement Virtual Routing and Forwarding tables (VRF) for IP context separation. Extensions to the base ôBGP/MPLS VPNö approach are defined in [PE-GRE] and [PE-IPsec] for the use of other tunneling mechanisms in the SP's backbone. The ôVirtual Routerö-based approach [VR-VPN] allows for the use of any tunneling mechanism for packet forwarding in the SPÆs backbone network. ôVirtual Routersö (VR) are implemented in the PE devices for VPN context separation. Tunnels are established between the VRs of the same VPN in the different PEs. For each VPN, the VPN reachability information is distributed by tunneling VPN-specific routing protocol messages through the tunnels that interconnect the VRs. The BGP-based mechanism described in [BGP-VPN] can be used for VPN membership discovery. 2.1.2. Provider Provisioned CE-based VPNs In CE-based VPNs, the customer network is supported by tunnels which are set up between CE devices. The tunnels may make use of various encapsulations (such as MPLS, GRE, IPsec, IP-in-IP, or L2TP tunnels) to send traffic over a shared IP/MPLS SPÆs backbone network. For provider provisioned CE-based VPNs, the provisioning and management of the tunnels is performed by the SP. The provider may also configure routing protocols on the CE devices. Routing in the private network is partially under the control of the customer, and partially under the control of the SP. However, since the VPN tunnels originate and terminate at the CE devices, the SP network is not involved in any specific VPN task, and does not take part in IP level forwarding based on the customer networkÆs IP address space. The tunneled customer packets might even be encrypted and as such be unreadable by the SPÆs network elements. The SPÆs involvement in the customerÆs VPN is solely by means of its management and provisioning system. In the PPVPN WG, the efforts in the CE-based VPN space currently focus on the use of IPsec to protect the inter-site customer traffic [IPsec-VPN]. Expires September 2003 6 PPVPN QoS framework March 2003 2.1.3. Provider Provisioned MPLS-Based L2 VPNs ATM and Frame Relay provider networks were commonly used to provide point-to-point layer 2 services to end-customers. Recently, there has been renewed interest in providing such layer 2 services over IP or MPLS-based networks. Typically, this would be achieved by provisioning ôvirtual circuitsö that run over IP or MPLS tunnels in the provider networks. a) Point-to-Point The most basic type of L2 VPN service is a point-to-point one where Layer 2 traffic from one customer site is transported transparently to a remote customer site. The L2 packets from the customer site are encapsulated by the ingress PE device, mapped to a pre-defined LSP for that virtual connection, transported to the egress PE device, and delivered to the remote CE. [Martini-ENC] defines the way in which such Layer 2 packets are encapsulated over the provider VC tunnel. Martini-transport [Martini-TRANS] defines LDP protocol extensions to support this feature. A different approach to Layer 2 VPNÆs is described in [Kompella], where similar features are provided using BGP extensions, in a manner fairly similar to the Layer 3 BGP-based approach. b) VPLS & TLS There has been high interest in extending the L2 point-to-point service for Ethernet services in order to build Transparent LAN services (TLS). The primary goal of such services is to make the service provider network appear as a single distributed Layer 2 Ethernet switch in relation to the various CE devices of the customer. There have been various proposals in this area [Lasserre], [Kompella- DTLS], and [Sajassi]. In most cases, each CE site only requires a single connection to the provider. The PE device is responsible for learning and forwarding of customer packets to the appropriate remote PE device. Encapsulation of customer packets occurs in the same manner as the point-to-point case. The difference between the various proposals is in where the MAC learning/broadcasts/forwarding are to occur. In some cases, all learning can be done on the ingress PE. In other cases, learning occurs on both the ingress and egress PE. In yet other cases, the PE is considered to be distributed in order to provide scalability. BGP or LDP may be used as protocols to aid in auto-discovery of PEs that are connected to CE(s) for a particular customer. 2.2.VPN Tunneling Mechanisms of Interest Expires September 2003 7 PPVPN QoS framework March 2003 The following tunneling mechanisms are within the scope of this document. o MPLS [RFC3031] [RFC3035] MPLS allows for tunnel multiplexing (æinnerÆ and æouterÆ LSPs) and for the transport of data packets other than IP. Tunnels may be established using RSVP-TE or (CR-)LDP. o IP-in-IP [RFC2003] [RFC2473] IP-in-IP encapsulation allows an IP datagram to be encapsulated within another IP datagram. Once the encapsulated datagram arrives at the intermediate destination (as specified in the outer IP header), it is decapsulated, yielding the original IP datagram, which is then delivered to the destination indicated by the original destination address field. IP-in-IP encapsulation doesnÆt explicitly allow for multiplexing, though this can be achieved by using different (outer) IP addresses for different VPNs. IP-in-IP needs extensions to carry packets other than IP. There are no standard mechanisms to establish and maintain IP-in-IP tunnels. IP-in-IP tunneling itself does not have intrinsic QoS/SLA capabilities. But, mechanisms such as RSVP extensions [RFC2764] or DiffServ extensions [RFC2983] may be used with it. o GRE [RFC2784] Generic Routing Encapsulation (GRE) specifies a protocol for encapsulating an arbitrary payload protocol over an arbitrary delivery protocol. A GRE tunnel is a communication path between two endpoints established by the use of GRE. A ækey fieldÆ [RFC2890] extension to GRE is defined to support multiplexing. GRE is assumed to support any type of payload protocol. There are no standard ways for setting up and maintaining GRE tunnels. GRE itself does not have intrinsic QoS/SLA capabilities. These capabilities depend on the delivery protocol (IP/MPLS). Other mechanisms such as Diffserv or RSVP extensions [RFC2746] may be used with it. o IPsec [RFC2401] [RFC2402] [RFC2406] [RFC2409] IP Security (IPsec) provides security services at the IP layer [RFC2401]. It comprises authentication header (AH) protocol [RFC2402], encapsulating security payload (ESP) protocol [RFC2406], and Internet key exchange (IKE) protocol [RFC2409]. AH protocol provides data integrity, data origin authentication, and an anti-replay service. ESP protocol Expires September 2003 8 PPVPN QoS framework March 2003 provides data confidentiality and limited traffic flow confidentiality. It may also provide data integrity, data origin authentication, and an anti-replay service. AH and ESP may be used in combination. IPsec may be employed in either transport or tunnel mode. In transport mode, either an AH or ESP header is inserted between the IPv4 header and the transport protocol header. In tunnel mode, an IP packet is encapsulated with an outer IP packet header. Either an AH or ESP header is inserted between them. AH and ESP establish a unidirectional secure communication path between two endpoints, which is called a security association. In tunnel mode, two security associations compose a tunnel between PE devices. The IKE protocol is used to set up IPsec tunnels. The SPI field of AH and ESP is used to multiplex security associations (or tunnels) between two peer devices. It is not possible however, to multiplex traffic from different VPNs within the same security association. IPsec needs extensions to carry packets other than IP. Alternatively, GRE may be used with it. IPsec itself does not have intrinsic QoS/SLA capabilities. Other mechanisms such as æRSVP Extensions for IPsec Data FlowsÆ [RFC2207] or DiffServ extensions [RFC2983] may be used with it. CE-based VPNs may use tunnel-mode IPsec (or may perform transport-mode IPsec on IP-in-IP encapsulated packets) with ESP to encrypt the customerÆs packets before sending them to the PE device. This means that the information in the customerÆs IP packet (IP header and payload) is unusable by the PE device. An important consequence is that the SPÆs network cannot use this information for QoS-related tasks. Some of the IP packetÆs IP header information (such as the DSCP) might be copied or translated though into equivalent information in the (visible) outer IP header, at the CE device. o L2TP [L2TPv3] The Layer Two Tunneling Protocol (L2TP) provides a dynamic tunneling mechanism for multiple Layer 2 (L2) circuits across a packet-oriented data network. The base L2TP protocol consists of (1) the control protocol for dynamic creation, maintenance, and teardown of L2TP sessions, and (2) the L2TP data encapsulation to multiplex and demultiplex L2 data streams between IP-connected L2TP nodes. L2TP itself does not have intrinsic QoS/SLA capabilities. Other mechanisms, such as æL2TP Differentiated Services ExtensionÆ [DS-L2TP] may be used with it. Expires September 2003 9 PPVPN QoS framework March 2003 2.3.QoS Building Blocks Many building blocks are necessary to enable a network to provide QoS. They include: 1) Traffic parameter signaling and associated resource reservation (also known as Call Admission Control (CAC)). The resource reservation is generally performed based on signaled traffic parameters (e.g., RSVP-TE or CR-LDP allows traffic parameters to be associated with each LSP). 2) Traffic queuing and appropriate scheduling. Common examples of the scheduling techniques include strict priority, Round Robin, or Weighted Fair Queuing. The schedulers may be either work conserving or non-work conserving (shapers). 3) Traffic classification in order to associate a packet to a traffic flow, and to corresponding QoS resources. Depending upon the service scenarios, the rules can be either very simple (e.g., all the traffic from port #1) or a complex set (e.g., multi-field classification such as IP addresses, VLAN tags, port numbers, ToS bytes, MPLS labels). When an incoming packet matches to a classification rule, the packet is treated as per its QoS requirements at all the building blocks within the node (policing, queueing, congestion control and scheduling) 4) Traffic policing to ensure that a flow does not exceed its traffic contract. Traffic policers can mark the traffic with colors (green, yellow or red) based on the conformance definition and take action either to pass or drop or remark (if the packet is already colored) the traffic. For example, [RFC2697] describes a single rate three color marker and [RFC2698] describes a two rate three color marker. The traffic that is passed and colored is appropriately buffered in the node and scheduled for transmission onto a link. In turn, the scheduling should ensure that all the packets (though colored differently) of a given flow belonging to a given class are delivered in sequence. A service provider may deploy traffic policers at the SAPs to limit the traffic, at the network edge. There can also be network scenarios where traffic policers may need to be deployed within the network at some traffic merge points and egress points depending upon the network scenario. 5) Buffer management and congestion control techniques. Buffer management involves how the total available buffer is allocated to the traffic classes (e.g, complete partitioning, complete sharing or partial sharing with mimimum allocation). The buffer management schemes should be able to handle multiple class types and colors. Congestion control is the treatment of traffic when buffer congestion occurs. Techniques that may be employed include tail drop push out, etc. This type of congestion control is performed at the Expires September 2003 10 PPVPN QoS framework March 2003 onset of congestion. Other commonly employed techniques use congestion avoidance for buffer control. Random Early Discard (RED) is one such active queue management technique that may drop traffic based on average queue depths (instead of instantaneous) and a drop probability. 3. Service Models 3.1.QoS Frameworks in PPVPNÆs 3.1.1. Scope As mentioned in the Introduction, the scope of this document is to discuss QoS from VPN Endpoint to VPN Endpoint. In other words, we do not address QoS in the last leg to the CE. The last leg is not specific of PPVPNÆs and is addressed by other WGÆs. We define the VPN Endpoint in the port of a providerÆs equipment at the very edge of the PPVPN cloud. Specifically, we consider two cases: Case A: Non-hierarchical VPNÆs: QoS is end-to-end in the ppvpn cloud, from VPN Endpoint to VPN Endpoint in the PEÆs; and Case B: Hierarchical VPNÆs: QoS is from VPN Endpoint in the MTU to VPN Endpoint in the MTU. In this document, the "service model" is defined as the service specification and the QoS treatment to be applied to the packets as traffic is carried between two VPN endpoints. The VPN endpoint is also the Service Access Point (SAP). The services are specified through Service Level Specifications (SLS) that are described in Section 4.2. Obviously, the QoS specifications depend upon the capabilities of the network that is carrying the service. For example, if the network implements a Diffserv solution, a PHB can be specified for the service. The SLS specification should correspondingly make sure that the desired QoS treatment can be supported in the network. 3.1.2. Reference models Case A: Non-hierarchical VPNÆs Case A.1: Point-to-Point This is straightforward. QoS is between the two VPN endpoints, it is a QoS pipe with specified characteristics. No other cases are possible. This is illustrated in the following figure. ----------- ---------- Expires September 2003 11 PPVPN QoS framework March 2003 | | | | ----- | | ----- -- |VPN | | | |VPN | -- |CE|---|End | PE-1 |-----------------| PE-2 |End |---|CE| -- |point| | | |Point| -- ----- | | ----- | | | | ----------- ---------- Case A.2: Multipoint This is the most general scenario, illustrated in the following figure: ----------- ----------- ------ | | | -- |VPN | | | | |CE|---|Endpnt| | | ------ -- ------ | | |VPN | -- | PE-1 |-----------------| PE-2 |Endpnt|---|CE| ------ | | ------ -- -- |VPN | | | | |CE|---|Endpnt| | | | -- ------ | ----------- ----------- / \ / \ / \ / ------------------- | | | | | PE-3 | | | | ------ ------ | | |VPN | |VPN | | -|Endpnt|-|Endpnt|- ------ ------ | | -- -- |CE| |CE| -- -- Note that the following scenario is just a special case of the previous one (i.e., as far as QoS is concerned, it is multipoint even if there is only one VC in the network; in fact, the customer does not care if his/her VPN endpoints are in the same PE or not; he/she just wants QoS): ----------- ----------- ------ | | ------ -- |VPN | | | |VPN | -- Expires September 2003 12 PPVPN QoS framework March 2003 |CE|---|Endpnt| | | |Endpnt|---|CE| -- ------ | | ------ -- | PE-1 |-----------------| PE-2 | ------ | | ------ -- |VPN | | | |VPN | -- |CE|---|Endpnt| | | |Endpnt|---|CE| -- ------ | | ------ -- ----------- ----------- In the multipoint scenario, several VPN topologies are relevant. They include: - Mesh of tunnels between nodes (any-to-any) - Hub-n-Spoke - Other topologies (sink trees, etc.) With multipoint, we need to make the distinction between pipe or hose models. Three cases are possible. A. PIPE MODEL In the pipe model, the QoS specification is per pair of VPN endpoints. A special case is one where the specifications between all pairs of VPN endpoints are homogeneous, in which case the QoS specifications are given between any pair of endpoints. In the non-homogeneous case, the specifications may differ depending on the pair of VPN endpoints, and possibly even differ for each direction of communication, depending on whether the QoS specifications are symmetric or not. In these more complex scenarios, the provider will likely offer a small number of templates of QoS specifications, to help in the definition of QoS. The pipe model requires that for each new customer site on a PE, a pair be configured between it and every other remote site for that customer on every PE. If there are multiple sites for a given customer connecting to the same PE, each of these would be considered to have a separate pipe for the purposes of QoS. Thus, if the number of sites for a customer is N, for every new site provisioned at a PE, up to 2N pairs of will need to be configured. Clearly, this constitutes considerable provisioning overhead, and means of reducing this overhead are needed. One possibility to simplify this provisioning nightmare may be for the operator to initially provision the same size pipes between all customer sites and later modify the traffic profile for customer ôpipesö depending on observed traffic patterns between specific customer sites. Another relevant possible mechanism to cope with these issues is single-side signaling. Expires September 2003 13 PPVPN QoS framework March 2003 The issue of symmetry has a direct impact on the required signaling. If the QoS specifications are symmetrical, then when a new site is added to the VPN, and the mesh of LSPs are established, the remote points can be easily provisioned using an automated signaling mechanism. The operator can download a set of filter and traffic profile between this new VPN site and all its peers û this downloading is only done once at the new site. Signaling then propagates this information to the remote sites. The remote ends then install the filter and traffic profile to be applied against traffic originating from the remote end to the PE that sent it the profiles. If the QoS specifications are not symmetrical, automated signaling may not be as straightforward. Either the operator has to download a filter and traffic profile at both ends of the VC LSP or it downloads the two different filter/traffic profiles at a single end and then automatically signals to the remote VPN site. This can be handled using single-side signaling. The pipe model is the only model that guarantees strict control of what can be provided on a per pair basis. Fairness can be provided in this model. To guarantee QoS in this model, there is the need to identify the VPN endpoints, in order to distinguish traffic destined to different VPN endpoints residing in the same PE. See Section 3.2 for a discussion of the issue of identification of endpoints. B. HOSE MODEL The hose model avoids these issues. Pure hose just specifies the QoS characteristics at the demarcation point(s) between CE and PE. The hose model is appealing because of its conceptual simplicity. Unfortunately, since the QoS specifications do not explicitly include a description of the traffic destinations, the QoS support of the hose model is rather problematic in real networks. These difficulties are quite evident in a hose-based VPN service. The hose model does not need the identification of VPN endpoints. At Layer 2, what specified in [Lasserre] is adequate to support QoS in this model. A key simplification in terms of provisioning with the hose model is the fact that, when adding a new customer site, it is not necessary to download traffic filters and profiles for traffic destined to every single other customer site. Instead, if this is the first site for this customer on the PE, a filter and traffic profile can be downloaded. Later, as new sites are Expires September 2003 14 PPVPN QoS framework March 2003 added for the same customer on this PE, the traffic profile may be adjusted to account for the changed traffic that will enter the network for this customer. In the hose model, there is no need to transmit any information to customer sites on remote PE for QoS purposes. The central issue with the hose model is the detailed allocation of resources for the VPNs inside the network. With the hose model, precise provisioning of hard QoS guarantees is very difficult or even impractical to achieve, since it would require excessive resources to be prepared in the network to handle all possible traffic distributions. Statistical arguments can be made to reduce the allocated resources, at the cost of introducing a corresponding statistical component in the QoS guarantees. Only preliminary work is available on such statistical arguments, and no well-established framework for resource allocation exists. A more practical and very attractive approach with the hose model is a ôrelativeö (rather than absolute) notion of QoS. In this view, different VPNs are assigned weights, and their QoS treatment is relative to the individual weights. The QoS guarantees are not ôhard.ö However, using this approach in conjunction with proper Call Admission Control and traffic conditioning procedures, ôrelatively preciseö QoS guarantees can be provided. This approach fits very well with the Diff- Serv QoS framework. C. MIXED HOSE & PIPE MODEL In this case, for a given VPN, there is a pipe between PEÆs, but multiple VPN endpoints residing in the same PE share the pipe(s). Identification of VPN endpoints is not needed. At Layer 2, what specified in [Lasserre] is adequate to support QoS in this model. The hose model is within the VPN. A given VPN has pipes allocated between PEÆs and then members of the VPN share the pipes. This model is quite meaningful, since it looks very much like what would happen in a LAN. Indeed, in this model, if somebody is not getting the desired QoS, it is because of misbehavior of some other members in that VPN, rather than because of misbehavior of somebody outside the VPN. To the customer, pure hose model and mixed hose and pipe model must look the same, since the customer is unaware of whether endpoints share the same PE or not. The provider may use the mixed hose and pipe model to provide QoS. In this case, the provider needs to properly aggregate resources for the shared pipes in order to provide QoS. Again, an open issue in case of Expires September 2003 15 PPVPN QoS framework March 2003 hose specifications, is the sizing of the pipes to meet those specifications. Pipe and hose models have to be able to coexist in the same network, since they appeal to different types of customers or different applications, and can correspond to services with different price points. Case B: Hierarchical VPNÆs Scenario: ------ ------ ------ | | ------ -- |VPN | | ------- ------- | |VPN | -- |CE|--|Endpnt| | | | | | | |Endpnt|--|CE| -- ------ | - | | - | ------ -- | MTU-1|----| |PE-rs1|----|PE-rs2| |----|MTU-2 | ------ | - | | - | ------ -- |VPN | | | | | | | |VPN | -- |CE|--|Endpnt| | ------- ------- | |Endpnt|--|CE| -- ------ | | ------ -- ------ ------- | | | | | ---------- | | | | | ---------------| |-------------- | PE-rs3 | | | | -- | ---| |--- -- | | ------------------- | | | MTU-3 | | ------ ------ | | |VPN | |VPN | | -|Endpnt|-|Endpnt|- ------ ------ | | -- -- |CE| |CE| -- -- A primary motivation for this scenario is the desire of not identifying the endpoints in the MTUÆs (for scalability). As a consequence, it is most naturally suited for the hose model. Expires September 2003 16 PPVPN QoS framework March 2003 What specified in [Lasserre] is adequate to support QoS in this case. Load balancing is not possible in this scenario, since the fact that the same CE is connected to two different MTUÆs is totally hidden in the network. Although the hose model is the most natural fit in a hierarchical scenario, different customers have different requirements and one model cannot not fit all. Again, hose and pipe model should coexist in this configuration. In the case of pipe model, the VPN endpoints have to be identified, which would decrease scalability. Similarly to the case of pure pipe model, additional means have to be added to what specified in [Lasserre] for the identification of VPN endpoints. In the hierarchical scenario, there is an intermediate possibility of identifying the endpoints in the PEÆs and not identifying those in the MTUÆs, ending up with hose model from MTUÆs to PEÆs, and pipes connecting endpoints in the PEÆs. This hybrid scenario does not seem to attract much interest, and is not further considered in this document. 3.2.VPN Endpoints (Service Access Point) Identification The identification of VPN endpoints is most relevant in Layer 2 VPNÆs. The identification of VPN endpoints has implications on inner label distribution (for example, unsolicited on demand does not work).Identification of endpoints reduces scalability (larger number of VC labels, more signaling). The need of identification of VPN endpoints constitutes a major difference in performing resource allocation for Layer 2 vs. Layer 3 PPVPNs. In Layer 3 VPNs, the demarcation point between the customer and the provider-network can be identified by a private or public IP address. As such, existing IP resource reservation models/signaling protocols, i.e. Diff-Serv, Int-Serv, MP-BGP, RSVP RSVP-TE, LDP, CR- LDP can be readily applied to perform resource reservation between the customer/provider demarcation points. This is not the case for Layer 2 VPNs, where the demarcation point between the customer and the provider network is a Layer 2 endpoint which usually does not have an associated private/public IP address. As such, most existing resource reservation schemes for Layer 2 VPNs have been focused on the resource reservation for the ingress-PE-to- egress-PE outer-tunnel(s), while the signaling of the resource reservation for inner VCs, particularly inside and across the ingress and egress PEs has not been addressed. Expires September 2003 17 PPVPN QoS framework March 2003 This is also due to the fact that most Layer 2 VPN proposals, e.g. [Martini-TRANS], [Lasserre], have so far limited themselves to cover only Best-Effort Layer 2 VPNs. It is therefore necessary to (1) establish a widely accepted L2VPN endpoint identifier(s), and (2) provide extensions in existing L3 resource reservation models/ signaling protocols to incorporate such L2VPN endpoint identifiers. For instance, in the case of MPLS-based L2VPNs, Layer 2 VPN endpoints are associated with new types of MPLS Forwarding Equivalent Classes, e.g. the VC-FEC proposed by [Martini-TRANS] and the MTU-FEC proposed by the [Lasserre] and [Shah] approaches. While such new types of FECs have been defined by various drafts, the extensions to incorporate them into the related signaling protocols have been sporadic at best, e.g. extensions of RSVP-TE to support VC-FEC or MTU-FEC have received very little, if any, attention so far. Such extensions are essential to allow us to unify the resource reservation model/ mechanisms for both Layer 2 and Layer 3 PPVPNs. 3.3.Hard and Soft QoS Guarantees, Aggregated Models and Non-Aggregated Models In recent years, QoS has received enormous attention in the technical community. Several QoS frameworks have been proposed, studied in depth, and standardized by various working groups. Despite the large body of work on QoS, however, considerable confusion still exists on what can be achieved by the different frameworks, and sometimes even accepted terminology bears different meaning to different people. In this section, we define the available models for QoS provisioning. For the purposes of this document, we distinguish two classes of QoS guarantees: A. QoS guarantees that are precisely provided to individual end users, regardless of traffic conditions; we refer to this type of guarantees as ôhard guarantees.ö B. QoS guarantees that are provided to aggregates, or classes of users; these guarantees translate to guarantees to the individual users that are not as precise as the hard guarantee; we refer to this type of guarantees as ôsoft guarantees.ö Another way to look at this issue is whether the QoS guarantees are provided to the individual users or only to traffic aggregates. In the latter case, the guarantees of the traffic aggregates translate into guarantees for the individual users whose traffic forms a Expires September 2003 18 PPVPN QoS framework March 2003 specific aggregate. Typically, the guarantees provided to the individual users in this case are statistical or relative in nature. These QoS models correspond to different scheduling frameworks that are used in the network to provide the desired guarantees. Three scheduling frameworks are well established. 1. Per flow scheduling and conditioning at the edges of the network, with limited scheduling in the cloud, to achieve guarantees to aggregates, and relative or statistical guarantees to the users. This is probably the most popular approach, and is defined in the Diff-Serv framework. Diff-serv aggregates users into a relatively small number of Per-Hop Behaviors (PHBs). This approach is relatively simple, and works quite satisfactorily for many applications, when used in conjunction with proper admission control. This type of framework is also used to support priority schemes such as 802.1p, which defines a class-based framework that comprises 8 different classes of services. 2. Per flow scheduling both at the edges and in the core of the network, to achieve hard guarantees to individual users in terms of bandwidth, delay, and jitter. This approach is exemplified in the Int-Serv framework. Because of its complexity and scarce scalability, this approach is not widely deployed. 3. Per flow scheduling and conditioning at the edges, elaborate scheduling on aggregates in the cloud to achieve hard guarantees to individual users. Motivated primarily by the desire of supporting real-time applications, a number of proprietary frameworks are appearing to improve on what the performance of diff-serv, without adding significant complexity. These approaches target the scheduling in the core nodes, and are typically compatible with a diff-serv vision in terms of signaling and communication between nodes. However, since the emphasis of these approaches is on hard guarantees to the individual users, the way resources may be allocated and managed in the network may be different from a pure diff-serv approaches. It is important to reiterate that in these frameworks, hard guarantees can be achieved even if users are aggregated within the network, through proper scheduling and queuing throughout the network. Frameworks providing guarantees per aggregate need to coexist with those providing guarantees to individual customers. Similarly, in the case of VPNs, individual connections need to coexist with tunneled connections. This is not so difficult, since individual connections are a special case of tunneled connections. Different QoS frameworks should be able to be mapped into each other in order to guarantee inter-working. For example, Layer 2 VPNs supporting 802.1p may be provided on an MPLS network implementing Expires September 2003 19 PPVPN QoS framework March 2003 diff-serv. Accordingly, a mapping of 802.1p priorities into diff- serv classes needs to be used. 3.4.QoS OF the VPN, QoS WITHIN the VPN A crucial issue with PPVPNs is the fact that, in principle, they introduce an additional level of complexity in the provision of QoS. A given VPN may correspond to a single level of QoS. Alternatively, a customer may want to send different types of traffic within the same VPN, and the network needs to support each type, while maintaining the notion of VPN. Even if the customer sees the VPN with multiple levels of QoS as a single VPN, the provider may accomplish it by establishing a single VPN, or by having multiple VPNÆs and grouping them. Three solutions are possible. 1. Provider establishes one VPN, there is a single tunnel between endpoints, but there is scheduling and conditioning of the traffic components at the edges. This is adequate in many, but not all cases. 2. Provider establishes one VPN, but there are multiple tunnels between endpoints, one per desired level of QoS. 3. Provider establishes one VPN per desired level of QoS, and then groups the VPNÆs so they look like a single one to the customer. A first issue that arises with grouping is the broadcast domain, it needs to broadcast on only one of the levels. Also learning is an issue, it needs to learn across the VPNÆs in the same group. In more general terms, the establishment of multiple levels of QoS within a VPN requires additional signaling and resource allocation mechanisms. The available work in this direction, if any, is very preliminary, and certainly not sufficient to support a flexible model of QoS provisioning within a VPN. 4. QoS Guarantees, Service Level Agreements, and Management in PPVPNs 4.1.QoS Guarantees This section summarizes the various QoS guarantees that can be provided in a VPN by defining SLSs. * Bandwidth guarantees The services can be specified with and without (best-effort) bandwidth guarantees. When a service model has associated bandwidth parameters, the service provider should make sure that Expires September 2003 20 PPVPN QoS framework March 2003 enough bandwidth is provisioned within the network (using any model, point-to-point or hub-and-spoke). Generally the guarantees are specified and policed against certain traffic models. For example, Diff-Serv allows the traffic specification as a Single rate or Two rate traffic parameters, that have associated parameters. o Based on Diff-Serv models (srTCM, trTCM) * Single Rate: CIR, CBS, EBS This model guarantees an average rate of CIR to the service with certain burst tolerances specified by CBS and EBS. The traffic policer/marker based on single rate (srTCM) can mark the traffic as green, yellow or red depending on the traffic conformance to the specification. * Two Rate: CIR, CBS, PIR, PBS This model allows a burst tolerance to both average (CIR) and peak (PIR) of the traffic. The two rate marker/policer (trTCM) can mark the traffic as green, yellow and red depending on the traffic conformance to the specification. o Examples of ranges and granularities o 1Mb/s min service granularity * Delay guarantees Delay is a measure of maximum packet transfer delay between the ingress and egress SAPs. Delay can be specified as worst case bound or as a quantile. o Types of delay guarantees * Based on Diff-serv EF model for High-priority traffic * Based on Average RED thresholds * Based on number of hops o Examples of ranges and granularities * Jitter guarantees Jitter is a measure of packet delay variation encountered by packets when they are transferred between ingress and egress SAPs. Jitter can be specified as a worst-case bound or as a quantile. o Examples * Other guarantees * Packet Loss Packet loss is specified as a probability. It is defined as the ratio of lost packets between ingress and egress SAP to the total packets received at the ingress SAP. Expires September 2003 21 PPVPN QoS framework March 2003 o Packet Loss per-class, per-VPN o Aggregate Packet Loss measure for Multipoint * Fairness Fairness guarantees address how certain resources are accessed by different entities. Ideally, all entities belonging to a certain class or with similar service agreements should be treated ôfairlyö in accessing available resources. Many issues related to fairness remain open, and the whole notion of fairness remains hard to quantify. Indeed, a single network-wide notion of fairness is probably not achievable. * Administrative Examples of this type of guarantees include the provision of VPNs that are routed only through certain geographical areas (or avoid certain areas), or are routed through certain facilities, or only through ports with specific characteristics (for example, only high-speed ports). The provision of this type of guarantees typically requires explicit routing capabilities. * Availability, Reporting, and Restoration o 100% (1+1), Restoration < t * Best effort traffic in PPVPN o Multiple priorities 4.2.VPN Service Level Specification (SLS) 4.2.1. SLS structure A generic Service Level Specification template (SLS) is proposed in [SLS]. This SLS template is assumed to be the elementary building block of the SLS that is offered by the Service Provider to his customer. A VPN SLS is assumed to be a combination of several instances of the SLS template. 4.2.2. Layer 3 SLS template We briefly summarize the different parts of the SLS template that were defined in [SLS]. Scope The scope of an SLS uniquely identifies the geographical/topological region over which the QoS is to be enforced by indicating the boundaries of that region. Although an SLS is associated with unidirectional traffic flows, this does not exclude the provisioning by providers of bidirectional transport service contracts, by Expires September 2003 22 PPVPN QoS framework March 2003 combining one or more SLSs. Valid combinations in the context of VPN contracts are Pipe (1,1), Hose (1,N) and Funnel (N,1). Flow Description The flow description of an SLS indicates for which IP packets the QoS guarantees for that specific service offering is to be enforced. An SLS contains one (and only one) flow description, which may be specified by providing one or more of the following attributes: - Differentiated Services information - Source information - Destination information - Application information In case of MF-classification all attributes may be specified, including the DSCP field. The following are some of the Traffic Filter fields to be considered for transmission to the remote PE: source IP, destination IP, DSCP, 802.1Q bits, destination port, source port, destination port mask, source port mask, protocol. These fields should be based on the capabilities of classification mechanisms that will be present on PEs. Traffic Envelope and Traffic Conformance The traffic envelope describes the traffic (conformance) characteristics of the IP packet stream identified by the flow description. The traffic envelope is a set of Traffic Conformance Parameters, describing what the packet stream should look like to get the guarantees indicated by the performance parameters (defined in Section 4.1). The Traffic Profile fields should interoperate with various pre- defined Diffserv policers such as srTCM, trTCM and tswTCM. Thus, this would include: cir, pir etc. Excess Treatment Excess traffic may be dropped, shaped and/or remarked. The SLS must specify the appropriate action by the Excess Treatment attribute. If Excess Treatment is not indicated, then excess traffic is dropped. Performance Guarantees The performance parameters describe the service guarantees the network offers to the customer for the packet stream described by the flow description and over the geographical/topological extent given by the scope. There are four performance parameters: o delay, time interval, optional quantile o jitter, time interval, optional quantile o packet loss, time interval Expires September 2003 23 PPVPN QoS framework March 2003 o throughput, time interval Delay, jitter and packet loss guarantees are for the in-profile traffic in case of binary conformance testing. If none of the SLS performance parameters are quantified, then the performance parameters "delay" and "packet loss" may be "qualified". Possible qualitative values (for delay and/or loss): high, medium, low. Measurement and reporting The monitoring and reporting section of the SLS specifies when and how QoS performance needs to be monitored and reported. This section may include: - Reporting address: Address of the entity to which performance reports need to be sent - For each performance parameter, the following may be specified: o measurement interval o reporting type o notification threshold - For the availability parameters, the SLS may contain: o total number of outages o total outage time o maximum allowed mean downtime per year (MDT) o maximum allowed time to repair (TTR) Availability The availability of the offered network service needs to be related to the protection and restoration options that are available in the network. We propose the following availability categories: - unprotected: no protection guaranteed - restored: service interruption <10s - protected: service interruption <1s - fast protected: service interruption <100ms The availability of the VPN also depends on the specific VPN model that is used. It needs to be clearly indicated if the availability guarantees apply to the complete VPN, or only to the hub or spoke in a hub-and-spoke VPN. Service schedule The service schedule indicates the start time and end time of the service, i.e. when is the service available. This might be expressed as collection of the following parameters: - Time of the day range - Day of the week range Expires September 2003 24 PPVPN QoS framework March 2003 - Month of the year range 4.2.3. Layer 2 SLS template The Layer 2 SLS template is TBC. If the pipe model is provided, this needs to include the unique identification of the pipe endpoint. 4.2.4. SLS example The picture below gives an example of a VPN with three attached sites A, B and C, each of which can send or receive a certain amount of traffic into the VPN. +---+ 50 Mb/s +-------------------+ 20 Mb/s +---+ + A +<-------->| |<-------->+ B + +---+ | | +---+ | | +-------------------+ ^ | 30 Mb/s v +---+ + C + +---+ In the case of a full mesh VPN model, the SLS contract would consist of three hose SLS instances: - a hose with root A and capacity 50 Mb/s, with restrictions on the egress points B (20 Mb/s) and C (30 Mb/s) - a hose with root B and capacity 20 Mb/s - a hose with root C and capacity 30 Mb/s, with a restriction on the egress point B (20 Mb/s) In the case of a hub-and-spoke model, the latter two hose SLSs would be replaced by pipe SLSs with capacity 20 Mb/s and 30 Mb/s respectively. 4.3.Management of QoS VPN In order to facilitate network management of VPNs from a QoS perspective consideration needs to be given to the protocol extensions required. MIB extensions are needed. A sensible approach, rather than developing entirely new MIBs is to expand existing MIBs to include parts of the Diffserv MIB and other PPVPN MIBs that may emerge. 5. Other QoS issues in PPVPNÆs Expires September 2003 25 PPVPN QoS framework March 2003 5.1.Learning issues As mentioned in Section 3.4 above, multiple levels of QoS within a VPN may require to support learning across groups of VPNs. 5.2.Marking issues Currently this section focuses only on MPLS tunnels. If both VPN tunnels and end-point tunnels use either RSVP-TE or CR-LDP and use E-LSPs for tunnel setup, the EXP bits must be appropriately marked in both labels. If the end-point tunnels are setup using Martini type encapsulation, then the inner label doesn't carry any EXP bits but the outer label must be appropriately marked using any L2 classification rules, priority bits in 802.1p/q. Generally most L2 connectivity requires sequenced packet delivery (no out of order packets) and preserved QoS marking. While QoS marking can be easily preserved, L2 packet sequence cannot be guaranteed when transported using MPLS tunnels due to different QoS treatment needs. For now, it is assumed that there are mechanisms (outside the scope of this document) in the network to preserve L2 packet order. Various such tunneling models can co-exist in the context of PPVPN and it is assumed that the labels are appropriately marked at the edges to get the QoS treatment needed in the network. 5.3.Garbage collection, resource de-allocation/recovery (To be added in future versions of this document.) 5.4. Specific QoS issues This section lists QoS issues that only pertain to a specific VPN solution. 5.4.1. QoS in RFC 2547 VPNÆs (To be added in future versions of this document.) 5.4.2. QoS in VR VPNÆs (To be added in future versions of this document.) 5.4.3. QoS in L2 LDP-based VPNÆs (To be added in future versions of this document.) 5.4.4. QoS in L2 BGP-based VPNÆs (To be added in future versions of this document.) Expires September 2003 26 PPVPN QoS framework March 2003 5.4.5. QoS in IPSec VPNÆs (To be added in future versions of this document.) 5.4.6. QoS in L2TP VPNÆs (To be added in future versions of this document.) 5.4.7. QoS in GRE VPnÆs (To be added in future versions of this document.) 6. Traffic Engineering, QoS Routing, Protection/Restoration and their Relation with PPVPN QoS 6.1.Traffic Engineering and QoS Routing Internet traffic engineering is defined as that aspect of Internet network engineering dealing with the issue of performance evaluation and performance optimization of operational IP networks [TE]. One of the most distinctive functions performed by Internet traffic engineering is the control and optimization of the routing function, to steer traffic through the network in the most effective way. For intra-domain optimization, extensions for traffic engineering have been defined for the Interior Gateway Protocol (IGP), e.g. OSPF [OSPF-TE] or IS-IS [ISIS-TE]. The information disseminated by these IGP extensions can be used to perform a constraint-based routing computation on the network. When MPLS is used to set up explicit routes in the network, the signaling can be done with RSVP-TE [RSVP- TE] or CR-LDP [CR-LDP]. The IGP and signaling extensions and procedures that are needed for supporting Diff-Serv traffic engineering requirements defined in [DSTE-REQ] (beyond those already specified in [OSPF-TE][ISIS- TE][RSVP-TE][CR-LDP]) in environments relying on distributed Constraint Based Routing are detailed in [DS-TE]. Alternatively, traffic engineering can be performed in a centralized way by means of a global optimization algorithm. In that case, the availability of global network and traffic information and additional computational resources may lead to better optimization, at the cost of reduced responsiveness. With PPVPNs, traffic engineering may be performed at different levels: 1. per VPN, per endpoint pair (requires identification of endpoints) 2. per VPN, per aggregate 3. per class Expires September 2003 27 PPVPN QoS framework March 2003 6.2.Protection and Restoration When protection and restoration schemes are used, QoS resources may have to be allocated both in the primary and the protection paths. Typically, in order to best utilize available resources, the allocation algorithms used on the protection paths differ from those used on the primary paths. For example, sharing of resources in the protection paths may be desirable to avoid reserving excessive network capacity to protection. 6.2.1. Review of protection schemes of interest Well known protection schemes may be applied to the tunnels forming the VPN. They include: MPLS Path protection It calculates (possibly in a centralized way) for each LSP a primary and a protection path. When a failure is detected, the signaling has to go to the ingress router of the LSP which in its turn re-directs traffic to the backup LSP. This signaling can take a while, especially in large networks. But on the other hand, the protection path is optimal. This protection mechanism offers 1:1 protection. Load balancing For each LSP, as many disjoint paths (n) as possible are determined in advance and the traffic is equally split over (n-1) paths. The nth path serves as a backup path in case of network failure. Again, the signaling has to go all the way to the ingress router and the routing of the paths is optimal. This mechanism offers 1:n protection. Fast local protection - Detour: provides local protection. The way it protects LSPs is by providing a detour from each node to the destination of the LSP, excluding the next node. This technique offers a fast rerouting of the LSPs, but it is clear that it needs a lot of LSPs to be provided. As such it may be desirable to apply merging on the detour LSPs. Note, however, that the alternative (sender-template specific setup) is more widely applicable. - Bypass: for each link-node-link set (or link) in the network a bypass LSP is pre-provisioned. At the moment one of these sets (links) fails, the traffic of all LSPs traversing this segment is switched to the bypass tunnel. This protection mechanism offers a fast rerouting of the LSPs in case of a network failure, but on the other hand a lot of LSPs are needed to offer this protection. 6.2.2. Network-specific aspects 1. Topological density Network density has an important impact on the efficiency and speed with which protection techniques can be applied. With increasing Expires September 2003 28 PPVPN QoS framework March 2003 network density, the amount of alternative paths with (nearly) equal length increases. This is favorable for all protection approaches, but especially for fast local protection techniques relying on the availability of alternative paths close to the point of failure and for low-granularity protection techniques which allows these to achieve maximum load-balancing of protection capacity. 2. Traffic load With increased load, resource sharing on backup paths becomes more important. This is facilitated with a centralized approach. In case of the single node/link failure model, it can also be done efficiently with Bypass protection. 6.2.3. PPVPN-specific aspects 1. Efficiency and computational complexity Protection paths can be calculated in a distributed way in the network elements or in a centralized entity. Centralized computation allows taking into account all traffic simultaneously as well as allowing the use of sophisticated optimization algorithm for maximum resource sharing. This resource efficiency comes at the price of increased computational efficiency. We can expect this trade-off to be related to the specific VPN model that is chosen. If a mesh of LSPs is shared among all VPNs we can expect this mesh to be fairly stable and pre-provisioned, thus mandating the extra computational effort in order to get the resource benefit. For VPN-specific LSPs, we might assume faster response times and shorter lifetimes, hence leaning perhaps more towards a distributed, less optimized approach. 2. LSP size Large size LSPs are more difficult to protect efficiently unless load balancing over several protection paths is available. This is the case for end-to-end path protection and BYPASS fast local protection techniques. It is less obvious for DETOUR fast local protection which relies more on the per-LSP granularity to achieve load-balancing. If a mesh of LSPs is shared among all VPNs, these LSPs can be expected to be fairly large-size, thus suggesting the use of a protection method that allows efficient load balancing over multiple protection paths. 6.2.4. Required signaling extensions û building blocks (To be added in future versions of this document.) o LDP-based o RSVP-TE-based o BGP-based Expires September 2003 29 PPVPN QoS framework March 2003 6.2.5. Traffic engineering considerations in protection. (To be added in future versions of this document.) o Turn on automatic rerouting of LSPs? o Multi-area TE and protection o Resource sharing between backup LSPs 7. QoS Signaling in PPVPNs In general, PPVPNs can be established using various PE-to-PE tunneling technologies including MPLS, L2TP, IP-in-IP, IPsec as well as GRE. Given our objective of supporting PPVPN services with QoS guarantees, we will focus our discussions on MPLS tunneling alone due to the existence of scalable MPLS-based solutions in resource reservation, QoS signaling and traffic engineering. We only mention in passing that the subject of QoS support has not been treated in any detail in the recent Internet drafts [PPVPN-L2TP, PE-IPSEC, PE- GRE] regarding the other PE-to-PE tunneling technologies. 7.1.Resource Allocation Different Legs in the resource reservation path of a PPVPN o Starting point: Customer-facing port of the ingress PE --- this is also the starting point(s) of the per VPN inner VC(s) o Inside the ingress PE, resource are reserved at the per inner VC level of granularity. The inner VC level resource requirements can either be provided to the ingress PE via configuration or they may be extracted from an UNI-signaling protocol, if any, which is initiated from the customer and terminated at the ingress PE. Once extracted from the UNI signaling protocol, such inner-VC level resource requirements should preferably be signaled to the egress PE. o Network facing port of the ingress PE û this is also the starting point of the ôingress PE to egress PEö outer-tunnel ; Multiple per VPN inner VCs having the same pair of ingress/egress PEs are aggregated / multiplexed into the same outer-tunnel. [Also need to consider the case where multiple parallel outer-tunnels are used to connect between the same ingress/egress PE pair for fault- tolerant, traffic-engineering and load-balancing purpose.] o Path through the core network --- resource requirements of per-VPN VCs are aggregated and reserved at the outer-tunnel level such that the inner VCs are not visible in the core network. The step-size of resource allocation/deallocation for an outer tunnel should be bigger than the requirement of an individual VC to introduce hysteresis effect and reduce the frequency of adjusting outer-tunnel resource allocation level as inner VCs are setup and torn down. This Expires September 2003 30 PPVPN QoS framework March 2003 is essential for reducing signaling load and thus increasing signaling scalability in the core network. Schemes such as those defined in RFC3075 and the Hierarchical LSP- draft can be used for signaling/ maintaining aggregation of inner VCs into outer tunnels. In particular, if RSVP-TE is used for signaling the changes of outer-tunnel resource-reservation, RSVP- TEÆs requirement of make-before-break would almost mean the setup of a replacement outer-tunnel upon changes in resource reservation level. Moreover, the make-before-break mechanism in most cases also implies changes of outer-tunnel label values for a large number of inner VCs at the ingress PE û some form of indirection is desirable to speedup/reduce the overhead required to update a large number of outer-tunnel /inner-VC label-pair at the PE. o Network facing port of the egress PE û this is the termination point of the outer tunnel (although the actual outer-tunnel label may have been removed by the penultimate hop already). The inner VCs carried by the outer tunnel are demultiplexed from the outer tunnel and become visible again. In the case where penultimate-hop-popping is performed for the outer tunnel label, additional mechanism is needed to create/maintain/signal the binding between an inner VC and its outer tunnel in the egress PE. The creation of this binding is missing in a lot of existing cases, as different mechanisms/ signaling protocols are used to setup the outer-tunnel and inner VCs separately, e.g. in MPLS-based L2VPN, outer-tunnels are setup using RSVP-TE while the inner VCs are setup using an extended version of LDP based on the Martini draft. The binding are particularly useful during outer-tunnel re-routing/ re- establishment where the ingress port/ line-card of the egress PE for the outer-tunnel may have changed as a result of re-routing and the resource allocation for the inner VCs carried by the re-routed outer tunnel would need to be moved from the original ingress port to the new one. Unless the binding between the outer-tunnel and its inner VCs are kept by the egress PE, such movement would require operator intervention and thus ruin the chance of end-to-end fast re- routing/restoration of PPVPN service. o Inside the egress PE, resource are reserved at the per inner VC level of granularity. o Each inner VC is then terminated at its customer-facing port of the egress PE. 7.2.Resource Reservation tasks for PPVPNs In order to provide QoS guarantees in a PPVPN, resource reservation has to be performed (a) within the ingress and egress PEs hosting endpoints and (b) for the PE-to-PE outer tunnels as they traverse the provider network. While resource reservation for the outer Expires September 2003 31 PPVPN QoS framework March 2003 tunnels are usually done at an aggregated level where individual VC (aka pseudo-wire or emulated virtual circuit) requirements become invisible, resource allocation within the ingress/egress PE is usually done at a finer granularity. For instance, for point-to- point L2 pseudo-wire service, this is done at a per VC level granularity; for RFC2547 L3VPNs, bandwidth reservation can be performed on a per L3VPN endpoint basis; for VPLS, this can be done on a per L2VPN endpoint basis. [This actually corresponds to allocate bandwidth to each logical port of a Virtual Forwarding Switch (VFS) within a PE hosting one or more endpoints of the corresponding L2VPN.] 7.3."Partial" QoS Signaling Approaches, Existing Proposals Most of the existing PPVPN architectures, including RFC2547 L3VPNs, Martini-based L2VPNs [Martini-ENC, Martini-TRANS, Lasserre], as well as BGP/MPLS-based L2VPN proposals [Rosen, Kompella], have limited their use of QoS/ resource-reservation signaling protocol, typically RSVP-TE or CR-LDP, to the setup of PE-to-PE tunnels only. While signaling protocols such as LDP and BGP (with appropriate PPVPN extensions) are used for setting up basic connectivity within the VPN (by distributing the corresponding VC labels together with VPN endpoint reachability information), resource allocations/ reservations within the ingress/egress PEs are left for manual provisioning instead. In fact, the current PPVPN extensions for LDP as specified in [Martini-TRANS, Single-sided] and those for BGP as specified in [MP-BGP] cannot even associate any QoS attribute with the MPLS-labels or VPN reachability/ endpoint information that they distribute. By restricting the use of resource reservation signaling to the setting up of PE-to-PE outer tunnels, additional PPVPN- specific extensions of the resource reservation protocols, i.e. RSVP-TE or CR-LDP, can be avoided. This is because the inner VCs as well as their corresponding VPN endpoints are invisible to the resource reservation protocol during the setup of the outer tunnel. As far as RSVP-TE or CR-LDP is concerned, its task is to setup an LSP with QoS requirement between the loopback IP addresses of the ingress/ egress PEs without involving any PPVPN specifics. This results in the dependence on manual provisioning for VC-level resource reservations within the PEs, which eventually prevents the support of truly single-sided provisioning. The situation is particularly true under the pipe-QoS-model. Consider the case where a change of VC-level resource allocation is needed at a remote PE due to the addition/removal of a VPN endpoint in a local PE. Without resource reservation/ QoS signaling support at the VC-level between the PEs, manual provisioning/ human intervention will be required for both the local and remote PEs. Arguably, the hose-QoS model may be less affected if one accepts the assumption that, under the hose model, resource allocation for each VPN endpoint should remain unchanged as additional endpoints are added to/ removed from the same VPN. Expires September 2003 32 PPVPN QoS framework March 2003 Another drawback of the current "partial" QoS signaling approach is that outer tunnel labels and inner VC labels are usually distributed/ signaled by different protocols, e.g. using RSVP-TE for distributing outer labels while LDP or BGP is used for distributing the inner ones. The use of multiple signaling protocols for label distribution not only increases operational complexity but may also lead to difficulties in fault detection/ management. For example, the LDP connection and outer-tunnel of a VC may traverse through different paths in the provider network. The failure in one, but not both, of these paths may put the VC to an inconsistent failure state. The use of different signaling protocols for outer and inner label distribution has also make it difficult to signal an egress PE the binding information between an outer-tunnel and its component VCs. In fact, all of the existing PPVPN QoS signaling schemes described above does not provide such binding. In particular, if penultimate hop popping is used, it is impossible for the destination PE to determine the outer tunnel of a given VC. Notice that such binding information can be very useful when an outer-tunnel is automatically re-routed to a different ingress interface of the destination PE, e.g., to circumvent a failure inside the provider network, and the corresponding VC-level resource within the egress PE have to be relocated accordingly. As a beginning step in this direction, [Cai] has proposed extensions to use RSVP-TE for both VC and outer tunnel setup in L2VPNs. However, this work has gathered little attention so far. Moreover, the mandate in [Martini2] of using LDP for VC-label distribution may need to be revisited in order to allow for the use of other VC-label distribution protocols to realize better coordination between inner and outer label distribution and binding. 7.4.Towards Full QoS signaling support for PPVPNs It is clear that, in order to support truly single-sided provisioning for PPVPN with QoS guarantees, QoS/ resource reservation signaling should support not only PE-to-PE outer tunnel setup but also VC-level resource reservation within the PEs. Towards this end, existing MPLS-based resource reservation protocols, e.g., RSVP-TE and CR-LDP, would require PPVPN-specific extensions in the following areas: 7.4.1. Signaling of VPN endpoint identifiers For L2VPNs, candidate VPN endpoint identifiers/ structures to be considered include (1) the attachment endpoint identifiers (AEI) / group index proposed in [Single-sided], and (2) the VC-FEC TLV in [Martini-TRANS] with the generalized semantics of the VCID field as specified in [Lau]. Notice that, in order to support MAC address learning in MPLS-based L2VPNs (such as that required by VPLS), it is desirable for VC-setup signaling messages, i.e. Label Request/Mapping ones, to carry both the source as well as the Expires September 2003 33 PPVPN QoS framework March 2003 destination VPN endpoint identifiers associated with a VC. (This is because, in MPLS-based L2VPNs, MAC address learning is performed by first establishing the binding between the source MAC address and the VC-label carried by a packet. The ingress VPN endpoint of the packet can then be derived based on the VC-label carried by the packet using the mapping between an VC-label and its source (ingress) VPN endpoint which is created during the setup of VC.) Towards this end, it is noteworthy that the RSVP-TE extensions proposed by [Cai] as well as the LDP extensions proposed by [Martini-TRANS] and [Lasserre] do not include the ingress VPN endpoint identifier of a VC in the label distribution messages. As such, remote MAC address learning can only resolve the source PE, as well as the VPNid of a packet. In the case where a PE hosts multiple endpoints of the same L2VPN, additional local bridging based on local learning information has to be performed to resolve the destination VPN endpoint of a packet. By hiding multiple endpoints of the same VPN hosted by a PE from other remote PEs, these schemes achieve better signaling scalability at the expense of disallowing signaling support for the true pipe-QoS-model in which end-to-end QoS signaling between any pair of endpoints within a VPN would be required. For L3VPNs, a VPN-endpoint can be identified by concatenating a route-distinguisher with the private IP address associated with the endpoint, following the approach taken by RFC2547. Once the representation/construct for VPN endpoint identifiers is decided, resource reservation protocols such as RSVP-TE and/or CR-LDP should be extended to carry both the source and destination VPN endpoint identifiers associated with the VC to be setup. For instance, the LDP FEC element specified in Section 4.2 of [Single-sided] can readily be carried over to a CR-LDP extension to support the source/ destination VPN attachment endpoint identifiers in [Single-sided]. Similarly, new RSVP objects can be introduced along the line of [Cai] to extend RSVP-TE to carry both source and destination VPN endpoint identifiers of a VC. 7.4.2. Coordinating VC and Outer Tunnel QoS Signaling To increase network manageability, as well as for reasons explained above, it is desirable to use the same signaling protocol for setting up a VC and its outer tunnel. When explicit VC-level resource-reservation/allocation is required at a PE, it is also useful to signal the binding of an outer tunnel and the VCs it carries between the ingress/egress PE. Signaling scalability can also be further enhanced by using the resource aggregation mechanisms similar to those specified in [RSVP-AGG] and Section 7 of [LSP-HIE], i.e. to tunnel VC-level reservation/refreshes between the ingress and egress PE via the outer tunnel provided that PPVPN- specific extensions are incorporated in the corresponding reservation protocols. For instance, extension of signaling messages to carry the necessary VPN endpoint identifiers etc. Another area of Expires September 2003 34 PPVPN QoS framework March 2003 potential interest is to extend the outer-tunnel E-LSP signaling mechanisms, e.g. those proposed by [Ganti1, Ganti2], to the VC level so that one can provide multiple class-of-service within a PPVPN by setting up VC-level E-LSPs. 8. Future Work The following additional sections are planned for future releases of this document: A. QoS PPVPNÆs: Scenarios In this section, we explain how some specific scenarios can be implemented, given the VPN solutions described above, including: - Diffserv in PPVPNÆs - Intserv in PPVPNÆs - 802.1p in PPVPNÆs B. Automatic Provisioning of QoS in PPVPN - Auto-discovery in QoS VPN - Automatic provisioning of the outer tunnel - Automatic provisioning of the inner tunnel C. Signaling issues for PPVPNÆs with Multiple QoS Levels D. Scalability issues in QoS VPNÆs - Signaling & routing scalability with DS-TE (not PPVPN- specific) - MAC addresses - Routes (not QoS PPVPN specific) - Inner labels E. Advanced QoS Architectural Issues - Multi-homing - Multi-AS QoS Issues - QoS Brokers 9. Security Considerations (To be added in future versions of this document.) References [PPVPN-L2TP] E. Elwin et. al. "PPVPN Architecture using L2TP", draft-elwin-ppvpn-l2tp-arch-00.txt, Feb. 2002. [PE-IPSEC] E.C. Rosen et al., "Use of PE-PE IPsec in RFC2547 VPNs", draft-ietf-ppvpn-ipsec-2547-03.txt, Feb. 2003. [PE-GRE] Y. Rekhter, "Use of PE-PE GRE or IP in RFC2547 VPNs", draft-ietf-ppvpn-gre-ip-2547-01.txt, Feb. 2002. Expires September 2003 35 PPVPN QoS framework March 2003 [Single-Sided] E. C. Rosen, "LDP-Based Signaling for L2VPNs", draft-rosen-ppvpn-l2-signaling-02.txt, September 2002. [Cai] T. Cai et. al., "Signaling Virtual Circuit Label Using RSVP-TE", draft-cai-ppvpn-vc-rsvp-te-00.txt, Nov. 2001. [RSVP-AGG] F. Baker et al., "Aggregation of RSVP for IPv4 and IPv6 Reservations", RFC3175, Sept. 2001. [LSP-HIE] K. Kompella et.al, "LSP Hierachy with Generalized MPLS TE", draft-ietf-mpls-lsp-hierarchy-08.txt, Sept. 2002. [Martini-ENC] L. Martini et. al, "Encapsulation Methods for Transport of Ethernet Frames over IP and MPLS Networks", draft- ietf-pwe3-ethernet-encap-02.txt, February 2003. [Martini-TRANS] L. Martini et. al, "Transport of Layer 2 Frames over MPLS", draft-ietf-pwe3-control-protocol-01.txt, Nov. 2002. [Lasserre] M. Lasserre et. al, "Virtual Private LAN Services over MPLS", draft-lasserre-vkompella-ppvpn-vpls-03.txt, January 2003. [Lau] W. Lau et al., "Extensions for QoS support in MPLS-based Transparent LAN Services", draft-lau-ppvpn-qos-tls-mpls-00.txt, March 2002. [Kompella] K. Kompella et. al, "Layer 2 VPNs over Tunnels", draft-kompella-ppvpn-l2vpn-02.txt, June 2002. [BGP-VPN] Ould-Brahim, H., et al., ôUsing BGP as an Auto- Discovery Mechanism for Network-based VPNs,ö Work in progress [DS-L2TP] Calhoun, P., et al., ôL2TP Differentiated Services Extension,ö RFC 3308. [MP-BGP] T. Bates, et al., "Multiprotocol extensions for BGP- 4", draft-ietf-idr-rfc2858bis-02.txt, Apr. 2002. [Ganti1] S. Ganti et al, "MPLS Support of Differentiated Services using E-LSP", draft-ganti-mpls-diffserv-elsp-02.txt, June 2002. [Ganti2] S. Ganti et al., "DS-TE Requirements for Support of Multiple-COS on an E-LSP", draft-ganti-tewg-diffserv-multicos- elsreq-00.txt, Feb. 2002. [IPsec-VPN] De Clercq, J., et al., ôAn Architecture for provider provisioned CE-based VPNs using IPsecö, Work in progress Expires September 2003 36 PPVPN QoS framework March 2003 [L2TPv3] Lau, J., ôLayer Two Tunneling Protocol (Version 3)- L2TPv3", Work in progress [VPN-L2FW] Andersson, L., et al. ôL2VPN Frameworkö, Work in progress [VPN-L3FW] Callon, R., et al., ôA Framework for Layer 3 Provider Provisioned Virtual Private Networksö, Work in progress [VPN-term] Andersson, L., Madsen, T., ôPPVPN Terminologyö, Work in progress [VR-VPN] Knight, P., et al., ôNetwork based IP VPN Architecture using Virtual Routersö, Work in progress [Kompella-DTLS] K. Kompella et al., Decoupled Virtual Private LAN Services,ö Work in progress [Sajassi] A. Sajassi and H. Salama, ôVPLS Based on IP Multicast,ö Work in progress [Shah] H. Shah et al., ôSignaling between PE and L2PE/MTU for Decoupled VPLS and Hierarchical VPLS,ö Work in progress [SLS] D. Goderis et al., "Service Level Specification Semantics and Parameters," work in progress, draft-tequila-sls-02.txt, February 2002. [RFC2003] Perkins, C., "IP Encapsulation within IP," RFC 2003, October 1996. [RFC2207] Berger, L. and O'Malley, T., "RSVP Extensions for IPSEC Data Flows," RFC 2207, September 1997. [RFC2401] Kent, S. and Atkinson, R., "Security Architecture for the Internet Protocol," RFC 2401, November 1998. [RFC2402] Kent, S. and Atkinson, R., "IP Authentication Header," RFC 2402, November 1998. [RFC2406] Kent, S. and Atkinson, R., "IP Encapsulating Security Payload (ESP)," RFC 2406, November 1998. [RFC2409] Harkins, D. and Carrel, D., "The Internet Key Exchange (IKE)," RFC 2409, November 1998. [RFC2473] Conta, A. and Deering, S., "Generic Packet Tunneling in IPv6 Specification," RFC 2473, December 1998. Expires September 2003 37 PPVPN QoS framework March 2003 [rfc2547bis] Rosen, E., et al., ôBGP/MPLS VPNsö, Work in progress [RFC2746] Terzis, A. et al., "RSVP Operation Over IP Tunnels," RFC 2746, January 2000. [RFC2784] Farinacci, D. et al., "Generic Routing Encapsulation (GRE)," RFC 2784, March 2000. [RFC2890] Dommety, G., "Key and Sequence Number Extensions to GRE," RFC 2890, September 2000. [RFC2983] D. Black, ôDifferentiated Services and Tunnels.ö [RFC3031] Rosen E. et al., "Multiprotocol Label Switching Architecture," RFC 3031, January 2001. [RFC3035] Davie, B. et al., "MPLS using LDP and ATM VC Switching," RFC 3035, January 2001. [RFC 2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 [TE] D.O. Awduche, A. Chiu, A. Elwalid, I. Widjaja, and X. Xiao, "Overview and Principles of Internet Traffic Engineering," RFC 3272, informational, August 2001. [OSPF-TE] Katz, Yeung, and Kompella, ôTraffic Engineering Extensions to OSPF Version 2,ö draft-katz-yeung-ospf-traffic- 09.txt, October 2002. [ISIS-TE] Smit, Li, ôIS-IS extensions for Traffic Engineering,ö draft-ietf-isis-traffic-04.txt, June 2002. [RSVP-TE] Awduche et al, "RSVP-TE: Extensions to RSVP for LSP Tunnels", RFC 3209, December 2001. [CR-LDP] Jamoussi et al., "Constraint-Based LSP Setup using LDP", RFC 3212, January 2002. [DSTE-REQ] Le Faucheur et al., ôRequirements for support of Diff-Serv-aware MPLS Traffic Engineering,ö draft-ietf-tewg- diff-te-reqts-07.txt, February 2003. [DS-TE] F. Le Faucheur, T. Nadeau, J. Boyle, K. Kompella, W. Townsend, D. Skalecki, " Protocol extensions for support of Diff-Serv-aware MPLS Traffic Engineering," draft-ietf-tewg- diff-te-proto-02.txt, October 2002. Expires September 2003 38 PPVPN QoS framework March 2003 Author's Addresses Fabio Chiussi Lucent Technologies 101 Crawfords Corner Road, Room 4G502 Phone: 732-949-2407 Holmdel, NJ 07733 Email: fabio@lucent.com Jeremy De Clercq Alcatel Francis Wellesplein 1 B-2018 Antwerpen Email: jeremy.de_clercq@alcatel.be Belgium Sudhakar Ganti Tropic Networks 135 Michael Cowpland Drive Email: sganti@tropicnetworks.com Kanata, Ontario, Canada, K2M 2E9 Biswajit Nandy Tropic Networks 135 Michael Cowpland Drive Email: bnandy@tropicnetworks.com Kanata, Ontario, Canada, K2M 2E9 Wing Cheong Lau Lucent Technologies 101 Crawfords Corner Road Phone: 732-949-0979 Holmdel, NJ 07733 Email: lau@lucent.com Nabil Seddigh Email: nseddigh@hotmail.com Sven Van den Bosch Alcatel Francis Wellesplein 1 Phone: 32-3-240-8103 B-2018 Antwerpen Email: sven.van_den_bosch@alcatel.be Belgium Full Copyright Statement "Copyright (C) The Internet Society (date). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be Expires September 2003 39 PPVPN QoS framework March 2003 followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Expires September 2003 40