Francois Le Faucheur, Editor Thomas Nadeau Cisco Systems, Inc. Martin Tatham BT Thomas Telkamp David Cooper Global Crossing Jim Boyle PDNets Waisum Lai, Co-editor Luyuan Fang Jerry Ash AT&T Pete Hicks Core Express Angela Chiu Celion Networks William Townsend Tenor Networks Darek Skalecki Nortel Networks IETF Internet Draft Expires: May, 2002 Document: draft-ietf-tewg-diff-te-reqts-02.txt November, 2001 Requirements for support of Diff-Serv-aware MPLS Traffic Engineering Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are Working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." Le Faucheur, et. al 1 Requirements for Diff-Serv Traffic EngineeringNovember 2001 The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This document presents the Service Provider requirements for support of Diff-Serv aware MPLS Traffic Engineering (DS-TE) as discussed in the Traffic Engineering Working Group Framework document [TEWG-FW]. 1. Introduction 1.1. Problem Statement Diff-Serv is becoming prominent in providing scalable network designs supporting multiple classes of services. In some Diff-Serv networks where optimization of transmission resources on a network-wide basis is not sought, MPLS Traffic Engineering (TE) mechanisms may simply not be used. In other networks, where optimization of transmission resources is sought, Diff-Serv mechanisms [DIFF-MPLS] need to be complemented by existing MPLS Traffic Engineering mechanisms [TE-REQ] [ISIS-TE] [OSPF-TE] [RSVP-TE] [CR-LDP] which operate on an aggregate basis across all classes of service. In this case, Diff-Serv and MPLS TE both provide their respective benefits. Where fine-grained optimization of transmission resources is sought, it is necessary to perform traffic engineering at a per-class level instead of an aggregate level, in order to further enhance networks in performance and efficiency as discussed in [TEWG-FW]. By mapping the traffic from a given class of service on a separate LSP, it allows this traffic to utilize resources available to the given class on both shortest path and non-shortest paths and follow paths that meet engineering constraints which are specific to the given class. This is what we refer to as "Diff-Serv-aware Traffic Engineering (DS-TE)". This document focuses exclusively on the specific environments which would benefit from DS-TE. Some examples include: - networks where bandwidth is scarce (e.g. transcontinental networks) - networks with significant amounts of delay-sensitive traffic - networks where the relative proportion of traffic across classes of service is not uniform Le Faucheur et. al 2 Requirements for Diff-Serv Traffic EngineeringNovember 2001 This document focuses on intra-domain operation. Inter-domain operation is not considered. 1.2. Definitions For the convenience of the reader, relevant Diffserv ([DIFF-ARCH], [DIFF-NEW]) definitions are repeated herein. Behavior Aggregate (BA): a collection of packets with the same codepoint crossing a link in a particular direction. PHB Scheduling Class (PSC): A PHB group for which a common constraint is that ordering of at least those packets belonging to the same microflow must be preserved. Ordered Aggregate (OA): a set of BAs that share an ordering constraint. The set of PHBs that are applied to this set of Behavior Aggregates constitutes a PHB scheduling class. We also repeat the following definition from [TE-REQ]: Traffic Trunk: an aggregation of traffic flows of the same class which are placed inside a Label Switched Path. [Note: in the context of the present document, "flows of the same class" is to be interpreted as "flows from the same Forwarding Equivalence Class which are to be treated equivalently from the DS-TE perspective".] 1.3. Mapping of traffic to LSPs A network may have multiple classes of traffic it wishes to service. Recalling from [DIFF-MPLS], there are several options on how the Diff-Serv Ordered Aggregates (OAs) can be split into multiple Traffic Trunks for mapping onto LSPs when using DSTE. One option is to split traffic into Traffic Trunks which each comprises traffic from multiple OAs. This option is typically used when aggregate traffic engineering is deployed using current MPLS TE mechanisms. Note that this LSP is then, by definition, an E-LSP as defined in [DIFF-MPLS]. Another option is to split traffic into Traffic Trunks which each comprises traffic from a single OA. DS-TE assumes this mode of operation. Note that the LSP used to transport such a Traffic Trunk may then be an L-LSP (as defined in [DIFF-MPLS]) or may be an E-LSP onto which a single (or a subset of the) OA is transported. Thus, traffic from a given node to another given node, is split into multiple Traffic Trunks which are transported over separate LSPs, which can potentially follow a different path through the network. DS-TE precisely takes advantage of this fact and indeed computes a separate path for each LSP. In so doing, DS-TE can take into account Le Faucheur et. al 3 Requirements for Diff-Serv Traffic EngineeringNovember 2001 the specific requirements of the Traffic Trunk transported on each LSP (e.g. bandwidth requirement, preemption priority). Moreover DS- TE can take into account the specific resources which have been allocated inside the network to different sets of Traffic Trunks (e.g. PHB resources allocated to be used by all the Traffic Trunks of a given OA) and specific engineering constraints to be enforced for these sets of Traffic Trunks (e.g. limit all Traffic Trunks from a given OA to x% of link capacity). In brief, DS-TE achieves per LSP constraint based routing with paths that tightly match the specific objectives of each Traffic Trunk and of each OA to whom the traffic Trunk belongs. 2. Application Scenarios 2.1. Scenario 1: Limiting Proportion of Classes on a Link An IP/MPLS network may need to carry a significant amount of VoIP traffic compared to its link capacity. For example, 10,000 uncompressed calls at 20ms packetization result in about 1Gbps of IP traffic, which is significant on an OC-48c based network. In case of topology changes such as link/node failure, VoIP traffic levels can even approach the full bandwidth on certain links. For delay/jitter reasons it is undesirable to carry more than a certain percentage of VoIP traffic on any link. The rest of the available link bandwidth can be used to route other classes corresponding to delay/jitter insensitive traffic (e.g. Best Effort Internet traffic). The exact determination of this "certain" percentage is outside the scope of this requirements draft. During normal operations, the VoIP traffic should be able to preempt other classes of traffic (if these other classes are designated as preemptable and they have lower preemption priority), so that it will be able to use the shortest available path, only constrained by the maximum defined link utilization ratio/percentage of the VoIP class. Existing TE mechanisms only allow constraint based routing of traffic based on a single bandwidth constraint common to all classes, which does not satisfy the needs described here. This leads to the requirement for DS-TE to be able to enforce a different bandwidth constraint for different classes of traffic. In the above example, the bandwidth constraint to be enforced for VoIP traffic may be the "certain" percentage of each link capacity, while the bandwidth constraint to be enforced for the rest of the classes might have their own constraints or have access to the rest of the link capacity. 2.2. Scenario 2: Maintain relative proportion of traffic classes Le Faucheur et. al 4 Requirements for Diff-Serv Traffic EngineeringNovember 2001 Suppose an IP/MPLS network supports 3 classes of traffic. The network administrator wants to perform Traffic Engineering to distribute the traffic load. Assume also that proportion across traffic classes varies significantly depending on the source/destination POPs. With existing TE mechanisms, the proportion of traffic from each class on a given link will vary depending on multiple factors including: - in which order the different TE-LSPs are routed - the preemption priority associated with the different TE-LSPs - link/node failure situations This may make it difficult or impossible for the network administrator to configure the Diff-Serv PHBs (e.g. queue bandwidth) to ensure that each traffic class gets the appropriate treatment. This leads again to the requirement for DS-TE to be able to enforce a different bandwidth constraint for different classes of traffic. This could be used to ensure that, regardless of the order in which tunnels are routed, regardless of their preemption priority and regardless of the failure situation, the amount of traffic of each class routed over a link matches the Diff-Serv scheduler configuration on that link for the corresponding class (e.g. queue bandwidth). As an illustration of how DS-TE would address this scenario, the network administrator may configure the service rate of Diff-Serv queues to (45%,35%,20%) for classes (1,2,3) respectively. The administrator would then split the traffic into separate Traffic Trunks for each class and associate a bandwidth to each LSP transporting those Traffic Trunks. The network administrator may also want to configure preemption priorities of each LSP in order to give highest restoration priority to the highest priority class and medium priority to the medium class. Then DS-TE could ensure that after a failure, class 1 traffic would be rerouted with first access at link capacity but without exceeding its service rate of 45% of the link bandwidth. Class 2 traffic would be rerouted with second access at the link capacity but without exceeding its allotment. Note that where class 3 is the Best-Effort service, the requirement on DS-TE may be to ensure that the total amount of traffic routed across all classes does not exceed the total link capacity of 100% (as opposed to separately limiting the amount of Best Effort traffic to 20 even if there was little class 1 and class 2 traffic). In this scenario, DS-TE allowed for the maintenance of a more steady distribution of classes, even during rerouting. This relied on the required capability of DS-TE to adjust the amount of traffic of each class routed on a link based on the configuration of the scheduler and the amount of bandwidth available for each class. Alternatively, some network administrators may want to solve the problem by having the scheduler dynamically adjusted based on the Le Faucheur et. al 5 Requirements for Diff-Serv Traffic EngineeringNovember 2001 amount of bandwidth of the LSPs admitted for each class. This is an additional requirement on DS-TE. 2.3. Scenario 3: Guaranteed Bandwidth Services In addition to the Best effort service, an IP/MPLS network operator may desire to offer a point-to-point "guaranteed bandwidth" service whereby the provider pledges to provide a given level of performance (bandwidth/delay/loss...) end-to-end through its network from an ingress port to and egress port. The goal is to ensure all "guaranteed" traffic within a subscribed traffic contract, will be delivered within stated tolerances. One approach for deploying such "guaranteed" service involves: - dedicating a Diff-Serv PHB (or a Diff-Serv PSC as defined in [DIFF-NEW]) to the "guaranteed" traffic - policing guaranteed traffic on ingress against the traffic contract and marking the "guaranteed" packets with the corresponding DSCP/EXP value Where very high level of performance is targeted for the "guaranteed" service, it may be necessary to ensure that the amount of "guaranteed" traffic remains below a given percentage of link capacity on every link. Where the proportion of "guaranteed" traffic is high, constraint based routing can be used to enforce such a constraint. However, the network operator may also want to simultaneously perform Traffic Engineering of the rest of the traffic (i.e. non- guaranteed traffic) which would require that constraint based routing is also capable of enforcing a differnet bandwidth constraint, which would be less stringent than the one for guaranteed traffic. Again, this combination of requirements can not be addressed with existing TE mechanisms. DS-TE mechanisms allowing enforcement of a different bandwidth constraint for guaranteed traffic and for non- guaranteed traffic are required. 3. Detailed Requirements for DS-TE This section specifies the functionality that the above scenarios require out of DS-TE implementations. Actual technical protocol mechanisms and procedures to achieve such functionality are outside the scope of this document. 3.1. DS-TE Compatibility While DS-TE is required in a number of situations such as the ones described above, it is important to keep in mind that using DS-TE may impact scalability (as discussed later in this document) and Le Faucheur et. al 6 Requirements for Diff-Serv Traffic EngineeringNovember 2001 operational practices. DS-TE should only be used when existing TE mechanisms combined with Diff-Serv cannot address the network design requirements. Many network operators may choose to not use DS-TE, or to only use it in a limited scope within their network. Thus, the DS-TE solution must be developed in such a way that: (i) it raises no interoperability issues with existing deployed TE mechanisms. (ii) it allows DS-TE deployment to the required level of granularity and scope (e.g. only in a subset of the topology, or only for the number of classes required in the considered network) 3.2. Class-Types The fundamental requirement for DS-TE is to be able to enforce different bandwidth constraints for different sets of Traffic Trunks. [TEWG-FW] introduces the concept of Class-Types when discussing operations of MPLS Traffic Engineering in a Diff-Serv environment. DS-TE refines this definition into the following: Class-Type (CT): the set of Traffic Trunks which share the exact same Bandwidth Constraint, or set of Bandwidth Constraints, on all links, for the purpose of Constraint Based Routing and Admission Control. Note that different LSPs transporting Traffic Trunks from the same CT may be using the same or different preemption priorities as explained in more details in section 3.4 below. For illustration purposes, let's consider the case of a network running 4 Diff-Serv classes of services respectively based on the EF PHB, the AF1x PSC, the AF2x PSC and the BE PHB. The network administrator may decide to deploy DS-TE in the following way: o from every DS-TE Head-end to every DS-TE Tail-end, split traffic into 4 Traffic Trunks: one for traffic of each Diff- Serv class o because the QoS objectives for the AF1x Traffic Trunks and for the AF2x Traffic Trunks may be of similar nature (e.g. both targeting low loss albeit at different levels perhaps), the same (set of) Bandwidth Constraint(s) may be applied collectively over the AF1x Traffic Trunks and the AF2x Traffic Trunks. Thus, the network administrator may only define three CTs: one for the EF Traffic Trunks, one for the AF1x and AF2x Traffic Trunks and one for the Best Effort Traffic Trunks. Le Faucheur et. al 7 Requirements for Diff-Serv Traffic EngineeringNovember 2001 DS-TE must support at least 2 CTs and up to a maximum of 8 CTs. Those are referred to as CTc, 0 <= c <= MaxCT-1 = 7. In a given network, DS-TE must not require the network administrator to always deploy the maximum number of CTs. The network administrator must be able to deploy only the number of CTs actually utilized. 3.3. Bandwidth Constraints We refer to a Bandwidth Constraint Model as the set of rules defining: - the maximum number of Bandwidth Constraints; and - which CTs each Bandwidth Constraint applies to and how. By definition of CT, each CT is assigned either a Bandwidth Constraint, or a set of Bandwidth Constraints. We refer to the Bandwidth Constraints as BCb, 0 <= b <= MaxBC-1 Different models of Bandwidth Constraints are conceivable for control of the CTs. For example, a simple model of separate Bandwidth Constraints per CT could be defined. This model is defined by: - MaxBC= MaxCT - All LSPs supporting Traffic Trunks from CTc use no more than BCc For illustration purposes, on a link of 100 unit of bandwidth where three CTs are used, the network administrator might then configure BC0=30, BC1= 50, BC3=20 such - All LSPs supporting Traffic Trunks from CT0 use no more than 30 (e.g. Voice <= 30) - All LSPs supporting Traffic Trunks from CT1 use no more than 50 (e.g. Premium Data <= 50) - All LSPs supporting Traffic Trunks from CT2 use no more than 20 (e.g. Best Effort <= 20) As another example, a "Russian Doll" model of Bandwidth Constraints may be defined whereby: - MaxBC= MaxCT - All LSPs supporting Traffic Trunks from CTc (with 0<=c<=b) use no more than BCc For illustration purposes, on a link of 100 units of bandwidth where three CTs are used, the network administrator might then configure BC0=60, BC1= 80, BC3=100 such - All LSPs supporting Traffic Trunks from CT0 use no more than 60 (e.g. Voice <= 60) - All LSPs supporting Traffic Trunks from CT0 or CT1 use no more than 80 (e.g. Voice + Premium Data <= 80) Le Faucheur et. al 8 Requirements for Diff-Serv Traffic EngineeringNovember 2001 - All LSPs supporting Traffic Trunks from CT0 or CT1 or CT2 use no more than 100 (e.g. Voice + Premium Data + Best Effort <= 100). More complex Bandwidth Constraints model can also be conceived. Those could involve arbitrary relationships between BCb and CTc. Those could also involve additional concepts such as associating minimum reserved bandwidth to a CT. At the time of writing, it is not clear whether a single model of Bandwidth Constraints is sufficient, which one it should be and how flexible this model really needs to be and what are the implications on the DS-TE technical solution and its implementations. We expect that Bandwidth Constraints models will be refined as part of the specification of the DSTE technical solution. 3.4. Preemption [TEWG-FW] defines the notion of preemption and preemption priority. DS-TE must retain full support of such preemption. The preemption attribute defined in [TE-REQ] must be retained and applicable across all Class Types. The preemption attributes of setup priority and holding priority must retain existing semantics, and in particular the preemption must operate independently of Class Types. This means that if LSP1 contends with LSP2 for resources, LSP1 may preempt LSP2 if LSP1 has a higher preemption priority (i.e. lower numerical priority value) regardless of LSP1's OA/CT and LSP2's OA/CT. In other words, DS-TE must offer eight(8) preemption priority levels to be used by an LSP, that are completely orthogonal to that any other LSP's attributes (eg LSP's OA, LSP's Class Type, etc.). Thus, DS-TE must allow two different LSPs transporting Traffic Trunks of the same Class-Type to use different preemption priorities, and allow the LSP with higher priority to preempt the other one when they contend for resources. Similarly, DS-TE must allow two different LSPs transporting Traffic Trunks from different Class-Types to use different preemption priorities, and allow the LSP with higher priority to preempt the other one when they contend for resources. 3.5. Dynamic Adjustment of Diff-Serv PHBs As discussed in section 2.2, DS-TE may support adjustment of Diff- Serv PHBs parameters (e.g. queue bandwidth) based on the amount of TE-LSPs established for each Class/Class-Type. Le Faucheur et. al 9 Requirements for Diff-Serv Traffic EngineeringNovember 2001 Where this behavior is supported, it must allow for disabling via configuration (thus reverting to PHB treatment with static scheduler configuration independent of DS-TE operations). The dynamic adjustment must take account of the performance requirements of each class when computing required adjustments. 3.6. Overbooking Existing TE mechanisms allow overbooking to be applied on LSPs for Constraint Based Routing and admission control. Historically this has been achieved in TE deployment through factoring overbooking ratios at the time of sizing the LSP bandwidth and/or at the time of configuring the Maximum Reservable Bandwidth on links. DS-TE must also allow overbooking and must effectively allow different overbooking ratios to be enforced for different CTs. DS-TE should optionally allow the effective overbooking ratio of a given CT to be tweaked differently in different parts of the network. 3.7. Restoration With existing TE, restoration policies use standard priority mechanisms such as, for example, the preemption priority to effectively control the order/importance of LSPs for restoration purposes. DS-TE must ensure that similar application of the use of standard priority mechanisms for implementation of restoration policy are not prevented since those are expected to be required for achieving the survivability requirements of DS-TE networks. Further discussion of restoration requirements are presented in the output document of the TEWG Requirements Design Team [SURVIV-REQ]. 4. Solution Evaluation Criteria A range of solutions is possible for the support of the DS-TE requirements discussed above. For example, some solutions may require that all current TE protocols syntax (IGP, RSVP-TE, CR-LDP) be extended in various ways. For instance, current TE protocols could be modified to support multiple bandwidth constraints rather than the existing single aggregate bandwidth constraint. Alternatively, other solutions may keep the existing TE protocols syntax unchanged but modify their semantic to allow for the multiple bandwidth constraints. This section identifies the evaluation criteria that should be used to assess potential DS-TE solutions for selection. Le Faucheur et. al 10 Requirements for Diff-Serv Traffic EngineeringNovember 2001 4.1. Satisfying detailed requirements The solution must address all the scenarios described in section 2 and satisfy all the requirements listed in section 3. 4.2. Flexibility - number of Class Types that can be supported, compared to number identified in Requirements section - number of Classes within a Class-Type 4.3. Extendibility - how far can the solution be extended in the future if requirements for more Class-Types are identified in the future. 4.4. Scalability - impact on network scalability in what is propagated, processed, stored and computed (IGP signaling, IGP processing, IGP database, TE-Tunnel signaling ,...). - how does scalability impact evolve with number of Class- Types/Classes actually deployed in a network. In particular, is it possible to keep overhead small for a large networks which only use a small number of Class- Types/Classes, while allowing higher number of Class- Types/Classes in smaller networks which can bear higher overhead) 4.5. Backward compatibility/Migration - backward compatibility/migration with/from existing TE mechanisms - backward compatibility/migration when increasing/decreasing the number of Class-Types actually deployed in a given network. 5. Security Considerations The solution developed to address the requirements defined in this document must address security aspects. DS-TE is not expected to add specific security requirements beyond those of Diff-Serv and existing TE. Networks which employ Diff-Serv techniques might offer some protection between classes for denial of service attacks. Though depending on how the technology is employed, it is possible for some (lower scheduled) traffic to be more susceptible to traffic anomalies (which include denial of service attacks) occurring within other (higher scheduled) classes. Le Faucheur et. al 11 Requirements for Diff-Serv Traffic EngineeringNovember 2001 References [AF] Heinanen, J et al., "Assured Forwarding PHB Group", RFC2597 [CR-LDP] Jamoussi et al., "Constraint-Based LSP Setup using LDP", draft-ietf-mpls-cr-ldp-05.txt, February 2001 [DIFF-MPLS] Le Faucheur et al, "MPLS Support of Diff-Serv", draft- ietf-mpls-diff-ext-09.txt, April 2001 [DIFF-NEW] Grossman, "New Terminology for Diffserv", work in progress, draft-ietf-diffserv-new-terms-04.txt, March 2001. [ISIS-TE] Smit, Li, IS-IS extensions for Traffic Engineering, draft- ietf-isis-traffic-02.txt, September 2000. [OSPF-TE] Katz, Yeung, Traffic Engineering Extensions to OSPF, draft-katz-yeung-ospf-traffic-04.txt, August 2001. [RSVP-TE] Awduche et al, "RSVP-TE: Extensions to RSVP for LSP Tunnels", draft-ietf-mpls-rsvp-lsp-tunnel-08.txt, February 2001. [TEWG-FW] Awduche et al, A Framework for Internet Traffic Engineering, draft-ietf-tewg-framework-04.txt, April 2001. [TE-REQ] Awduche et al, Requirements for Traffic Engineering over MPLS, RFC2702, September 1999. [SURVIV-REQ] W.S. Lai, D. McDysan, J. Boyle, M. Carlzon, R. Coltun, T, Griffin, E. Kern, and T. Reddington, "Network Hierarchy and Multilayer Survivability," work in progress, October 2001. Authors' Address: Francois Le Faucheur Cisco Systems, Inc. Village d'Entreprise Green Side - Batiment T3 400, Avenue de Roumanille 06410 Biot-Sophia Antipolis France Phone: +33 4 97 23 26 19 Email: flefauch@cisco.com Martin Tatham BT Adastral Park, Martlesham Heath, Ipswich IP5 3RE Le Faucheur et. al 12 Requirements for Diff-Serv Traffic EngineeringNovember 2001 UK Phone: +44-1473-606349 Email: martin.tatham@bt.com Thomas Telkamp Global Crossing Olympia 6 1213 NP Hilversum The Netherlands Phone: +31 35 655 651 Email: telkamp@gblx.net David Cooper Global Crossing 960 Hamlin Court Sunnyvale, CA 94089 USA Phone: (916) 415-0437 Email: dcooper@gblx.net Jim Boyle Protocol Driven Networks, Inc. 1381 Kildaire Farm Road #288 Cary, NC 27511 USA Phone: (919) 852-5160 Email: jboyle@pdnets.com Luyuan Fang AT&T Labs 200 Laurel Avenue Middletown, New Jersey 07748 Phone: (732) 420-1921 Email: luyuanfang@att.com Gerald R. Ash AT&T Labs 200 Laurel Avenue Middletown, New Jersey 07748 Phone: (732) 420-4578 Email: gash@att.com Wai Sum Lai AT&T Labs 200 Laurel Avenue Middletown, New Jersey 07748 Phone: (732) 420-3712 Email: wlai@att.com Pete Hicks CoreExpress, Inc 12655 Olive Blvd, Suite 500 Le Faucheur et. al 13 Requirements for Diff-Serv Traffic EngineeringNovember 2001 St. Louis, MO 63141 Phone: (314) 317-7504 Email: pete.hicks@coreexpress.net Angela Chiu Celion Networks 1 Sheila Drive, Suite 2 Tinton Falls, NJ 07724 Phone: (732) 747-9987 Email: angela.chiu@celion.com William Townsend Tenor Networks 100 Nagog Park Acton, MA 01720 Phone: +1 978-264-4900 Email: btownsend@tenornetworks.com Thomas D. Nadeau Cisco Systems, Inc. 250 Apollo Drive Chelmsford, MA 01824 Phone: (978) 244-3051 Email: tnadeau@cisco.com Darek Skalecki Nortel Networks 3500 Carling Ave, Nepean K2H 8E9 Phone: (613) 765-2252 Email: dareks@nortelnetworks.com Le Faucheur et. al 14