Internet Working Group J.-L. Le Roux Internet Draft J.-Y. Mazeas Proposed Category: Standard Track France Telecom Expires: January 2006 J.-P. Vasseur S. Boutros Cisco Systems D. Papadimitriou Alcatel July 2005 Procedure to handle (G)MPLS-TE control plane saturation draft-leroux-ccamp-ctrl-saturation-01.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Le Roux, et al. Standard Track - Expires January 2006 [Page 1] Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005 Abstract This document defines extensions to RSVP-TE (Resource Reservation Protocol-Traffic Engineering) and IGP (IS-IS and OSPF), in order to notify about control plane resources saturation, when an LSR runs out of control plane resources available to support any additional LSP. Such notification may serve as trigger for the impacted Head-end LSR to take appropriate actions. Table of Contents 1. Terminology.................................................2 2. Introduction................................................3 3. RSVP-TE Saturation..........................................4 4. Signalling extensions.......................................4 4.1. New RSVP Error Code.........................................4 4.2. Mode of operations..........................................5 4.2.1. Procedures for a saturated LSR..............................5 4.2.2. Procedures for a Head-End LSR...............................5 5. Routing extensions..........................................5 5.1. Encoding of the Saturation TLV..............................5 5.2. Elements of procedure.......................................6 5.2.1. Procedures for a Saturated LSR..............................7 5.2.2. Procedures for a Head-End LSR...............................7 6. Inter-Provider considerations...............................7 7. Security Considerations.....................................8 8. Acknowledgements............................................8 9. References..................................................8 9.1. Normative references........................................8 9.2. Informative references......................................8 10. Authors' Address:...........................................9 11. Intellectual Property Statement.............................9 Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119. 1. Terminology LSR: Label Switching Router TE-LSP: MPLS Traffic Engineering Label Switched Path Head-End LSR: Head of a TE-LSP Tail-End LSR: Tail of a TE-LSP PSB: RSVP Path State Block Le Roux, et al. Standard Track - Expires July 2005 [Page 2] Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005 RSB: RSVP Reservation State Block RSVP-TE Saturation: State of an LSR that cannot accept any new TE-LSP due to configured thresholds, or control plane limitations ASBR: Autonomous System Border Router 2. Introduction Many service providers have deployed RSVP-TE [RFC3209] to setup TE- LSPs and achieve Traffic Engineering objectives. In some circumstances, MPLS-TE deployments in large networks may require maintaining a high number of TE-LSPs on transit LSRs, which may lead to the instantiation of a high number of RSVP states and hence consume a large amount of LSR control plane resources (e.g. memory, CPU). Control plane capacities of an LSR are of course not infinite and there may be some circumstances whereby an LSR may run out of a specific control plane resource such as memory or CPU available for the RSVP process; such resource shortage may lead to the inability for the LSR to handle additional TE-LSPs. For instance, the LSR has no longer enough memory to instantiate a new RSVP state block (PSB, RSB [RFC2209]). Furthermore, there may be circumstances whereby the operator may want to limit, by configuration, the maximum number of RSVP-TE sessions that can be accepted and maintained by an LSR at the Ingress, Transit or Egress position. This document defines a particular RSVP-TE state (referred to as the "Saturated" state) whereby the LSR cannot handle any additional TE- LSP, either because the maximum number of TE-LSPs allowed is reached or because there is no longer enough control plane resources (CPU, memory,à) allocated to the RSVP process to instantiate and maintain a new session state block. There may be cases, particularly in large scale RSVP-TE deployments, where an LSR receives a Path message for a new LSP, while it is under the Saturated state. In such situation, the LSR should simply reject the LSP setup, and notify the head-end LSR of its control plane saturation state. For that purpose, this document specifies: - A RSVP-TE extension allowing a saturated LSR to reject any new LSP and notify (using a new Error code and a set of sub-codes carried in a Path Error message) the head-end LSR about its saturation; - IGP extensions (that complement the RSVP-TE extension) in order for a LSR to advertise its current saturation status (saturated or Le Roux, et al. Standard Track - Expires July 2005 [Page 3] Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005 not). This allows other head-end LSRs avoiding the set up of new TE LSP through saturated nodes and also discovering that a given node is no longer saturated. These routing and signalling notifications are complementary. They may serve as trigger for the Head-End LSRs to take appropriate actions. 3. RSVP-TE Saturation A RSVP-TE engine is usually designed such that the maximum number of LSPs it supports equals the total number of LSP with a resource allocation corresponding to the min LSP bandwidth (set e.g. to 2 Mbps) configured in the context of its admission control mechanism. However, soft-provisioning techniques (e.g. LSP established with no bandwidth reservation) are challenging this rule of thumb since the number of supported LSPs needs now to take not only data plane resources allocation into account but also control plane resources. An RSVP node enters the Saturated state when it runs out of specific control plane resource such as the memory or CPU (globally or for the RSVP process) or above a pre-configured threshold or when a pre- configured maximum number of LSPs allowed is reached. Criteria to trigger such saturated state are local to the LSR and outside of the scope of this document. Note that it is recommended to introduce some hysteresis for saturation state transition, so as to avoid oscillations. For instance if the number of TE-LSPs is bounded (by configuration), two thresholds should be configured: An upper-threshold and a lower- threshold with upper-threshold > lower-threshold. An LSR enters the saturated state when the number of TE-LSPs reaches the upper- threshold and leaves the saturated state when it goes down the lower- threshold. 4. Signalling extensions 4.1. New RSVP Error Code This document specifies a new Error Code (suggested value to be confirmed by IANA) for the RSVP ERROR_SPEC object: 26 Control Plane Saturation The following defines error values sub-codes for the new Control Plane Saturation Error Code: 1 Saturation unspecified 2 Memory saturation 3 CPU saturation 4 Max number of LSPs reached 100-255 Reserved Le Roux, et al. Standard Track - Expires July 2005 [Page 4] Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005 Procedures are detailed in section 4.2 below. 4.2. Mode of operations 4.2.1. Procedures for a saturated LSR On receipt of a Path message for a new LSP, a saturated LSR compliant with this document SHOULD reject the LSP setup and send back a PathErr Message with the Path_State_Removed flag set in the ERROR_SPEC object, with the Error Code "Saturation" and optionally an Error sub-code value mentioning the reasons for the saturation. The address carried within the ERROR_SPEC object SHOULD be set to the TE router id of the saturated node. The IF_ID ERROR_SPEC object MAY be used in case saturation can be determined as coming from a given adjacency. To avoid inability of the saturated node to generate PathErr messages, it is expected that the locally pre-configured threshold on control plane resources is such that it allows generation of such messages. 4.2.2. Procedures for a Head-End LSR On receipt of a PathErr message with the Error Code "Saturation" and with the Path_State_Removed flag not set, the Head-End LSR SHOULD send a PathTear message for the rejected LSP. 5. Routing extensions It is desirable to augment the signaling extensions by specific link state routing ([ISIS] and [OSPF]) extensions that would allow an LSR to advertise the state of its RSVP process (saturated or not). This information could then be taken into account by all LSRs (and not only LSRs on the path of a rejected LSPs), in order to avoid a saturated node when computing the path of a new TE-LSP, and also to automatically discover when a node is no longer saturated. A new TLV is defined for OSPF and ISIS, named the ôSaturation TLVö, to be included in the Router Information LSA for OSPF [OSPF-CAP] and in the CAPABILITY TLV for ISIS [ISIS-CAP]. The TLV structure consists of a fixed 8 bit flags field. Currently 4 flags are defined, to indicate if the LSR is saturated or not as well as the reasons for the saturation. 5.1. Encoding of the Saturation TLV This document defines a new TLV (named the Saturation TLV) allowing an LSR advertising if its control plane is saturated or not, where, - With OSPF, the Saturation TLV is a TLV of the Router Information LSA defined in [OSPF-CAP]. The TLV type is TBD (To be assigned Le Roux, et al. Standard Track - Expires July 2005 [Page 5] Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005 by IANA). - With ISIS, the Saturation TLV is a sub-TLV of the ISIS CAPABILITY TLV defined in [ISIS-CAP]. The sub-TLV type is TBD (To be assigned by IANA). All relevant generic TLV encoding rules (including TLV format, padding and alignment, as well as IEEE floating point format encoding) defined in [OSPF] and [ISIS] are applicable to this new sub-TLV. The Saturation TLV is a series of 8 bit flags. Its format is illustrated below: 0 1 2 3 4 5 6 7 +-+-+-+-+-+-+-+-+ |S|C|M|N| Res | +-+-+-+-+-+-+-+-+ Four bits are currently defined: -S bit: When set this indicates that the node is saturated and cannot support any new TE-LSP. When not set this indicates that the node is not saturated and can support new TE-LSPs. -C bit: When set this indicates that the saturation is due to CPU overload. -M bit: When set this indicates that the saturation is due to memory shortage . -N bit: When set this indicates that the maximum number of LSPs allowed is reached. C, M or N are not exclusive. They can be set only if S is set. If S is set and other flags are cleared this means that the reason for the saturation is unspecified. Note that new flags may be defined in the future. The absence of the saturation TLV means that the saturation status is unspecified. 5.2. Elements of procedure A router SHOULD originate a new IS-IS LSP/OSPF LSA whenever a saturation state change occurs or whenever required by the regular IS-IS/OSPF procedure (LSP/LSA refresh). It is expected that a proper implementation will support dampening algorithms so as to dampen IGP flooding, thus preserving the IGP scalability. Le Roux, et al. Standard Track - Expires July 2005 [Page 6] Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005 The flooding scope of this saturation state advertisement SHOULD be limited to a single IGP area. This implies that this TLV SHOULD be carried within an OSPF type 10 Router Information LSAs or within an ISIS CAPIBILITY TLV with the S flag cleared. Note that a saturation satus change MAY trigger CSPF computation, but MUST not trigger normal SPF computation. 5.2.1. Procedures for a Saturated LSR Once an LSR enters the saturation state, the conditions of which are implementation specific, it SHOULD originate LSPs/LSAs with the S bit of the Saturation TLV set and potentially with one or more "reason" bits set. Once a LSR leaves the saturation state, the conditions of which are implementation specific, it SHOULD originate LSPs/LSAs with all bits of the Saturation TLV cleared. 5.2.2. Procedures for a Head-End LSR A head-end LSR computing a path of a TE-LSP should avoid saturated nodes. Note also that when a head-end LSR detects that a node on the path of an already established TE-LSP, becomes saturated, it should not reroute this TE-LSP. Note that when a head-End LSR detects that an LSR is no longer saturated it may re-use this LSR for establishing new TE LSPs, and may try to reoptimize some TE-LSPs, should the path along the no- longer saturated LSR be desirable. This procedure may be delayed, potentially using some LSP setup jittering between head-end LSRs, in order to avoid a signalling storm that may again saturate the node, that is, to avoid TE-LSP routing oscillations. Note that the procedures discussed above are local implementation issues. 6. Inter-Provider considerations For confidentially purpose in an inter-provider MPLS-TE context (see [INTER-AS-REQ]), a provider may desire to hide to other providers, a saturation that could occur in its own network. For that purpose an ASBR may modify the RSVP error code/sub-code before forwarding the PathErr message to an upstream provider. For instance, if a provider wants to hide the reason for the saturation, it should update the error sub-code to 1. If the provider wants to entirely hide the saturation, it may change the error code. Note that this is a local policy-based decision. Le Roux, et al. Standard Track - Expires July 2005 [Page 7] Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005 7. Security Considerations This document does not raise any new security issue. 8. Acknowledgements We would like to thank Thomas Morin, Bruno Decraene, Adrian Farrel, Arthi Ayyangar, Muthurajah Sivabalan, Rudy Figaro, David Ward, Reshad Rahman, and Stefano Previdi for the really useful comments and suggestions. 9. References 9.1. Normative references [RFC] Bradner, S., 'Key words for use in RFCs to indicate requirements levels', RFC 2119, March 1997. [BCP79] Bradner, S., Ed., 'Intellectual Property Rights in IETF Technology', BCP 79, RFC 3979, March 2005. [RSVP] Braden, R., Zhang, L., Berson, S., Herzog, S. and S. Jamin, 'Resource ReSerVation Protocol (RSVP) -- Version 1, Functional Specification', RFC 2205, September 1997. [RFC2209] Braden, R., Zhang, L., 'Resource ReSerVation Protocol (RSVP) -- Version 1 Message Processing Rules', RFC2209, September 1997. [RSVP-TE] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V. and G. Swallow, 'RSVP-TE: Extensions to RSVP for LSP Tunnels', RFC 3209, December 2001. [ISIS] "Intermediate System to Intermediate System Intra-Domain Routing Exchange Protocol " ISO 10589. [OSPF] Moy, J., "OSPF Version 2", RFC 2328, April 1998. [ISIS-CAP] Vasseur, J.P., Aggarwal, R., Shen, N. et al. "IS-IS extensions for advertising router information", draft-ietf-isis- caps-03.txt, work in progress. [OSPF-CAP] Lindem, A., Shen, N., Aggarwal, R., Shaffer, S., Vasseur, J.P., "Extensions to OSPF for advertising Optional Router Capabilities", draft-ietf-ospf-cap-07.txt, work in progress. 9.2. Informative references [INTER-AS-REQ] Zhang, R., Vasseur, J.P., "MPLS Inter-AS Traffic Engineering requirements", draft-ietf-tewg-interas-mpls-te-req- 09.txt, work in progress Le Roux, et al. Standard Track - Expires July 2005 [Page 8] Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005 10. Authors' Address: Jean-Louis Le Roux France Telecom 2, avenue Pierre-Marzin 22307 Lannion Cedex France jeanlouis.leroux@francetelecom.com Jean-Yves Mazeas France Telecom 2, avenue Pierre-Marzin 22307 Lannion Cedex France jeanyves.mazeas@francetelecom.com Jean-Philippe Vasseur CISCO Systems, Inc. 300 Beaver Brook Boxborough, MA 01719 USA Email: jpv@cisco.com Sami Boutros Cisco Systems 2000 Innovation Drive, Kanata, Ontario, Canada K2K 3E8 sboutros@cisco.com Dimitri Papadimitriou Alcatel Francis Wellensplein 1, B-2018 Antwerpen, Belgium Email: dimitri.papdimitriou@alcatel.be 11. Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at Le Roux, et al. Standard Track - Expires July 2005 [Page 9] Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005 http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society (2005). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Le Roux, et al. Standard Track - Expires July 2005 [Page 10]