Y. Yu IRCAT Working Group T. Tsou Internet Draft R.Li Intended status: Standard Track Huawei Expires: December 2011 June 11, 2011 Inter-router Cooperative and Teaming for massive scalability draft-yu-ircat-problem-statement-02.txt Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on December 11, 2011. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Yu&Tsou&Li Expires December 11, 2011 [Page 1] Internet-Draft IRCAT Problem Statement June 2011 Abstract To achieve the extremely high scalability, Cooperative router group is a reasonable approach. Problems are stated in this document regarding cooperative router group. New mechanisms are needed to be developed and to be standardized. Table of Contents 1. Introduction...................................................2 2. Terminology....................................................3 3. Inter Cooperative Router Group.................................3 4. Problem Statement..............................................4 4.1. Router Group Membership...................................4 4.2. Router Capability Exchange................................4 4.3. Cooperative Router Group Topology.........................4 4.3.1. An Example of Interior Router Group Network..........5 4.4. Cooperative Router Group Working Load Collaboration.......6 4.4.1. Local Resources Monitoring...........................6 4.4.2. Router Working Load Migration........................6 4.5. Unified Router Group Control Plane........................7 4.6. Router Control Plane and Forwarding Plane Separation......7 5. Conclusions and Recommendations................................7 6. Security Considerations........................................8 7. IANA Considerations............................................8 8. Acknowledgments................................................8 9. Reference:.....................................................8 1. Introduction To achieve scalability, routers are replaced and are becoming larger and larger. One way to achieve such a very high scalability is to design and deploy tightly-coupled homogeneous hardware cluster router. Currently, the tightly-coupled homogeneous hardware cluster router requests the same type of router from same vendor, which means they are identical. There could be another possibility of using different routers (maybe from different vendors) to construct a loosely-coupled heterogeneous cluster router by running one or more protocols to be standardized. The key point of such a technology is about Cooperation and Teaming among routers. Yu&Tsou&Li Expires December 11, 2011 [Page 2] Internet-Draft IRCAT Problem Statement June 2011 The cooperation includes, but not limited to, such scenario in which one router may handle some control plane work on behalf of another router. From the outside, the cooperative router group is required to meet the following minimal requirements: - They are viewed as a single router with its own router ID and is operated as a single network node. - They are always available and some redundancy and backups are built between routers joining the same router group. - They are extremely highly scalable as a new router can be inserted to join such a group when needed. 2. Terminology IRCAT: Inter Router Cooperative and Teaming VRF: Virtual Routing and Forwarding 3. Inter Cooperative Router Group To meet the massive scalability from the newer emerging application, single chassis router or single entity router faces tough time. Even though CPU and memory speed grow very fast, building a giant router is still expensive. Single giant router loses the flexibility of seamlessly scales from low to high. So unite a group of routers as a single router is reasonable approach --- inter router cooperative group. In the cooperative router group, routers have same router ID and collaborate like single router to network. It is not required that routers are the same type. They can be heterogeneous or homogeneous routers in the cooperative router group. Cooperative router group is cooperative. Routers in group are collaborative to working on same tasks. For example, in cooperative router group, BGP is not running on single router. It is running on multiple routers in the router group. All BGP instances are load balanced the whole BGP load. They have same router ID and same IP addresses. The external BGP peers all connect to those common IP addresses. Cooperative router group can be seamlessly scale up and down without disruption of services. New routers can be added into or remove from Yu&Tsou&Li Expires December 11, 2011 [Page 3] Internet-Draft IRCAT Problem Statement June 2011 cooperative router group corresponding to the load of router group goes up and down. To make routers to be cooperative, cooperative router group is facing a series of problems. Some of problems can be solved by some existed mechanisms. But for some problems, new mechanisms are necessary to be developed and further study is needed. 4. Problem Statement 4.1. Router Group Membership When a new router is added into the router group, it won't be member of router group automatically. Automatic discovery, negotiation, authentication etc. are the necessary procedures. After newly added router becomes member of router group, its resources are added into router group global resources. If a router leaves the router group, router group must be able to detect this leave and remove this slice of resources from global resources. Router group membership mechanism may borrow some existing ideas, but it still has some uniqueness under the circumstance of cooperative router group. 4.2. Router Capability Exchange With the join of new group member, how much new resources are added in? Router capability is advertised out. Capability here refers the router's used and available CPU and memory capacities etc. Cooperative router group needs to know the whole resources, so capability database of all members should be kept in router group. Because capability of each member can be changed, capability database is updated from time to time. 4.3. Cooperative Router Group Topology Router group is cooperative and collaborative; this means members of router group have interior interconnections and internal communication. Interior network of router group can either be layer 2 or layer 3. Layer 2 interior network is simple and easy. Layer 3 interior network is better scaled and has some advances in multicast. Each member of router group has private addresses for internal communication. Those addresses are concealed to the outside of the cooperative router group. Yu&Tsou&Li Expires December 11, 2011 [Page 4] Internet-Draft IRCAT Problem Statement June 2011 Meanwhile, cooperative router group keeps its router ID and a set of public addresses. The outside of router group only knows the above public information. In section 4.3.1, an example explains more about private and public addresses of interior router group network. Router group topology has to be obtained if router members need internal unicast and multicast communication. Mac learning in layer 2 network or slightly changed IGP routing protocol in layer 3 networks can do this job well. But in cooperative router group, it needs more. - Router group only has topology information for private addresses. But it dose not have public address locations. - In router group circumstance, same public IP addresses are possible on multiple member routers because multiple instances of application are running on different routers. For example, BGP in router group may not be running on single member routers. It has multiple instances which are running on multiple routers. Those BGP instances have same router ID and common address to outside peers. If BGP peer packet which has destination to the common address, are punt into router group, but faces multiple places with same address, where should this packet be forwarded? 4.3.1. An Example of Interior Router Group Network 10.1.1.1 10.1.1.2 10.1.1.3 +-----------+ +-----------+ +-----------+ | router A | | router B | | router C | | (BGP 1) | ----- | (BGP 2) | ------- | (OSPF) | | (1.1.1.1) | | (1.1.1.1) | | | +-----------+ +-----------+ +-----------+ | LC | | LC | | LC | +-----------+ +-----------+ +-----------+ | | | | | | | | | Figure 1 Interior Router Group Network example As Figure 1, 3 routers A, B and C are in cooperative router group. Their private IP addresses are 10.1.1.1, 10.1.1.2, 10.1.1.3 respectly. Router group has public address 1.1.1.1. There are 2 BGP instances which are running on router A and B. OSPF instance is running on router C. BGP is load balanced, half of BGP peers are handled by router A and second half peers are handled by router B. BGP peers are all connect to IP address 1.1.1.1. In this example, the following scenarios are needed to be addressed. Yu&Tsou&Li Expires December 11, 2011 [Page 5] Internet-Draft IRCAT Problem Statement June 2011 - If a BGP packet is punt up, its destination IP address is 1.1.1.1 and the sent peer is supposed to be handled by BGP instance on router A, forward this packet to router A. - If a BGP packet is punt up, its destination IP address is 1.1.1.1 and the sent peer is supposed to be handled by BGP instance on router B, forward this packet to router B. - If an OSPF packet is punt up, forward this packet to router C. It shows a new mechanism needed to be developed in interior router group network. 4.4. Cooperative Router Group Working Load Collaboration Router group is cooperative. It is not right if some member routers are extremely busy or incur to almost memory panic, in the meantime some other member router have plenty resource but have nothing to do. Under this circumstance, cooperative router group should have a mechanism to move a part of working load from resource exhausted routers to some others which have enough resources. Otherwise cooperative router group is not "cooperative". Ideally this working load collaboration is automatic and dynamic. Working load move should be happened before router's resources are really exhausted. 4.4.1. Local Resources Monitoring It is too late if router completely run out of resource and then remembers it can move some working load out. Router monitors its local resource status from time to time. If it goes to alarm stage, working load migration triggers. 4.4.2. Router Working Load Migration If a router needs to off load some working load. It always it handles this with 2 different manners. The first one is, this router does not have other group member's capability information. It just asks for help loudly (broadcast) and then waits for helper's responses. After getting almost all helper's responses, now the router knows the targets of load migration. Then it negotiates with target routers and load migration starts if negotiation success. The second manner, this router always collects group member's capability information, i.e. capability database. So when the router needs to offload its working load, it already has enough knowledge to know the helpers (migration targets). This router can directly negotiates with migration target Yu&Tsou&Li Expires December 11, 2011 [Page 6] Internet-Draft IRCAT Problem Statement June 2011 routers and load migration starts if the negotiation success. The optimal migration targets selection is a complex process, how to simplify and compromise is another problem. 4.5. Unified Router Group Control Plane Each router has its own control plane, so there are quite a few control planes in a cooperative router group. On the other hand, routers can be added into and remove from router group dynamically and seamlessly. How to effectively utilize the control planes' resource is very tough problem. Control plane virtualization becomes the reasonable approach. A unified router group control plane becomes necessary. By virtualization, the unified router group control plane has global control plane resources pool and each router's control plane becomes a slice of resources in global control plane resource pool. When a new router joins into the router group, a new slice of router control plane resources is added into global resource pool and vice versa. 4.6. Router Control Plane and Forwarding Plane Separation This problem is related to the router group control plane virtualization. It is natural in single router that control plane only manages the local chassis forwarding hardware. In cooperative router group circumstance, especially with the unified router group control plane, control plane on other remote router can manages local forwarding hardware on behalf of local router control plane. So one forwarding hardware and one control plane mapping is not true under cooperative router group circumstance. Forwarding hardware can no longer find out its specific control plane, but to face the unified router group control plane. 5. Conclusions and Recommendations To meet the extremely high scaling requirements from service provider, we have concluded and recommend that: - Cooperative router group is practical and effective approach to meet the extremely high scaling requirement. - Cooperative router group addressed new mechanisms needed to be developed and further study and to be standardized. - Inter Router Cooperative and Teaming (IRCAT) is to standardize the protocols which are running on heterogeneous routers to let them construct as cooperative router group. Yu&Tsou&Li Expires December 11, 2011 [Page 7] Internet-Draft IRCAT Problem Statement June 2011 6. Security Considerations In cooperative router group, member routers are cooperative and trusted. A malicious group member can easily down the entire router group. So the sophisticated authentication is important when member joins the router group. Routers in router group still may be attacked from outside network. 7. IANA Considerations This document does not add additional Security Considerations. 8. Acknowledgments Cathy Wu helped to translate the format of this document. She is gratefully acknowledged. This document was prepared using 2-Word-v2.0.template.dot. 9. Reference: [1] [CloudNET] T. Wood, "The Case for Enterprise-ready Virtual Private Clouds" [2] [RFC4271] Y. Rekhter, T. Li, S. Hares, "A Border Gateway Protocol 4 (BGP-4)", RFC4271, 2006 Author's Addresses Yu&Tsou&Li Expires December 11, 2011 [Page 8] Internet-Draft IRCAT Problem Statement June 2011 Yang Yu Huawei Technologies 2300 Central Expressway, Santa Clara, CA 95050, USA Email: yyu@huawei.com Tina Tsou Huawei Technologies 2300 Central Expressway, Santa Clara, CA 95050, USA Email: tena@huawei.com Renwei Li Huawei Technologies 2300 Central Expressway, Santa Clara, CA 95050, USA Email: renwei.li@huawei.com Copyright (c) 2011IETF Trust and the persons identified as authors of the code. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: o Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Yu&Tsou&Li Expires December 11, 2011 [Page 9] Internet-Draft IRCAT Problem Statement June 2011 o Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. o Neither the name of Internet Society, IETF or IETF Trust, nor the names of specific contributors, may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Yu&Tsou&Li Expires December 11, 2011 [Page 10]