IRTF Routing Research Group F. Kastenholz, Internet Draft (ed) Document April 2002 Category: Informational Requirements For a Next Generation Routing and Addressing Architecture < draft-irtf-routing-reqs-groupa-00.txt> Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Kastenholz (ed) Informational - Expires 10/2002 1 Inter-domain Routing Requirements March 2002 Table of Contents 1 Abstract...................................................3 2 Conventions used in this document..........................3 3 Requirements...............................................4 3.1 Architecture..........................................4 3.2 Separable Components..................................5 3.3 Scalable..............................................6 3.4 Lots of Interconnectivity.............................7 3.5 Random Structure......................................8 3.6 Multi-homing..........................................9 3.7 Multi-path............................................9 3.8 Convergence..........................................10 3.9 Routing System Security..............................12 3.10 End Host Security....................................14 3.11 Rich Policy..........................................14 3.11.1 Routing Information Policies......................14 3.11.2 Traffic Control Policies..........................15 3.12 Incremental Deployment...............................16 3.13 Mobility.............................................16 3.14 Address Portability..................................17 3.15 Multi-Protocol.......................................17 3.16 Abstraction..........................................17 3.17 Simplicity...........................................18 3.18 Robustness...........................................18 3.19 Media Independence...................................19 3.20 Stand-alone..........................................19 3.21 Safety of Configuration..............................19 3.22 Renumbering..........................................19 3.23 Multi-prefix.........................................19 3.24 Cooperative Anarchy..................................19 3.25 Network Layer Protocols and Forwarding Model.........20 3.26 Routing Algorithm....................................20 3.27 Positive Benefit.....................................20 3.28 Administrative Entities and the IGP/EGP Split........20 4 Non-Requirements..........................................21 4.1 Forwarding Table Optimization........................21 4.2 Traffic Engineering..................................21 4.3 Multicast............................................22 4.4 QOS..................................................22 4.5 IP Prefix Aggregation................................22 4.6 Perfect Safety.......................................23 4.7 Dynamic Load Balancing...............................23 4.8 Renumbering of hosts and routers.....................23 4.9 Host Mobility........................................23 4.10 Clean Slate..........................................24 Kastenholz Informational -- Expires 10/2002 2 Inter-domain Routing Requirements March 2002 5 Discussion of other Inter-Domain Routing Requirements documents....................................................24 5.1 Comments on "Architectural Requirements for Inter-Domain Routing in the Internet"..................................24 5.2 Comments on "Future Domain Routing Requirements".....25 6 Security Considerations...................................28 7 IANA Considerations.......................................28 8 References................................................28 9 Acknowledgments...........................................28 10 Editor's Addresses........................................29 1 Abstract This note sets requirements for a new routing and addressing architecture for the Internet. These requirements were developed by the IRTF's Routing Research Group. This draft is the product of Group ~B, which is one of the subgroups in the IRTF-Routing Research Group working on requirements for routing solutions for the future. This document sets out requirements that we believe are important for a future routing architecture and routing protocols. The IRTF Routing Research Group (RRG) does not endorse this set of requirements or any other set of requirements as the one and only set of requirements. The seemingly simple requirement is to "fix routing and addressing". However, in order to "fix" it, we also need to understand how routing and addressing are _actually_ used today. This is not a straightforward task. Service providers and network administrators have many different operational and administrative needs. Quite often, the only tool available to meet those needs is the routing system and they have proven amazingly crafty at using that tool in ways that were never envisioned. Thus, one of the most important steps in developing these requirements has been to learn what the operational community really is doing with the routing system, why they are doing it, and then abstracting those tasks into requirements. 2 Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [2]. Kastenholz Informational -- Expires 10/2002 3 Inter-domain Routing Requirements March 2002 We believe that we have captured the essential requirements for one possible future routing and addressing architecture for the Internet. We do not consider features, functions, or capabilities not described in this note to be critically important for a future routing and addressing architecture and they should be considered optional. A specific architecture MAY include OR exclude them without conflicting with this note. There are many IETF documents that define specific terms. We generally choose not to use these strict, well-crafted, definitions. They were made with the current architecture and protocols in mind. A goal of this work is to foster development of new and different architectures. By using the "old" terms we may inadvertently limit or constrain avenues of enquiry and investigation, possibly preventing consideration of promising architectures. 3 Requirements This chapter presents the requirements. The requirements presented in this section are NOT presented in any order. A later version of this note may order the requirements in some kind of priority. 3.1 Architecture The new routing and addressing protocols, data structures, and algorithms MUST be developed from a clear, well thought out, documented, architecture. The new routing and addressing system must have an architectural specification which describes all of the routing and addressing elements, their interactions, what functions the system performs, and how it goes about performing them. The architectural specification does not go into issues such as protocol and data structure design. The Architecture SHOULD be agnostic with regard to specific algorithms and protocols. Doing architecture before doing detailed protocol design is good engineering practice. This allows the architecture to be reviewed and commented upon, with changes made as necessary, when it is still easy to do so. Also, by producing an architecture, the eventual users of the protocols (the operations community) will have a better understanding of how the designers of the protocols meant them to be used. Kastenholz Informational -- Expires 10/2002 4 Inter-domain Routing Requirements March 2002 3.2 Separable Components The architecture MUST place different functions into separate components. Separating functions, capabilities, and so forth, into individual components, and making each component "stand alone" is generally considered by system architects to be "A Good Thing". It allows individual elements of the system to be designed and tuned to do their jobs "very well". It also allows for piecemeal replacement and upgrading of elements as new technologies and algorithms become available. The architecture MUST have the ability to replace or upgrade existing components, and to add new ones, without disrupting the remaining parts of the system. Operators must be able to roll out these changes and additions incrementally (i.e. no "flag days"). These abilities are needed to allow the architecture to evolve as the Internet changes. The Architecture Specification shall define each of these components, their jobs, and their interactions. Some thoughts to consider along these lines are o Making topology and addressing separate subsystems. This may allow highly optimized topology management and discovery without constraining the addressing structure or physical topology in unacceptable ways. o Separate "fault detection and healing" from basic topology. From Mike O'Dell: "Historically the same machinery is used for both. While attractive for many reasons, the availability of exogenous topology information (i.e., the intended topology) should, it seems, make some tasks easier than the general case of starting with zero knowledge. It certainly helps with recovery in the case of constraint satisfaction. In fact, the intended topology is a powerful way to state certain kinds of policy. o Making policy definition and application a separate subsystem, layered overtop of the others. The architecture should also separate topology. routing and addressing from the application that use those components. This implies that applications such as policy definition, forwarding, and circuit and tunnel management are separate subsystems layered overtop of the basic topology, routing, and addressing systems. Kastenholz Informational -- Expires 10/2002 5 Inter-domain Routing Requirements March 2002 3.3 Scalable Scaling is the primary problem facing the routing and addressing architecture today. This problem must be solved and it must be solved for the long term. The Architecture MUST support a large and complex network. Ideally, it will serve our needs for the next 20 years. Unfortunately 1. We do not know how big the Internet will grow over that time, and 2. The architecture developed from these requirements may change the fundamental structure of the Internet, and therefore its growth patterns. This change makes it difficult to predict future growth patterns of the Internet. As a result, we can't quantify the requirement in any meaningful way. Using today's architectural elements as a mechanism for describing things, we believe that the network could grow to 1. Tens of thousands of ASes and 2. Tens to hundreds of millions of prefixes during the lifetime of this architecture. These sizes are given as a 'flavor' for how we expect the Internet to grow. We fully believe that any new architecture may eliminate some current architectural elements and introduce new ones. A new routing and addressing architecture designed to a specific network size would be inappropriate. First, the cost of routing calculations is based only in part on the number of AS's or prefixes in the network. The number and locations of the links in the network is also a significant factor. Second, past predictions of Internet growth and topology patterns have proven to be wildly inaccurate so developing an architecture to a specific size goal would at best be shortsighted. Therefore we will not make the scaling requirement based on a specific network size. Instead, the new routing and addressing architecture should have the ability to constrain the increase in load (CPU, memory space and bandwidth, and network bandwidth) on ANY SINGLE ROUTER to be less than these specific functions: 1. The computational power and memory sizes required to execute the routing protocol software and to contain the tables must be less than the growth in hardware capabilities described by Moore's Law, which has hardware capabilities doubling every 18 months or so. Other observations indicate that memory sizes double every 2 years or so. Kastenholz Informational -- Expires 10/2002 6 Inter-domain Routing Requirements March 2002 2. Network bandwidth and latency are some key constraints on how fast routing protocol updates can be disseminated (and therefore how fast the routing system can adapt to changes). Raw network bandwidth seems to quadruple every 3 years or so. However, it seems that there are some serious physics problems in going faster than 40gbits (OC768). We should not expect raw network link speed to grow much beyond OC768. In addition, for economic reasons, large swathes of the core of the Internet will still operate at lower speeds, possibly as slow as DS3. Furthermore, in some sections of the Internet even lower speed links are found. Corporate access links are often T1, or slower. Low-speed radio links exist. Intra-domain links may be T1 or fractional-T1 (or slower). Therefore, the architecture MUST NOT make assumptions about the bandwidth available. 3. The speeds of high-speed RAMS (SRAMs, used for caches and the like) are growing, though slowly. Because of their use in caches and other very specific applications, these RAMs tend to be small, a few megabits, and the size of these RAMs is not increasing very rapidly. On the other hand, the speed of "large" memories (DRAMs) is increasing even slower than that for the high speed RAMS. This is because the development of these RAMs is driven by the PC market, where size is very important, and low speed can be made up for by better caches. Memory access rates should not be expected to increase significantly. The growth in resources available to any one router will eventually slow down. It may even stop. Even so, the network will continue to grow. The routing and addressing architecture must continue to scale in even this extreme condition. We cannot continue to add more computing power to routers forever. Other strategies must be available. Some possible strategies are hierarchy, abstraction, and aggregation of topology information.. 3.4 Lots of Interconnectivity The new routing and addressing architecture MUST be able to cope with a high degree of interconnectivity in the Internet. That is, there are large numbers of alternate paths and routes among the various elements. Mechanisms are required to prevent this interconnectivity (and continued growth in interconnectivity) from causing tables, compute time, and routing protocol traffic to grow without bound. The "cost" to Kastenholz Informational -- Expires 10/2002 7 Inter-domain Routing Requirements March 2002 the routing system of an increase in complexity MUST be limited in scope; sections of the network that do not see, or do not care about, the complexity ought not pay the cost of that complexity. Over the past several years, the Internet has seen an increase in interconnectivity. Individual end sites (companies, customers, etc), ISPs, exchange points, and so on, all are connecting up to more "other things". Company's multi-home to multiple ISPs, ISPs peer with more ISPs, and so on. These connections are made for many reasons, such as getting more bandwidth, increased reliability and availability, policy, and so on. However, this increased interconnectivity has a price. It leads to more scaling problems as it increases the number of AS paths in the networks. Any new architecture must assume that the Internet will become "meshier". It MUST NOT assume, nor can it dictate, certain patterns or limits on how various elements of the network interconnect. Another facet of this requirement is that there may be multiple valid, loop free, paths available to a destination. When there are multiple valid, loop free, paths available, all such paths MUST be available for forwarding traffic. We wryly note that one of the original design goals of IP was to support a large, heavily interconnected, network, which would be highly survivable (such as in the face of a nuclear war). 3.5 Random Structure The routing and addressing architecture MUST NOT make any constraints on or assumptions about the topology or connectedness of the elements comprising the Internet. The routing and addressing architecture MUST NOT presume any particular network structure. The network does not have a "nice" structure. In the past we used to believe that there was this nice "backbone/tier-1/tier-2/end-site" sort of hierarchy. This is not so. Therefore, any new Architecture must not presume any such structure. Some have proposed that a geographic addressing scheme be used, requiring exchange points to be situated within each geographic 'region'. There are many reasons why we believe this to be a bad approach, but those arguments are irrelevant. The main issue is that the routing architecture should not presume a specific network structure. Kastenholz Informational -- Expires 10/2002 8 Inter-domain Routing Requirements March 2002 3.6 Multi-homing The Architecture MUST provide multi-homing for all elements of the Internet. That is, multihoming of hosts, subnetworks, end- sites, "low-level" ISPs, and backbones (i.e. lots of redundant interconnections) must be supported. Among the reasons to multi-home are reliability, load sharing, and performance tuning. The term "multihoming" may be interpreted in its broadest sense -- one "place" has multiple connections or links to another "place". The architecture MUST NOT limit the number of alternate paths to a multi-homed site. When multi-homing, it MUST be possible to use one, some (more than one but less than all) or all of the available paths to the multi-homed site. The multi-homed site must have the ability to declare which path(s) are used and under what conditions (for example, one path may be declared "primary" and the other "backup" and to be used only when the primary fails). A current problem in the Internet is that multihoming leads to undue increases in the size of the BGP routing tables. The new architecture must support multi-homing without causing undue routing table growth. 3.7 Multi-path As a corollary to multi-homing, the Architecture MUST allow for multiple paths from a source to a destination to be active at the same time. These paths need not have the same attributes. Policies are to be used to disseminate the attributes and to classify traffic for the different paths. There must be a rich "language" for use in specifying the rules for classifying the traffic and assigning classes of traffic to different paths (or prohibiting it from certain paths). The rules for classification should allow traffic to be classified based on o IPv6 FlowIDs o TOS byte o Source and/or Destination prefixes o Random selections at some probability o ... A mechanism is needed that allows operators to plan and manage the traffic load on the various paths. To start, this mechanism can be semi-automatic, or even manual. Eventually it ought to become fully automatic. Kastenholz Informational -- Expires 10/2002 9 Inter-domain Routing Requirements March 2002 When multi-path forwarding is used, options must be available to preserve packet ordering where appropriate (such as for individual TCP connections). Past experience with dynamic load-balancing and management over multiple paths has been bad. Typically, traffic would oscillate, first all traffic would go over one path, then it would all be 'migrated' to a different path, and then back again. Significant research is needed in this area. 3.8 Convergence The speed of convergence (also called the "stabilization time") is the time it takes for a router's routing processes to come up with a new, stable, "solution" (i.e. forwarding information base) after a change someplace in the network. In effect, what happens is that the output of the routing calculations stabilizes -- the Nth iteration of the software produces the same results as the N-1th iteration. The speed of convergence is generally considered to be a function of the number of subnetworks in the network and the amount of connections between those networks. As either number grows, the time it takes to converge increases. In addition, a change can "ripple" back and forth through the system. One change can go through the system, causing some other router to change its advertised connectivity, causing a new change to ripple through. These oscillations can take a while to work their way out of the network. It is also possible that these ripples never die out. In this situation the routing and addressing system is unstable; it never converges. Finally, it is more than likely that the routers comprising the Internet never converge simply because the Internet is so large and complex. Assume it takes S seconds for the routers to stabilize on a solution for any one change to the network. Also assume that changes occur, on average, every C seconds. Because of the size and complexity of the Internet, C is now less than S. Therefore, if a change, C1, occurs at time T, the routing system would stabilize at time T+S, but a new change, C2, will occur at time T+C, which is before T+S. The system will start processing the new change before it's done with the old. This is not to say that all routers are constantly processing changes. The effects of changes are like ripples in a pond. They spread outward from where they occur. Some routers will be processing just C1, others C2, others both C1 and C2, and others neither. Kastenholz Informational -- Expires 10/2002 10 Inter-domain Routing Requirements March 2002 We have two separate scopes over which we can set requirements with respect to convergence: 1. Single Change In this requirement a single change of any type (link addition or deletion, router failure or restart, etc.) is introduced into a stabilized system. No additional changes are introduced. The system must restabilize within TBDms. This requirement is a fairly abstract one as it would be impossible to test in a real network. 2. System-wide Defining a single target for maximum convergence time for the real Internet is absurd. As we mentioned earlier, the Internet is large enough and diverse enough so that it is quite likely that new changes are introduced somewhere before the system fully digests old ones. So, the first requirement here is that there must be mechanisms to limit the scope of any one change's visibility and effects. The number of routers that have to perform calculations in response to a change is kept small, as is the settling time. The second requirement is based on the following assumptions - the scope of a change's visibility and impact can be limited. That is, routers within that scope know of the change and recalculate their tables based on the change. Routers outside of the scope don't see it at all. - Within any scope, S, network changes are constantly occurring and the average inter-change interval is Tc seconds. - There are Rs routers within scope S - A subset of the destinations known to the routers in S, Ds, are impacted by a given change. - We can state that for Z% of the changes, within Y% of Tc seconds after a change, C, X% of the Rs routers have their routes to Ds settled to a useful answer (useful meaning that packets can get to Ds, thought perhaps not by the optimal path -- this allows some 'hunting' for the optimal solution) X, Y, Z, ARE TBD This requirement implies that the scopes can be kept relatively small in order to minimize Rs and maximize Tc. Kastenholz Informational -- Expires 10/2002 11 Inter-domain Routing Requirements March 2002 The growth rate of the convergence time MUST NOT be related to the growth rate of the Internet as a whole. This implies that the convergence time either 1. Not be a function of basic network elements (such as prefixes and links/paths), and/or 2. That the Internet be continuously divisible into chunks that limit the scope and effect of a change, thereby limiting the number of routers, prefixes, links, and so on involved in the new calculations. 3.9 Routing System Security The security of the Internet's routing system is paramount. If the routing system is compromised or attacked, the entire Internet can fail. This is unacceptable. Any new Architecture must be secure. Architectures by themselves are not secure. It is the implementation of an architecture; its protocols, algorithms, and data structures, that are secure. These requirements apply primarily to the implementation. The architecture MUST provide the elements that the implementation needs to meet these security requirements. Also, the architecture MUST NOT prevent these security requirements from being met. Security means different things to different people. In order for this requirement to be useful, we must define what we mean by security. We do this by identifying the attackers and threats we wish to protect against. They are: Masquerading The system, including its protocols, MUST be secure against intruders adopting the identity of other known, trusted, elements of the routing system and then using that position of trust for carrying out other attacks. Protocols MUST use cryptographically strong authentication. DOS Attacks The architecture and protocols SHOULD be secure against DOS attacks directed at the routers. The new architecture and protocols SHOULD provide as much information as it can to allow administrators to track down sources of DOS and DDOS attacks. No Bad Data Any new architecture and protocols must provide protection against the introduction of bad, bogus, or misleading, data by attackers. Of particular importance, an attacker must not be able to redirect traffic flows, with the intent of Kastenholz Informational -- Expires 10/2002 12 Inter-domain Routing Requirements March 2002 o Directing legitimate traffic away from a target, causing a denial-of-service attack by preventing legitimate data from reaching its destination, o Directing additional traffic (going to other destinations which are 'innocent bystanders') to a target, causing the target to be overloaded, or o Directing traffic addressed to the target to a place where the attacker can copy, snoop, alter, or otherwise affect the traffic. Topology Hiding Any new architecture and protocols must provide mechanisms to allow network owners to hide the details of their internal topologies, yet maintaining the desired levels of service connectivity and reachability. Privacy By "privacy" we mean privacy of the routing protocol exchanges between routers. In the past this has not been considered important for routing protocols. When the routers are on point-to-point links, with routers at each end, there is no need to encrypt the routing protocol traffic; there is no possibility of a third party intercepting the traffic, and if one of the two routers are compromised then it doesn't matter. This is not sufficient. We believe that it is important to have the ability to protect routing protocol traffic in two cases: 1. When the routers are on a shared network it is possible that there are hosts on the network that have been compromised. These hosts could surreptitiously monitor the protocol traffic. 2. When two routers are exchanging information "at a distance" (over intervening routers and, possibly, administrative domains). In this case, the security of the intervening routers, links, and so on, cannot be assured. Thus, the ability to encrypt this traffic is important. Therefore, we believe that the option to encrypt routing protocol traffic is required. Data Consistency A router should be able to detect and recover from any data that is received from other routers which is inconsistent. That is, it MUST NOT be possible for data from multiple routers, none of which is malicious, to "break" another router. Where security mechanisms are provided, they must use methods that are considered to be cryptographically secure (e.g. using cryptographically strong encryption and signatures -- no clear text passwords!). Kastenholz Informational -- Expires 10/2002 13 Inter-domain Routing Requirements March 2002 Use of security features SHOULD NOT be optional (except as required above). This may be "social engineering" on our part, but we believe it to be necessary. If a security feature is optional, the implementation of the feature MUST default to the "secure" setting. 3.10 End Host Security The Architecture MUST NOT prevent individual host-to-host communications sessions from being secured (i.e. it cannot interfere with things like IPSEC). 3.11 Rich Policy Before setting out Policy requirements, we need to define the term. Like "security", "policy" means many things to many people. For our purposes, we define policy as Policy is the set of administrative influences that alter the path determination and next-hop selection procedures of the routing software. The main motivators for influencing path and next-hop selection seem to be transit rules, business decisions and load management. The new Architecture must support rich policy mechanisms. Furthermore, the policy definition and dissemination mechanismsshould be separated from the network topology and connectivity dissemination mechanisms. Policy provides input to and controls the generation of the forwarding table and the abstraction, filtering, aggregation, and dissemination of topology information. Note that if the architecture is properly divided into subsystems then at a later time, new policy subsystems that include new features and capabilities could be developed and installed as needed. We divide the general area of policy into two sub-categories, routing information and traffic control. Routing Information Policies control what routing information is disseminated or accepted, how it is disseminated, and how routers determine paths and next-hops from the received information. Traffic Control Policies determine how traffic is classified and assigned to routes. 3.11.1 Routing Information Policies There must be mechanisms to allow network administrators, operators, and designers to control receipt and dissemination Kastenholz Informational -- Expires 10/2002 14 Inter-domain Routing Requirements March 2002 of routing information. These controls include, but are not limited to: - Selecting to which others routing information will be transmitted. - Specifying the "granularity" and type of transmitted information. The length of IPv4 prefixes is an example of "granularity". - Selection and filtering of topology and service information that is transmitted. This gives different 'views' of internal structure and topology to different peers. - Selecting the level of security and authenticity for transmitted information - Being able to cause the level of detail that is visible for some portion of the network to reduce the farther you get from that part of the network. - Selecting from whom routing information will be accepted. This control should be "provisional" in the sense of "accept routes from "foo" only if there are no others available". - Accepting or rejecting routing information based on the path the information traveled (using the current system as an example, this would be filtering routes based on an AS appearing anywhere in the AS path). This control should be "provisional" in the sense of "accept routes that traverse "foo" only if there are no others available". - Selecting the desired level of "granularity" for received routing information (this would include, but is not limited to, things similar in nature to the prefix-length filters widely used in the current routing and addressing system). - Selecting the level of security and authenticity of received information in order for that information to be accepted. - Determining the treatment of received routing information based on attributes supplied with the information. - Applying attributes to routing information that is to be transmitted and then determining treatment of information (eg, sending it "here" but not "there") based on those tags. - Selection and filtering of topology and service information that is received. 3.11.2 Traffic Control Policies The architecture SHOULD provide mechanisms that allow network operators to manage and control the flow of traffic. The traffic controls should include, but are not limited to: - The ability to detect and eliminate congestion points in the network (by re-directing traffic around those points) . - The ability to develop multiple paths through the network with different attributes and then assign traffic to those paths based on some discriminators within the packets (discriminators include, but are not limited to, IP Addresses or prefixes, DSCP values, and MPLS labels) . Kastenholz Informational -- Expires 10/2002 15 Inter-domain Routing Requirements March 2002 - The ability to to find and use multiple, equivalent, paths through the network (i.e. they would have the "same" attributes) and allocate traffic across the paths. - The ability to accept or refuse traffic based on some traffic classification (providing, in effect, transit policies). Traffic classification must at least include the source and destination IP addresses (prefixes) and the DSCP value. Other fields may be supported, such as o Protocol and port based functions, o TOS/QOS tuple (such as ports) o Per-host operations (i.e. /32s for IPv4 and /128s for IPv6), o Traffic matrices (e.g., traffic from prefix X and to prefix Y). 3.12 Incremental Deployment The reality of the Internet is that there can be no Internet- wide cutover from one architecture and protocol to another. This means that any new architecture and protocol MUST be incrementally deployable; ISPs must be able to set up small sections of the new architecture, check it out, and then slowly grow the sections. Eventually, these sections will "touch" and "squeeze out" the old architecture. The protocols that implement the Architecture MUST be able to interoperate at "production levels" with currently existing routing protocols. Furthermore, the protocol specifications MUST define how the interoperability is done. We also believe that sections of the Internet will never convert over to the new architecture. Thus, it is important that the new architecture and its protocols be able to interoperate with "old architecture" regions of the network indefinitely. The architecture's addressing system MUST NOT force existing address allocations to be redone: no renumbering! 3.13 Mobility There are two kinds of mobility; host mobility and network mobility. Host mobility is when an individual host moves from where it was to where it is. Network mobility is when an entire network (or subnetwork) moves. The architecture MUST support network level mobility. Kastenholz Informational -- Expires 10/2002 16 Inter-domain Routing Requirements March 2002 3.14 Address Portability One of the big "hot items" in the current Internet political climate is portability of IP addresses (both V4 and V6). The short explanation is that people do not like to renumber and do not trust automated renumbering tools. The Architecture MUST provide complete address portability. 3.15 Multi-Protocol The Internet is expected to be "multi-protocol" for at least the next several years. IPv4 and IPv6 will co-exist in many different ways during a transition period. The architecture MUST be able to handle both IPv4 and IPv6 addresses. Furthermore, protocols that supplant IPv4 and IPv6 may be developed and deployed during the lifetime of the architecture. The architecture MUST be flexible and extensible enough to handle new protocols as they arise. Furthermore, the architecture MUST NOT assume any given relationships between a topological element's IPv4 address and its IPv6 address. The architecture MUST NOT assume that all topological elements have IPv4 addresses/prefixes, nor can it assume that they have IPv6 addresses/prefixes. The architecture SHOULD allow different paths to the same destination to be used for different protocols, even if all paths can carry all protocols. In addition to the addressing technology, the architecture need not be restricted to only packet based multiplexing/demultiplexing technology (such as IP); support for other multiplexing/ demultiplexing technologies MAY be added. 3.16 Abstraction The architecture must provide mechanisms to for network designers and operators to o Group elements together for administrative control purposes, o Hide the internal structure and topology of those groupings for administrative and security reasons, o Limit the amount of topology information that is exported from the groupings in order to control the load placed on external routers, o Define rules for traffic transiting or terminating in the grouping. The architecture MUST allow the current Autonomous System structure to be mapped into any new abstraction schemes. Kastenholz Informational -- Expires 10/2002 17 Inter-domain Routing Requirements March 2002 Mapping mechanisms, algorithms, and techniques MUST be specified. 3.17 Simplicity The architecture MUST be simple enough so that Radia Perlman can explain all the important concepts in less than an hour. The requirement is that the routing architecture be kept as simple as possible. This requires careful evaluation of possible features and functions with a merciless weeding out of those that "might be nice". By keeping the architecture simple, the protocols and software used to implement the architecture are simpler. This simplicity in turn leads to: 1. Faster implementation of the protocols. If there are fewer bells and whistles, then there are fewer things that need to be implemented. 2. More reliable implementations. With fewer components, there is less code, reducing bug counts, and fewer interactions between components that could lead to unforeseen and incorrect behavior. 3.18 Robustness The architecture, and the protocols implementing it, should be robust. Robustness comes in many different flavors. Some considerations with regard to robustness include (but are not limited to): o Defective (even malicious) trusted routers. o Network failures. Whenever possible, valid alternate paths are to be found and used. o Failures must be localized. That is, the architecture must limit the "spread" of any adverse effects of a misconfiguration or failure. Badness must not spread. Of course, the general robustness principle of being liberal in what's accepted and conservative in what's sent must also be applied. EDITOR'S NOTE: Some of the contributors to this note have argued that robustness is an aspect of Security. I have exercised editor's discretion by making it a separate section. The reason for this is that to too many people "security" means "protection from breakins" and "authenticating and encrypting data". This requirement goes beyond those views. Kastenholz Informational -- Expires 10/2002 18 Inter-domain Routing Requirements March 2002 3.19 Media Independence While it is an article of faith that IP operates over a wide variety of media (such as Ethernet, X.25, ATM, and so on), IP routing must take an agnostic view towards any "routing" or "topology" services that are offered by the medium over which IP is operating. That is, the new architecture MUST NOT be designed to integrate with any media-specific topology management or routing scheme. The routing architecture must assume, and must work over, the simplest possible media. The routing and addressing architecture can certainly make use of lower-layer information and services, when and where available, and to the extent that IP routing wishes. 3.20 Stand-alone The routing architecture and protocols MUST NOT rely on other components of the Internet (such as DNS) for their correct operation. Routing is the fundamental process by which data "finds its way around the Internet" and most, if not all, of those other components rely on routing to properly forward their data. Thus, Routing cannot rely on any Internet systems, services or capabilities that in turn rely on Routing. If it did, a dependency loop would result. 3.21 Safety of Configuration The architecture, protocols, and standard implementation defaults must be such that a router installed "out of the box" with no configuration/etc by the operators will not cause "bad things" to happen to the rest of the routing system (no dialup customers advertising routes to 18/8!) 3.22 Renumbering The routing system MUST allow topological entities to be renumbered. 3.23 Multi-prefix The architecture MUST allow topological entities to have multiple prefixes (or the equivalent under the new architecture). 3.24 Cooperative Anarchy As RFC1726[5] said: Kastenholz Informational -- Expires 10/2002 19 Inter-domain Routing Requirements March 2002 A major contributor to the Internet's success is the fact that there is no single, centralized, point of control or promulgator of policy for the entire network. This allows individual constituents of the network to tailor their own networks, environments, and policies to suit their own needs. The individual constituents must cooperate only to the degree necessary to ensure that they interoperate. This decentralization, called "cooperative anarchy", is still a key feature of the Internet today. The new routing architecture MUST retain this feature. There can be no centralized point of control or promulgator of policy for the entire Internet. 3.25 Network Layer Protocols and Forwarding Model For the purposes of backward compatibility, any new routing and addressing architecture and protocols MUST work with IPv4 and IPv6 using the traditional "hop by hop" forwarding and packet- based multiplex/demultiplex models. However, the architecture need not be restricted to these models. Additional forwarding and multiplex/demultiplex models MAY be added. 3.26 Routing Algorithm The architecture SHOULD NOT require a particular routing algorithm family. That is to say, the architecture should be agnostic with regard to using link-state, distance-vector, or path-vector routing algorithms. 3.27 Positive Benefit Finally, the architecture must show benefits, in terms of increased stability, decreased operational costs, and increased functionality and lifetime over the current schemes. This benefit must remain even after the inevitable costs of developing and debugging the new protocols, enduring the inevitable instabilities as things get shaken out, and so on. 3.28 Administrative Entities and the IGP/EGP Split We explicitly recognize that the Internet consists of resources under control of multiple administrative entities. Each entity MUST be able to manage its own portion of the Internet as it sees fit. Moreover, the constraints that can be imposed on routing and addressing on the portion of the Internet under the control of one administration may not be feasibly extended to cover multiple administrations. Therefore, we recognize a natural and inevitable split between routing and addressing that is under a single administrative control and routing and addressing that involves multiple administrative entities. Moreover, while there may be multiple administrative Kastenholz Informational -- Expires 10/2002 20 Inter-domain Routing Requirements March 2002 authorities, the administrative authority boundaries may be complex and overlapping, rather than being a strict hierarchy. Furthermore, there may be multiple levels of administration, each with its own level of policy and control. For example, a large network might have "continental-level" administrations covering its European and Asian operations, respectively. There would also be that network's "inter-continental" administration covering the Europe-to-Asia links. Finally, there would be the "Internet" level in the administrative structure (analogous to the "exterior" concept in the current routing architecture). Thus, we believe that the administrative structure of the Internet must be extensible to many levels (more than the two provided by the current IGP/EGP split). The interior/exterior property is not absolute. The interior/exterior property of any point in the network is relative; a point on the network is interior with respect to some points on the network and exterior with respect to others. Administrative entities may not trust each other; some may be almost actively hostile towards each other. The architecture MUST accommodate these models. Furthermore, the architecture MUST NOT require any particular level of trust among administrative entities. 4 Non-Requirements The following are not required or are non-goals. This should not be taken to mean that these issues must not be addressed by a new architecture. Rather, addressing these issues or not is purely a matter for the architects. 4.1 Forwarding Table Optimization We believe that it is not necessary for the architecture to minimize the size of the forwarding tables (FIBS). Current memory sizes, speeds, and prices, along with processor and ASIC capabilities allow forwarding tables to be very large, O(E6), and fast (100M lookups/second) tables to be built with little difficulty. 4.2 Traffic Engineering Traffic Engineering is one of those terms that has become terribly overloaded. If you ask N people what traffic engineering is, you get something like N! disjoint answers. Therefore, we elect not to require "traffic engineering", per se. Instead, we have endeavored to determine what the ultimate Kastenholz Informational -- Expires 10/2002 21 Inter-domain Routing Requirements March 2002 intent is when operators "traffic engineer" their networks and then make those capabilities an inherent part of the system. 4.3 Multicast The new architecture is not designed explicitly to be an inter- domain multicast routing architecture. However, given the notable lack of a viable, robust, and widely deployed inter- domain multicast routing architecture, the architecture should not hinder the development and deployment of inter-domain multicast routing without adverse effect on meeting the other requirements. We do note however that one respected network sage has said (roughly) When you see a bunch of engineers standing around congratulating themselves for solving some particularly ugly problem in networking, go up to them, whisper "multicast", jump back, and watch the fun begin... 4.4 QOS The Architecture concerns itself primarily with disseminating network topology information so that routers may select paths to destinations and build appropriate forwarding tables. QOS is not a part of this function and we make no requirements with respect to QOS. However, QOS is an area of great and evolving interest. It is reasonable to expect that in the not too distant future, sophisticated QOS facilities will be deployed in the Internet. Any new architecture and protocols should be developed with an eye towards these future evolutions. Extensibility mechanisms, allowing future QOS routing and signaling protocols to "piggy- back" on top of the basic routing system are desired. We do require the ability to assign attributes to entities and then do path generation and selection based on those attributes. Some may call this QOS. 4.5 IP Prefix Aggregation There is no specific requirement that CIDR-style IP Prefix aggregation be done by the new architecture. Address allocation policies, societal pressure, and the random growth and structure of the Internet have all conspired to make prefix aggregation extraordinarily difficult, if not impossible. This means that large numbers of prefixes will be sloshing about in the routing system and that forwarding tables will grow quite big. This is a cost that we believe must be borne. Kastenholz Informational -- Expires 10/2002 22 Inter-domain Routing Requirements March 2002 Nothing in this non-requirement should be interpreted as saying that prefix aggregation is explicitly prohibited. CIDR-style IP Prefix aggregation might be used as a mechanism to meet other requirements, such as scaling. 4.6 Perfect Safety Making the system impossible to misconfigure is, we believe, not required. The checking, constraints, and controls necessary to achieve this could, we believe, prevent operators from performing necessary tasks in the face of unforeseen circumstances. However, safety is always a "good thing", and any results from research in this area should certainly be taken into consideration and, where practical, incorporated into the new routing architecture. 4.7 Dynamic Load Balancing Past history has shown that using the routing system to perform highly dynamic load balancing among multiple more-or-less-equal paths usually ends up causing all kinds of instability, etc, in the network. Thus, we do not require such a capability. However, this is an area that is ripe for additional research, and some believe that the capability will be necessary in the future. Thus, the architecture and protocols should be "malleable" enough to allow development and deployment of dynamic load balancing capabilities, should we ever figure out how to do it. 4.8 Renumbering of hosts and routers We believe that the routing system is not required to "do renumbering" of hosts and routers. That's an IP issue. Of course, the routing and addressing architecture must be able to deal with renumbering when it happens. 4.9 Host Mobility In the Internet Architecture, host-mobility is handled on a per-host basis by a dedicated, Mobile-IP protocol [6]. Traffic destined for a mobile-host is explicitly forwarded by dedicated relay agents. Mobile-IP [6] adequately solves the host- mobility problem and we do not see a need for any additional requirements in this area. Of course, the new architecture MUST NOT impede or conflict with Mobile-IP. Kastenholz Informational -- Expires 10/2002 23 Inter-domain Routing Requirements March 2002 4.10 Clean Slate For the purposes of development of the architecture, we assume that there is a 'clean slate'. Unless specified in section 3, we have no explicit requirements that elements, concepts or mechanisms of the current routing architecture are carried forward into the new one. 5 Discussion of other Inter-Domain Routing Requirements documents Recently, two other Internet Drafts have been published that set out some requirements for a new Inter-Domain routing architecture. These drafts are "Future Domain Routing Requirements" by Elwyn Davies, et al [3], and "Architectural Requirements for Inter-Domain Routing in the Internet" by Geoff Huston [4]. Rather than use these documents, we decided to develop our own set of requirements for a future inter-domain routing architecture. The reasons are 1. There are requirements in [3] and [4] with which we do not agree, plus we have additional requirements that are not in those documents. 2. We do not agree with the methodology in [3] and [4]. In particular, [3]'s list of requirements seems to us like an exhaustive list of things, falling dangerously close to the "My Layer MUST solve all of networking's problems" disease. One of the hardest parts of setting requirements is to determine what problems will NOT be solved. [3] and [4] each provide excellent historical background information. Rather than repeat the information here, readers are encouraged to consult those documents. Discussions and commentary on each internet-draft follow. 5.1 Comments on "Architectural Requirements for Inter-Domain Routing in the Internet" The IAB have developed an internet-draft titled "Architectural Requirements for Inter-Domain Routing in the Internet" [4]. Rather than using [4], we have produced our own document. A good part of [4] is a detailed explanation of the current problems in routing and addressing. This work is valuable. We chose not to replicate it in this note. The main reason we decided not to use [4] is that it is too rooted in the current BGP/AS model. We believe that the Kastenholz Informational -- Expires 10/2002 24 Inter-domain Routing Requirements March 2002 charter for the Routing Research Group is to work on longer- term futures, including the possibility of replacing that model. We wish to develop our requirements in a forward- looking manner and using requirements rooted in the old model may cause us to unwittingly constrain our thinking. [4] Enumerates four requirements for a new routing architecture. These requirements are fairly broad and loose. They are: Scalability We agree. The new architecture must be scalable. We have included a requirement for scalability. Stability and Predictability Yes, the routing and addressing architecture should be stable and predictable. However, we are not sure that overall this is possible. Convergence Fast convergence would be nice. However, the size and complexity of the Internet are such that we do not believe that the entire net can ever converge. Changes with global impact will always be happening and their effects will be constantly propagating through the network. We expect that any routing system will, at best, be able to converge incrementally. That is, in any one place, convergence with respect to one change can occur, but before the entire net converges, another change will occur, requiring a new set of calculations and a "new" convergence. We prefer to believe that convergence at any single point and after any single change must occur quickly. Also, since we expect that the network will constantly by (re- converging, so the load of those attempts must be minimal. Routing Overhead Routing overhead should be minimized. True. 5.2 Comments on "Future Domain Routing Requirements" A separate group is developing a document titled "Future Domain Routing Requirements" [3]. Rather than using [3], we have produced our own document. As with [4] we believe that this document is too rooted in the current BGP/AS model, which is not a useful context for developing long-term future architectures. We have a number of observations on [3], in no particular order: Kastenholz Informational -- Expires 10/2002 25 Inter-domain Routing Requirements March 2002 1. The introductory/historical/background is fine. In fact, that material should be considered "required reading" to understand the background and current problems of Internet Routing. We commend the authors of [3] for putting this material together. 2. The "goals" section (1.2) seems to be heavily oriented to specific issues and tasks that network designers and administrators face today. Our charter calls for us to consider the very long term. Thus, it is hard to say what specific tasks the operations community needs to perform. Therefore, we prefer to take a broader view of what routing should do and have written our document accordingly. 3. [3] Seems still wedded to the old BGP/AS model of doing inter-domain routing. That may well be an adequate model. However, the Routing Research Group's charter is to consider and develop long-term future directions in routing. We prefer to develop our own document, trying to avoid falling into the "traps" of the past. 4. The document seems to be an exhaustive wish list of things. The hardest part of doing requirements is to figure out what not to require. Coincident with that observation, there were a couple of "would be nice" and "might be needed" sorts of things. The fear is that they fell into the trap of "All problems must be solved in our layer", which leads to very poor architectures and designs. 5. They call for "support for NATs and other mid-boxes". If the Routing and Addressing architecture is "right" then there is no need for them, at least as far as Routing and Addressing are concerned. Also, we are confused as to what "support for NATs..." actually means. 6. The authors of [3] talk a fair bit about some low-level things they want. Our opinion is that we are looking "too far out" to talk about detailed, low-level requirements. We don't know what the operators of 2007 will want. Besides solving the basic problem of getting topological and addressing information "around", we need to think more about how to keep the architecture (and design and implementation) flexible and open so that in 2006 when someone says "We need a gonkulator!" we can say "Easy, it gets plugged in here..." 7. Multicast/Anycast We do not believe that multicast routing is a part of our charter. Kastenholz Informational -- Expires 10/2002 26 Inter-domain Routing Requirements March 2002 Anycast is a fairly nebulous technology. To require that routing "support" it at this stage in its development is premature. To require that routing support anycast means, in effect, that we first define what it is. 8. Support for renumbering We take this to mean either that o The routing-and-addressing system actually does the renumbering, to which we reply "Is this really something for routing to do?", or o The routing-and-addressing system must continue to operate when elements of the network are renumbered. We have a requirement for this 9. Statistics support This seems to us to be a low-level protocol issue. Yes the protocol needs a MIB. But we do not believe that this is an architectural or requirements issue. This also indicates to us that [3] is more concerned with lower-level protocol and implementation issues rather than taking the proverbial "step back" to look at "the big picture". 10. There were some comments about convergence that seemed to indicate that the authors of [3] are thinking of global convergence. We have observed that global convergence is probably not doable anymore. We believe that "permanent change" is the order of the day... 11. One of their open-issues indicates that the authors of [3] are thinking about whether the EGP/IGP split is good or not. It's good they are thinking about it. It's bad that they consider it open. We consider this split to be A Bad Thing -- there should be no such split in the architecture. 12. One example of the too fine a level of detail they are getting into is that in 7.2.2 they talk about having a number of specific path attributes. But, if say in 2007 we discover a need for a gonkulation attribute, it's not in the list in [3]. Perhaps we are taking their document too literally, but someone taking [3] and designing to it would maybe make attributes TLV's and leave it at that. They wouldn't consider the effects of unconsidered new attributes. We prefer to think of things in the reverse order: We know we need attributes, but we don't know what they all are. Therefore the attribute mechanism must be flexible and allow growth in weird new directions without causing problems on the rest of the system -- so how do we do that. Kastenholz Informational -- Expires 10/2002 27 Inter-domain Routing Requirements March 2002 6 Security Considerations We address security issues in the individual requirements. We do require that the Architecture and protocols developed against this set of requirements be "secure". 7 IANA Considerations This document is a set of requirements from which a new routing and addressing architecture will be developed. From that architecture, a new protocol, or set of protocols, may be developed. While this note poses no new tasks for IANA, the architecture and protocols developed from this document probably will have issues to be dealt with by IANA. 8 References [1] Bradner, S., "The Internet Standards Process û Revision 3", BCP9, RFC2026, October 1996. [2] Bradner, S., "Key words for use in RFCs to Indicate Requirements Levels", BCP 14 RFC 2119, March 1997. [3] Davies, E., et al, "Future Domain Routing Requirements", draft-davies-fdr-reqs-01.txt, July 2001, Work In Progress. [4] Huston, G., "Architectural Requirements for Inter-Domain Routing in the Internet" draft-iab-bgparch-01.txt, May 2001, Work In Progress. [5] Partridge, C., and F. Kastenholz, "Technical Criteria for Choosing IP The Next Generation (IPng)", RFC 1726. December 1994. [6] Perkins, C., "IP Mobility Support." RFC2002, October 1996. 9 Acknowledgments This originated in the IRTF Routing Research GroupÆs sub-group on Inter-domain routing requirements. The members of the group are Abha Ahuja Danny McPherson J. Noel Chiappa David Meyer Sean Doran Mike OÆDell JJ Garcia-Luna-Aceves Andrew Partan Susan Hares Radia Perlman Kastenholz Informational -- Expires 10/2002 28 Inter-domain Routing Requirements March 2002 Geoff Huston Yakov Rehkter Frank Kastenholz John Scudder Dave Katz Curtis Villamizar Tony Li Dave Ward We also appreciate the comments and review received from Ran Atkinson, Howard Berkowitz, Randy Bush, Avri Doria, Jeffery Haas, Dmitri Krioukov, Russ White, and Alex Zinin. Special thanks to Yakov Rehkter for contributing text and to Noel Chiappa. 10 Editor's Addresses Frank Kastenholz Unisphere Networks 10 Technology Park Westford, MA, 01886, USA Phone: +1 978 589 0286 Email: fkastenholz@unispherenetworks.com Kastenholz Informational -- Expires 10/2002 29 Inter-domain Routing Requirements March 2002 Full Copyright Statement Copyright (C) The Internet Society (2001). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Kastenholz Informational -- Expires 10/2002 30