TRILL WG J. Touch (ed.) Internet Draft USC/ISI Expires: May 2006 November 17, 2005 Transparent Interconnection of Lots of Links (TRILL): Problem and Applicability Statement draft-touch-trill-prob-00.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on May 17, 2006. Copyright Notice Copyright (C) The Internet Society (2005). All Rights Reserved. Abstract Current Ethernet (802.1) link layers use custom routing protocols that have a number of challenges. The routing protocols need to strictly avoid loops, even temporary loops during route propagation, because of the lack of header loop detection support. Routing tends not to take full advantage of alternate paths, or even non- Touch Expires May 17, 2006 [Page 1] Internet-Draft TRILL: Problem and Applicability November 2005 overlapping pairwise paths (in the case of spanning trees). The convergence of these routing protocols and stability under link changes and failures is also of concern. This document addresses these concerns and suggests that they are related to the need to be able to apply network layer routing (e.g., link state or distance vector) protocols at the link layer. This document assumes that solutions would not address issues of scalability beyond that of existing bridged (802.1D) links, but that a solution would be backward compatible with 802.1D, including hubs, bridges, and their existing plug-and-play capabilities. This document is a work in progress; we invite you to participate on the mailing list at http://www.postel.org/rbridge Table of Contents 1. Introduction...................................................3 2. The TRILL Problem..............................................4 2.1. Inefficient Paths.........................................4 2.2. Convergence Under Reconfiguration.........................5 2.3. Robustness to Link Interruption...........................6 2.4. Problems Not Addressed....................................6 3. Desired Properties of Solutions to TRILL.......................7 3.1. No Change to Link Capabilities............................7 3.2. Zero Configuration and Zero Assumption....................8 3.3. Forwarding Loop Mitigation................................8 3.4. Spanning Tree Management..................................9 3.5. Multiple Attachments......................................9 3.6. VLAN Issues...............................................9 3.7. Equivalence...............................................9 3.8. Optimizations............................................10 3.9. Internet Architecture Issues.............................10 4. Applicability.................................................11 5. Security Considerations.......................................12 6. IANA Considerations...........................................12 7. Conclusions...................................................12 8. Acknowledgments...............................................12 8.1. Normative References.....................................12 8.2. Informative References...................................13 Author's Addresses...............................................14 Intellectual Property Statement..................................14 Disclaimer of Validity...........................................14 Copyright Statement..............................................15 Acknowledgment...................................................15 Touch Expires May 17, 2006 [Page 2] Internet-Draft TRILL: Problem and Applicability November 2005 1. Introduction [CAVEAT: this document is in very rough form. Input and feedback is solicited NOTE: the terms 'campus' and 'rbridge' intentionally do not appear in this document] Conventional Ethernet link subnets have a number of attractive features, allowing hosts and routers to relocate within the subnet without requiring renumbering and are automatically configuring. Unfortunately, the basis of the simplicity of these subnets is the spanning tree, which although simple and elegant, can have substantial limitations. In subnets with high host reattachment frequency, convergence of the spanning tree protocol can be slow. Because all traffic flows over a single tree, all traffic is concentrated on a subset of links, increasing susceptibility to the effects of link failures and limiting the bandwidth across the subnet. [I use the term Ethernet link subnets; do I need to define this? It's not a segment, which I think of as being shared or hubbed but not bridged] The alternative to an Ethernet link subnet is often a network subnet. Network subnets can use link-state routing protocols that allow traffic to traverse least-cost paths rather than being aggregated on a spanning tree backbone, providing higher aggregate capacity and more resistance to link failures. Unfortunately, IP - the dominant network layer technology - requires that hosts be renumbered when relocated in different network subnets, interrupting network (e.g., tunnels, IPsec) and transport (e.g., TCP, UDP) associations that are in progress during the transition. It is thus useful to consider a new approach that combines the features of these two existing solutions, hopefully retaining the desirable properties of each. Such an approach would develop a new kind of bridge system that was capable of using network-style routing, while still operating at the link layer. This allows reuse of well-understood network routing protocols to benefit the link layer. This document describes the challenge of such a combined approach in detail. This problem is known as "Transparent Interconnection of Lots of Links" or "TRILL". The remainder of this document makes as few assumptions about a solution to TRILL as possible. Touch Expires May 17, 2006 [Page 3] Internet-Draft TRILL: Problem and Applicability November 2005 2. The TRILL Problem Ethernet subnets have evolved from 'thicknet' to 'thinnet' to twisted pair with hubs to twisted pair with switches, becoming increasingly simple to wire and manage. Each level has corresponding topology restrictions; thicknet is inherently linear, whereas thinnet and hub- connected twisted pair have to be wired as a tree. Switches allow network managers to avoid thinking in trees, where the spanning tree protocol finds a valid tree automatically; unfortunately, this additional simplicity comes with a number of associated penalties. The spanning tree often results in inefficient use of the link topology; traffic is concentrated on the spanning tree path, and all traffic follows that path even when other more direct paths may be available. The spanning tree configuration is affected by even small topology changes, and small changes can have large effects. Each of these inefficiencies can cause problems for current link layer deployments. 2.1. Inefficient Paths The Spanning Tree Protocol (STP) helps break cycles in a set of interconnected bridges, but it also can limit the bandwidth among that set and cause traffic to take circuitous paths. Consider the network shown in Figure 1, which shows a number of bridges and their interconnecting links. End hosts and routers are not shown; they would connect to the bridges that are shown, labeled A-H. Note that the network shown has cycles which would cause packet storms if hubs (repeaters) were used instead of STP-capable bridges. One possible spanning tree is shown by double lines. B=======\\ C // \ \\ / \\ D // \ \\/ \\ // A-----\-------H===== E \ // || \ // || \ // || G----------F Figure 1 Bridged subnet with spanning tree shown The spanning tree limits the capacity of the resulting subnet. Assume that the links are 100 Mbps. Figure 2 shows how traffic from hosts on A to hosts on C goes via the spanning tree path A-B-H-E-C (links replaced with '1' in the figure); traffic from hosts on G to F go via Touch Expires May 17, 2006 [Page 4] Internet-Draft TRILL: Problem and Applicability November 2005 the spanning three path G-H-E-F (links replaced by '2' in the figure). The link H-E is shared by both paths (alternating '1's and '2's), resulting in an aggregate capacity for both A..C and G..F paths of a total of 100 Mbps. B11111111 C 1 1 1 1 1 1 A H121212E 2 2 2 2 2 2 G F Figure 2 Traffic from A..C (1) and G..F (2) share a link If traffic from G to F were to go directly using full routing, e.g., from G-F, both paths could have 100 Mbps each, and the total aggregate capacity could be 200 Mbps (Figure 3). In this case, the H- F link carries only A-C traffic ('1's) and the G-F traffic ('2's) is more direct. B11111111 C 1 1 1 D 1 1 1 A H111111E G2222222222F Figure 3 Traffic from A..C (1) and G..F (2) with full routing There are a number of features of modern layer 3 routing protocols which would be beneficial if available at layer 2, but which cannot be integrated into the spanning tree system. Multipath routing can distribute load simultaneously among two different paths; alternate path routing supports rapid failover to backup paths. Layer 3 routing typically optimizes paths between pairs of endpoints, conventionally based on hopcount but also including bandwidth, latency, or other policy metrics. 2.2. Convergence Under Reconfiguration The spanning tree is dependent on the way a set of bridges are interconnected, i.e., the link layer topology. Small changes in this Touch Expires May 17, 2006 [Page 5] Internet-Draft TRILL: Problem and Applicability November 2005 topology can cause large changes in the spanning tree. Changes in the spanning tree can take time to propagate and converge. [QUESTION: is there a good visual example of this, one that we can walk through in the description?] [QUESTION: What is the timescale? O(# bridges)? O(#links?), etc?] [QUESTION: is port autolearning in this category too? i.e., are TRILL solutions trying to hide port reattachment from other nodes (or is that even necessary?)] The spanning tree protocol is inherently global to an entire layer 2 subnet; there is no current way to contain, partition, or otherwise factor the protocol into a number of smaller, more stable subsets that interact as groups. Contrast this with Internet routing, which includes both intradomain and interdomain variants, split to provide exactly that containment and scalability within a domain while allowing domains to interact freely independent of what happens inside. [QUESTION: anybody have a convenient reference for a proof? Are new spanning tree protocols not considering AS-like boundaries? (just checking)] 2.3. Robustness to Link Interruption Persistent changes to the link topology, as described in Section 2.2, are not the only effects on subnet stability. Transient link interruptions have similar effects, with similar scalability issues. It would be more useful for subnet configuration to be tolerant of such transients, e.g., supporting alternate, backup paths. [QUESTION: is there more to say here?] Contrast this to network layer intradomain and interdomain routing, both of which include provisions for backup paths. These backups allow routing to be more stable in the presence of transients, as well as to recover more rapidly when the transient disappears. 2.4. Problems Not Addressed There are other challenges to 802.1D link layer subnets that are not addressed in this document. These include: [NOTE: these are guesses; if any are wrong, we can move or revise] Touch Expires May 17, 2006 [Page 6] Internet-Draft TRILL: Problem and Applicability November 2005 o increased Ethernet link subnet scale o increased node relocation o Ethernet link subnet management protocol security o flooding attacks on a Ethernet link subnet Solutions to TRILL are not intended to support increasingly larger scales of Ethernet link subnets than current spanning tree protocols can support. This may be a side-effect of supporting more rapid reaction to subnet reconfiguration or multiple internal paths, or of the grouped modularity that network style routing affords, but is not the focus of this work. Similarly, solutions to TRILL are not intended to address link layer node migration, which can complicate the caches in learning bridges. Similar challenges exist in the ARP protocol, where link layer forwarding is not updated appropriately when nodes move to ports on other bridges. Again, the grouped modularity of network routing, like that of network layer ASes, can help hide the effect of migration. That is a side effect, however, and not a primary focus of this work. Current link control plane protocols, including Ethernet link subnet management (STP) and link/network integration (ARP), are vulnerable to a variety of attacks. Solutions to TRILL are not intended to directly address these vulnerabilities. Similar attacks exist in the data plane, e.g., source address spoofing, single address traffic attacks, traffic snooping, and broadcast flooding. TRILL solutions do not address any of these issues, although it is critical that they do not introduce new vulnerabilities in the process (see Section 5). 3. Desired Properties of Solutions to TRILL This section describes some of the desirable or required properties of any system that would solve the TRILL problems, independent of the details of such an architecture. Most of these are based on retaining useful properties of bridges, or maintaining those properties while solving the problems listed in Section 2. 3.1. No Change to Link Capabilities There must be no change to the service that Ethernet subnets already provide as a result of deploying a TRILL solution. Ethernet supports unicast, broadcast, and multicast natively. Although network protocols, notably IP, can tolerate link layers that do not provide all three, it would be useful to retain the support already in place Touch Expires May 17, 2006 [Page 7] Internet-Draft TRILL: Problem and Applicability November 2005 [6]. Zeroconf, as well as existing bridge autoconfiguration, are dependent on broadcast as well. There are subtle implications to such a requirement. Bridge autolearning already is susceptible to moving nodes between ports, because previously learned associations between port and link address change. A TRILL solution could be similarly susceptible to such changes. 3.2. Zero Configuration and Zero Assumption Both bridges and hubs are zero configuration devices; hubs having no configuration at all, and bridges being automatically self- configured. Bridges are further zero-assumption devices, unlike hubs. Bridges can be interconnected in arbitrary topologies, without regard for cycles or even (inadvertent) self-attachment. STP removes the impact of cycles automatically, and port autolearning reduces unnecessary broadcast of unicast traffic. A TRILL solution should strive to have similar zero configuration, zero assumption operation. This includes having TRILL solution components automatically discover other TRILL solution components and organize themselves, as well as to configure that organization for proper operation (plug-and-play). It also includes zero configuration backward compatibility with existing bridges and hubs, which may include interacting with some of the bridge protocols, such as STP. Autoconfiguration extends to optional services, such as multicast support via IGMP snooping, broadcast support via internal serial copy, and supporting multiple VLANs. 3.3. Forwarding Loop Mitigation Spanning tree avoids forwarding loops by design, even during update (?). Solutions to TRILL are intended to use adapted network layer routing protocols which may introduce transient loops during routing convergence. TRILL solutions thus need support for mitigating the effect of such routing loops. In the Internet, loop avoidance is provided by a decrementing hopcounts (TTL); in other networks, packets include a trace (serialized or unioned) of visited nodes [1]. These mechanisms (respectively) limit the impact of loops or detect them explicitly. A mechanism with similar effect should be included in TRILL solutions. [QUESTION: anyone have a good reference for serialized or union traces - or better names for them?] Touch Expires May 17, 2006 [Page 8] Internet-Draft TRILL: Problem and Applicability November 2005 3.4. Spanning Tree Management In order to address convergence under reconfiguration and robustness to link interruption (Sections 2.2 and 2.3), participation in the STP must be carefully managed. The goal is to provide the desired stability inside the TRILL solution and of the entire Ethernet link subnet while not interfering with the operation of STP outside the TRILL solution. This may involve TRILL solutions participating in the STP, where internally a more stable protocol is run such that the interactions between the TRILL solution and external STP is dampened, or it may involve severing the external STP into separate STPs on 'stub' external Ethernet link subnet segments. [we need pictures here; to appear in the next version] 3.5. Multiple Attachments [QUESTION: I'm not sure what this refers to; is it the same NIC attached at different points to a TRIL solution? If so, why should this be possible where it seems ignored in bridges?] 3.6. VLAN Issues A TRILL solution should support multiple VLANs. This may involve ignorance, just as many bridge devices do not participate in the VLAN protocols. It may alternately support direct VLAN support, e.g., by the use of separate internal routing protocol instances to separate traffic for each VLAN inside a TRILL solution. [QUESTION: is it possible to be ignorant of VLANs and still operate? Bridges are, right?] 3.7. Equivalence As with any extension to an existing architecture, it would be useful - though not strictly necessary - to be able to describe or consider a TRILL solution as a model of an existing link layer component. Such equivalence provides a validation model for the architecture. This provides a sanity check, i.e., "we got it right if we can replace a TRILL solution with an X" (where "X" might be a single bridge, a hub, or some other link layer abstraction). It does not matter whether "X" can be implemented on the same scale as the corresponding TRILL solution. It also does not matter if it can - there may be utility to deploying the TRILL solution components incrementally, in ways that a single "X" could not be installed. Touch Expires May 17, 2006 [Page 9] Internet-Draft TRILL: Problem and Applicability November 2005 For example, if TRILL solution were equivalent to a single bridge, it would mean that the TRILL solution would - as a whole - participate in the STP. This need not require that TRILL solution would propagate STP internally, any more than a bridge need do so in its on-board control. It would mean that the solution would interact with BPDUs at the edge, where the solution would - again, as a whole - participate as if a single node in the spanning tree. Note that this equivalence is not required; a solution may act as if a hub, or may not have a corresponding equivalent link layer component at all. 3.8. Optimizations There are a number of optimizations that may be applied to TRILL solutions. These must be applied in a way that does not affect functionality as a tradeoff for increased performance. Such optimizations address broadcast and multicast frame distribution, VLAN support, and snooping of ARP and IPv6 neighbor discovery. [NOTE: need to say more here.] 3.9. Internet Architecture Issues TRILL solutions are intended to have no impact on the Internet network layer architecture. In particular, the Internet and higher layer headers should remain intact when traversing a TRILL solution, just as they do when traversing any other link subnet technologies. This means that the IP TTL field cannot be co-opted for forwarding loop mitigation, as it would interfere with the Internet layer assuming that the link subnet was reachable with no changes in TTL (Internet TTLs are changed only at routers, as per RFC 1812) [1]. TRILL solutions should also have no impact on Internet routing or signaling, which also means that broadcast and multicast, both of which can pervade an entire Ethernet link subnet, must be able to transparently pervade a TRILL solution. Changing how either of these capabilities behaves would have significant effects on a variety of protocols, including RIP (broadcast), RIPv2 (multicast), ARP (broadcast), IPv6 neighbor discovery (multicast), etc. Note that snooping of network layer packets may be useful, especially for certain optimizations. These include snooping multicast control plane packets (IGMP) to tune link multicast to match the network multicast topology, as is already done in existing smart switches [3]. This also includes snooping IPv6 neighbor discovery messages to assist with governing TRILL solution edge configuration, as is the case in some smart learning bridges [7]. Other layers may similarly be snooped, notably ARP packets, for similar reasons for IPv4 [11]. Touch Expires May 17, 2006 [Page 10] Internet-Draft TRILL: Problem and Applicability November 2005 [Need a ref for the router-router 'igmp' protocol] 4. Applicability As might be expected, TRILL solutions are intended to be used to solve the problems described in Section 2. However, not all such installations are appropriate environments for such solutions. This section outlines the issues in the appropriate use of these solutions. TRILL solutions are intended to address problems of path efficiency and stability within a single Ethernet link subnet. Like bridges, individual TRILL solution components may find other TRILL solution components within a single Ethernet link subnet and aggregate into a single TRILL solution. TRILL solutions are not intended to span separate Ethernet link subnets where interconnected by network layer (e.g., router) devices, except via link layer tunnels that are in place prior to their deployment, where such tunnels render the distinct subnet undetectably equivalent from a single Ethernet link subnet. A currently open question is whether a single Ethernet link subnet should contain only one TRILL solution instance, either of necessity of architecture or utility. Multiple TRILL solutions, like Internet ASes, may allow internal TRILL routing protocols to be partitioned in ways that help their stability, but this may come at the price of needing the TRILL solutions to participate more fully as nodes (each modeling a bridge) in the Ethernet link subnet STP. Each architecture solution should decide whether multiple TRILL solutions are supported within a single Ethernet link subnet and mechanisms should be included to enforce whatever decision is made. TRILL solutions are not intended to address scalability limitations in bridged subnets. Although there may be scale benefits of other aspects of solving TRILL problems, e.g., of using network layer routing to provide stability under link changes or intermittent outages, this is not a focus of this work. As also noted earlier, TRILL solutions are not intended to address security vulnerabilities in either the data plane or control plane of the link layer. This means that TRILL solutions should not limit broadcast frames, ARP requests, or spanning tree protocol messages (if such are interpreted by the TRILL solution or solution edge). Touch Expires May 17, 2006 [Page 11] Internet-Draft TRILL: Problem and Applicability November 2005 5. Security Considerations TRILL solutions should not introduce new vulnerabilities compared to traditional bridged subnets. TRILL solutions are not intended to be a solution to Ethernet link subnet vulnerabilities, including spoofing, flooding, snooping, and attacks on the link control plane (STP, flooding the learning cache) and link-network control plane (ARP). Although TRILL solutions are intended to provide more stable routing than STP, this stability is limited to performance, and the subsequent robustness is intended to address non-malicious events. There may be some side-effects to the use of TRILL solutions that can provide more robust operation under certain attacks, such as those interrupting link service, but TRILL solutions should not be relied upon for such capabilities. Finally, TRILL solutions should not interfere with other protocols intended to address these vulnerabilities, such as those under development to secure IPv6 neighbor discovery. [need a ref for secure ipv6 nd] 6. IANA Considerations This document has no IANA considerations. This section should be removed by the RFC Editor prior to final publication. 7. Conclusions (TBA) 8. Acknowledgments Portions of this document are based on a preliminary solution description document [10]. 8.1. Normative References None. Touch Expires May 17, 2006 [Page 12] Internet-Draft TRILL: Problem and Applicability November 2005 8.2. Informative References [1] Baker, F., "Requirements for IP Version 4 Routers", RFC 1812 (Standards Track), June 1995. [2] Braden, R., "Requirements for Internet Hosts - Communication Layers", STD 3, RFC 1122, October 1989. [3] Cain, B., S. Deering, I. Kouvelas, B. Fenner, A. Thyagarajan, "Internet Group Management Protocol, Version 3," RFC 3376 (Proposed Standard), October 2002. [4] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual environments", RFC 1195, December 1990. [5] IEEE 802.1d bridging standard, "IEEE 802.1d bridging standard". [6] P. Karn (ed.), C. Bormann, G.Fairhurst, D. Grossman, R. Ludwig, J. Mahdavi, G. Montenegro, J. Touch, L. Wood, "Advice for Internet Subnetwork Designers," RFC-3819 / BCP 89, July 2004. [7] Narten, T., Nordmark, E. and W. Simpson, "Neighbor Discovery for IP Version 6 (IPv6)", RFC 2461 (Standards Track), December 1998. [8] Perlman, R., "RBridges: Transparent Routing", Proc. Infocom 2005, March 2004. [9] Perlman, R., "Interconnection: Bridges, Routers, Switches, and Internetworking Protocols", Addison Wesley Chapter 3, 1999. [10] Perlman, R., J. Touch, A. Yegin, "RBridges: Transparent Routing," (work in progress), Apr. 2004 - May 2005. [11] Plummer, D., "Ethernet Address Resolution Protocol: Or converting network protocol addresses to 48.bit Ethernet address for transmission on Ethernet hardware", STD 37, RFC 826 (Standard), November 1982. [12] Touch, J., Wang, Y., Eggert, L. and G. Finn, "A Virtual Internet Architecture", ISI Technical Report ISI-TR-570, Presented at the Workshop on Future Directions in Network Architecture (FDNA) 2003 at Sigcomm 2003, March 2003. [13] Mogul, J. and S. Deering, "Path MTU discovery", RFC 1191, November 1990. Touch Expires May 17, 2006 [Page 13] Internet-Draft TRILL: Problem and Applicability November 2005 [14] Lahey, K., "TCP Problems with Path MTU Discovery", RFC 2923 (Informational), September 2000. Author's Addresses Joe Touch USC/ISI 4676 Admiralty Way Marina del Rey, CA 90292-6695 U.S.A. Phone: +1 (310) 448-9151 Email: touch@isi.edu URL: http://www.isi.edu/touch Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE Touch Expires May 17, 2006 [Page 14] Internet-Draft TRILL: Problem and Applicability November 2005 INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society (2005). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Touch Expires May 17, 2006 [Page 15]