Internet Area WG J. Touch Internet Draft USC/ISI Intended status: Informational M. Townsley Expires: September 2010 Cisco March 26, 2010 Tunnels in the Internet Architecture draft-ietf-intarea-tunnels-00.txt Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on September 26, 2010. Touch, Townsley Expires September 26, 2010 [Page 1] Internet-Draft Tunnels in the Internet March 2010 Copyright Notice Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Abstract This document discusses the role of tunnels in the Internet architecture. It explains their relationship to existing protocol layers, and the challenges in supporting tunneling. Table of Contents 1. Introduction...................................................3 2. Conventions used in this document..............................4 3. Known Issues...................................................4 3.1. MTU discovery.............................................5 3.2. Fragmentation.............................................6 3.2.1. Outer Fragmentation..................................6 3.2.2. Inner Fragmentation..................................7 3.2.3. Fragmentation efficiency.............................8 3.2.4. Packing (ala GigE bursting).........................10 3.2.5. IP ID exhaustion....................................11 3.3. Signaling................................................12 4. Current Tunnel Standards......................................13 4.1. IP in IP.................................................13 4.1.1. MTU discovery.......................................13 4.1.2. Fragmentation.......................................14 4.1.3. Signaling...........................................14 4.2. IPsec....................................................14 4.2.1. MTU discovery.......................................15 4.2.2. Fragmentation.......................................15 4.2.3. Signaling...........................................15 5. Issues........................................................15 5.1. Tunnel model.............................................15 5.2. Parties participating....................................16 Touch, Townsley Expires September 26, 2010 [Page 2] Internet-Draft Tunnels in the Internet March 2010 6. Potential Ways Forward........................................17 7. Notes for future updates......................................18 8. Security Considerations.......................................19 9. IANA Considerations...........................................19 10. References...................................................20 10.1. Normative References....................................20 10.2. Informative References..................................20 11. Acknowledgments..............................................22 1. Introduction The Internet is loosely based on the ISO seven layer stack, in which data units traverse the stack by being wrapped inside data units one layer down (Figure 1). A tunnel is a mechanism for transmitting data units between endpoints by wrapping them inside data units other layers, e.g., IP in IP, or IP in UDP (Figure 2). +------+----+-----+--------------+ + Eth | IP | TCP | Data | +------+----+-----+--------------+ Figure 1 TCP inside IP inside Ethernet +------+----+-----+----+-----+--------------+ + Eth | IP'| UDP | IP | TCP | Data | +------+----+-----+----+-----+--------------+ Figure 2 IP in UDP in IP in Ethernet Tunnels help decouple topology from that provided by the physical network components. For example, they were critical in the development of multicast, where not all routers were capable of processing multicast packets. Multicast routers were interconnected by tunnels where not directly connected. Similar techniques have been used to support other protocols, such as IPv6. Use of tunnels is common in the Internet. The word "tunnel" occurs in over 100 RFCs, and is supported within numerous protocols, including: o IPsec - hides the original traffic destination [RFC4301] o L2TP - Tunnels PPP over IP, used largely in DSL/FTTH access networks to extend a subscriber's connection from an access line provider to an ISP [RFC3931] o Mobile IP - forwards traffic to the home agent [RFC2003] Touch, Townsley Expires September 26, 2010 [Page 3] Internet-Draft Tunnels in the Internet March 2010 o L2VPNs - provides a link topology different from that provided by physical links [RFC4664] o L3VPNs - provides a network topology different from that provided by ISPs [RFC4176] o SEAL - a generic mechanism for IP in IP tunneling designed to overcome the limitations of RFC2003 [RFC5320] o LISP - reduces routing table load within an enclave of routers [Fa10] o TRILL - enables L3 routing in an enclave of bridges [Pe10][RFC5556] o MPLS - ? {need description/ref} o PWE3 - ? {need description/ref} The variety of tunnel mechanisms begs the question of the roles of tunnels in the Internet architecture, and the potential need for coordination of these mechanisms. In particular, the ways in which MTU mismatch, error signals (e.g., ICMP), and is handled may benefit from a coordinated approach. It is useful to note that, regardless of the layer in which encapsulation occurs, tunnels emulate a link. As links, they are subject to link issues, e.g., MTU discovery, signaling, and the potential utility of native support for broadcast and multicast [RFC3819]. They have advantages over native links, being potentially easier to reconfigure and control. 2. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [RFC2119]. 3. Known Issues Most of the known issues with tunnels arise from the complications of encapsulation, or from the introduction of artificial endpoints along a data path. Encapsulation exacerbates MTU issues, often because a data unit will traverse at least one layer of a protocol stack more than once (e.g., as in Figure 2), which requires space for additional headers. This space complicates MTU discovery, and often results in fragmentation. Touch, Townsley Expires September 26, 2010 [Page 4] Internet-Draft Tunnels in the Internet March 2010 Tunnel encapsulation and decapsulation nodes act as network endpoints. They may source and sink much higher bandwidth streams from single IP addresses, and thus can be affected by many of the issues of other high bandwidth edge devices, such as fragmentation efficiency and IP ID exhaustion (in IPv4). These endpoints also introduce complexity in end-to-end and path signaling, in the translation between signals inside a tunnel and signals outside on the end-to-end path. 3.1. MTU discovery MTU discovery is a known challenge in the current Internet, and tunnels can complicate its proper operation. Encapsulation increases the size of a packet during tunnel transit that can exceed the MTU of the links of the tunnel path. This is especially true for recursive tunnels, i.e., tunnels that reuse layers of the protocol stack (e.g., IPv4 over IPv4). These issues are discussed in detail in [RFC4459]; the following provides a brief overview of the issues. Note that the impact of tunnels on MTU discovery may be mitigated somewhat by the ubiquity of workarounds already needed in the Internet, e.g., the deduction of a 'tunnel tax' for all MTUs (i.e., maxing out the MTU at 1200-1400 bytes, rather than 1500). Conventional path MTU discovery (PMTUD) relies on explicit negative feedback from routers along the path (ICMP "message to big" signals) [RFC1191]. This technique is susceptible to the "black hole" phenomenon, in which the ICMP messages never return to the source [RFC2923]. In the typical Internet case, lost ICMPs are often the result of filtering, e.g., for policy reasons. A more recent alternative is packetization-layer path MTU discovery (PLPMTUD) [RFC4821]. This variant relies on feedback from the endpoint, indicating either the success or failure of probe packets. It is not susceptible to "black holing", but requires explicit participation by the receiver. Either of these techniques (PMTUD, PLPMTUD) can be applied to tunnels. The encapsulator must react to "message to big" signals in either case, by either adjusting its fragmentation, relaying a corresponding signal to the packet origin outside the tunnel, or both. Fragmentation adjustment is easy to incorporate, but can result in inefficient transmission of packets over the tunnel (e.g., where every source packet is fragmented). Relaying the signal to the source can be much more efficient, but it can be difficult to determine what signal to forward. E.g., in PMTUD, routers along the tunnel may not return a sufficiently long prefix to determine the decapsulated packet origin. Touch, Townsley Expires September 26, 2010 [Page 5] Internet-Draft Tunnels in the Internet March 2010 Tunnels thus may need to participate in MTU discovery, either forwarding or recomputing ICMPs received inside the tunnel path. The tunnel may incorporate its own MTU discovery between ingress and egress, e.g., as proposed in SEAL [RFC5320]. 3.2. Fragmentation There are two places where fragmentation can occur in a tunnel, called Outer Fragmentation and Inner Fragmentation. 3.2.1. Outer Fragmentation The simplest case is Outer Fragmentation, as shown in Figure 3. The bottom of the figure shows the network toplogy, where packets start at the source, enter the tunnel at the encapsulator, exit the tunnel at the decapsulator, and arrive finally at the destination. The packet traffic is shown above the topology, where the end-to-end packets are shown at the top. The packets are composed of an inner header (iH) and inner data (iD); the term "inner") is relative to the tunnel, as will become apparent. When the packet (iH,iD) arrives at the encapsulator, it is placed inside the tunnel packet structure, here shown as adding just an outer header, oH, in step (a). When the encapsulated packet exceeds the MTU of the tunnel, the packet needs to be fragmented. In this case we fragment the packet at the outer header, with the fragments shown as (b1) and (b2). Note that the outer header indicates fragmentation (as ' and "),the inner header occurs only in the first fragment, and the inner data is broken across the two packets. These fragments are reassembled at the encapsulator in step (c), and the resulting packet is decapsulated and sent on to the destination. Touch, Townsley Expires September 26, 2010 [Page 6] Internet-Draft Tunnels in the Internet March 2010 +----+----+ +----+----+ | iH | iD |------+ - - - - - - - - - - +------>| iH | iD | +----+----+ | | +----+----+ v | +----+----+----+ +----+----+----+ (a) | oH | iH | iD | | oH | iH | iD | (c) +----+----+----+ +----+----+----+ | ^ | +----+----+-----+ | (b1) +----- >| oH'| iH | iD1 |-------+ | +----+----+-----+ | | | | +----+-----+ | (b2) +----- >| oH"| iD2 |------------+ +----+-----+ +-----+ +---+ +---+ +-----+ | | / \ ======================= / \ | | | Src |=======| Enc |=======================| Dec |=======| Dst | | | \ / ======================= \ / | | +-----+ +---+ +---+ +-----+ Figure 3 Fragmentation of the outer packet Outer fragmentation isolates Source and Destination from tunnel encapsulation duties. This can be considered a benefit in clean, layered network design, but also may result in complex decapsulator design, especially where tunnels aggregate large amounts of traffic, such as IP ID overload (see Sec. 3.2.5). Outer fragmentation is valid for any tunnel encapsulation protocol that supports fragmentation (e.g., IPv4 or IPv6), where the tunnel endpoints act as the host endpoints of that protocol. Along the tunnel, the inner header is contained only in the first fragment, which can interfere with mechanisms that 'peek' into lower layer headers, e.g., as for ICMP, as discussed in Sec. 3.3. 3.2.2. Inner Fragmentation Inner Fragmentation distributes the impact of tunneling across both the decapsulator and destination, and is shown in Figure 4. Again, the network topology is shown at the bottom of the figure, and the original packets show at the top. Packets arrive at the encapsulator, and are fragmented there based on the inner header into (a1) and (a2). The fragments arrive at the decapsulator, which removes the outer header and forwards the resulting fragments on to the Touch, Townsley Expires September 26, 2010 [Page 7] Internet-Draft Tunnels in the Internet March 2010 destination. The destination is then responsible for reassembling the fragments into the original packet. +----+----+ +----+----+ | iH | iD |-------+- - - - - - - - - - - - - >| iH | iD | +----+----+ | +----+----+ v ^ +----+-----+ +----+-----+ | (a1) | iH'| iD1 | | iH'| iD1 |------+ +----+-----+ +----+-----+ | | +----+--- +----+-----+ | (a2) | iH"| iD2 | | iH"| iD2 |------+ +----+-----+ +----+-----+ | ^ | +----+----+----- | (b1) +----- >| oH | iH'| iD1 |-------+ | +----+----+-----+ | | | | +----+----+-----+ | (b2) +----- >| oH | iH"| iD2 |-------+ +----+----+-----+ +-----+ +---+ +---+ +-----+ | | / \ ======================= / \ | | | Src |=======| Enc |=======================| Dec |=======| Dst | | | \ / ======================= \ / | | +-----+ +---+ +---+ +-----+ Figure 4 Fragmentation of the inner packet As noted, inner fragmentation distributes the effort of tunneling across the decapsulator and destinations; this can be especially important when the tunnel aggregates large amounts of traffic. Note that this mechanism is thus valid only when the original source packets can be fragmented on-path, e.g., as in IPv4. Along the tunnel, the inner headers are copied into each fragment, and so are available to mechanisms that 'peek' into headers (e.g., ICMP, as discussed in Sec. 3.3). Because fragmentation happens on the inner header, the impact of IP ID is reduced. 3.2.3. Fragmentation efficiency There are different ways to fragment a packet. Consider a network with an MTU as shown in Figure 5, where packets are encapsulated over the same network layer as they arrive on (e.g., IP in IP). If a Touch, Townsley Expires September 26, 2010 [Page 8] Internet-Draft Tunnels in the Internet March 2010 packet as large as the MTU arrives, it must be fragmented to accommodate the additional header. X===========================X (MTU) +----+----------------------+ | iH | DDDDDDDDDDDDDDDDDDDD | +----+----------------------+ | | X===========================X (MTU) | +---+----+------------------+ (a) +->| H'| iH | DDDDDDDDDDDDDDDD | | +---+----+------------------+ | | | | X===========================X (MTU) | | +----+---+----+-------------+ | (a1) +->| nH'| H | iH | DDDDDDDDDDD | | | +----+---+----+-------------+ | | | | +----+-------+ | (a2) +->| nH"| DDDDD | | +----+-------+ | | +---+------+ (b) +->| H"| DDDD | +---+------+ | | +----+---+------+ (b1) +->| nH'| H"| DDDD | +----+---+------+ Figure 5 Fragmenting via maximum fit Figure 5 shows this process, using Outer Fragmentation as an example (the situation is the same for Inner Fragmentation, but the headers that are affected differ). The arriving packet is first split into (a) and (b), where (a) is of the MTU of the network. However, this tunnel then traverses over another tunnel, whose impact the first tunnel ingress has not accommodated. The packet (a) arrives at the second tunnel ingress, and needs to be encapsulated again, but because it is already at the MTU, it needs to be fragmented as well, into (a1) and (a2). In this case, packet (b) arrives at the second tunnel ingress and is encapsulated into (b1) without fragmentation, because it is already below the MTU size. In Figure 6, the fragmentation is done evenly, i.e., by splitting the original packet into two roughly equal-sized components, (c) and (d). Note that (d) contains more packet data, because (c) includes the Touch, Townsley Expires September 26, 2010 [Page 9] Internet-Draft Tunnels in the Internet March 2010 original packet header because this is an example of Outer Fragmentation. The packets (c) and (d) arrive at the second tunnel encapsulator, and are encapsulated again; this time, neither packet exceeds the MTU, and neither requires further fragmentation. X===========================X (MTU) +----+----------------------+ | iH | DDDDDDDDDDDDDDDDDDDD | +----+----------------------+ | | X===========================X (MTU) | +---+----+----------+ (c) +->| H'| iH | DDDDDDDD | | +---+----+----------+ | | | | X===========================X (MTU) | | +----+---+----+----------+ | (c1) +->| nH | H'| iH | DDDDDDDD | | +----+---+----+----------+ | | +---+--------------+ (d) +->| H"| DDDDDDDDDDDD | +---+--------------+ | | +----+---+--------------+ (d1) +->| nH | H"| DDDDDDDDDDDD | +----+---+--------------+ Figure 6 Fragmenting evenly 3.2.4. Packing (ala GigE bursting) Encapsulating individual packets to traverse a tunnel can be inefficient, especially where headers are large relative to the packets being carried. In that case, it can be more efficient to encapsulate many small packets in a single, larger tunnel payload. This technique, similar to the effect of packet bursting in Gigabit Ethernet, reduces the overhead of the encapsulation headers (Figure 7). It reduces the work of header addition and removal at the tunnel endpoints, but increases other work involving the packing and unpacking of the component packets carried. Touch, Townsley Expires September 26, 2010 [Page 10] Internet-Draft Tunnels in the Internet March 2010 +-----+-----+ | iHa | iDa | +-----+-----+ | | +-----+-----+ | | iHb | iDb | | +-----+-----+ | | | | +-----+-----+ | | | iHc | iDc | | | +-----+-----+ | | | v v v +----+-----+-----+-----+-----+-----+-----+ | oH | iHa | iHa | iHb | iDb | iHc | iDc | +----+-----+-----+-----+-----+-----+-----+ Figure 7 Packing packets into a tunnel 3.2.5. IP ID exhaustion In IPv4, the IP Identification (ID) field is a 16-bit value that is unique for every packet for a given source address, destination address, and protocol, such that it does not repeat within the Maximum Segment Lifetime (MSL) [RFC791][RFC1122]. Although the ID field was originally intended for fragmentation and reassembly, it can also be used to detect and discard duplicate packets, e.g., at congested routers (see Sec. 3.2.1.5 of [RFC1122]). For this reason, and even more so that IPv4 packets can be fragmented anywhere along a path, all packets between a source and destination of a given protocol must have unique ID values over a period of an MSL, which is typically interpreted as two minutes (120 seconds). The uniqueness of the IP ID is a known problem for high speed devices, because it limits the speed of a single protocol between two endpoints [RFC4963]. With the maximum IP packet size of 64KB, a 16- bit ID field that does not repeat within 120 seconds means that the sum of all TCP connections between two endpoints is limited to roughly 286 Mbps; for more typical MTUs of 1500 bytes, this drops to 6.4 Mbps. Although this strongly suggests that the uniqueness of the IP ID is moot, tunnels exacerbate this condition. A tunnel often aggregates traffic from a number of different source and destination addresses, of different protocols, and encapsulates them in a header with the same ingress and egress addresses, all using a single encapsulation protocol. The result is one of the following: Touch, Townsley Expires September 26, 2010 [Page 11] Internet-Draft Tunnels in the Internet March 2010 1. The IP ID rules are enforced, and the tunnel throughput is severely limited. 2. The IP ID rules are enforced, and the tunnel consumes large numbers of ingress/egress IP addresses solely to ensure ID uniqueness. 3. The IP ID rules are ignored. The last case is the most obvious solution, because it corresponds to how endpoints currently behave. Fortunately, fragmentation is somewhat rare in the current Internet at large, but it can be common along a tunnel. Fragments that repeat the IP ID risk being reassembled incorrectly, especially when fragments are reordered or lost. Although such errors may be detected at the transport layer, this results in excessive overall packet loss, as well as wasting bandwidth between the egress and ultimate packet destination. 3.3. Signaling In the current Internet architecture, signals tend to go upstream, either from routers along a path or from the destination, back toward the source (Figure 8). Such signals are typically contained in ICMP messages, but can involve other protocols such as RSVP, transport protocol signals (e.g., TCP RSTs), or multicast. +--------------------------------------------------------------+ | | | +---------------------------+ | | | | | v v | | +-----+ | +-----+ | | | | | | Src |=========================R=============================| Dst | | | | | +-----+ +-----+ Figure 8 Signaling paths in an Internet Tunnels interfere with these known signaling paths. As shown in Figure 9, signals from routers along the tunnel path (R2), as well as those from the tunnel egress, need to be relayed by the ingress. This relaying may be difficult, because R2 may not return enough information to the ingress to support relaying (e.g., when ICMP returns only the outermost headers in a "message to big", and the source transport port information is lost). Signals from routers Touch, Townsley Expires September 26, 2010 [Page 12] Internet-Draft Tunnels in the Internet March 2010 downstream of the egress (R3 in Figure 9) need to traverse the tunnel in reverse. In all cases, the tunnel ingress needs to determine how to relay the signals from inside the tunnel into signals back to the source. For some protocols this is either simple or impossible (such as for ICMP), for others, it can even be undefined (e.g., multicast). + - - - - +-------------------------------+ | | | v v | +-----+ +---+ +---+ +-----+ | | / \ ======================= / \ | | | Src |==R1===| Enc |==========R2===========| Dec |===R3==| Dst | | | \ / ======================= \ / | | +-----+ +---+ | +---+ +-----+ ^ ^ | | | | + - - - - +---------------+ Figure 9 Signaling paths introduced by a tunnel 4. Current Tunnel Standards This section reviews two common Internet tunnel standards. They are notable because they both ultimately rely on IP in IP encapsulation, although they each handle MTU discovery, fragmentation, and signaling differently. [There are other tunnel mechanisms, such as IPv4 in IPv6, which may be added to this discussion later.] 4.1. IP in IP The simplest tunnel encapsulation mechanism is IP in IP, explained here for IPv4 [RFC2003]. This protocol was standardized for use in mobile IP, so that packets sent from a source to a Home Agent could be forwarded unmodified to the different address of the Mobile Node [RFC3344]. It has come to be used much more generally, e.g., to support multicast, as well as in overlay network systems [Er94][To01]. 4.1.1. MTU discovery When an IPv4 packet arrives at an IP-in-IP ingress, the DF flag from the inner packet is copied to the outer header. This enforces DF of the packet within the tunnel when requested by the packet source. Touch, Townsley Expires September 26, 2010 [Page 13] Internet-Draft Tunnels in the Internet March 2010 Packets which are too large are dropped at the ingress, and a corresponding ICMP "message to big" is returned to the source. Internally, IP-in-IP tunneling requires that the tunnel MUST support ICMP-based path MTU discovery (i.e., PMTUD). Note that due to common filtering of ICMP messages, this requirement is impossible to determine and thus to enforce. 4.1.2. Fragmentation IP-in-IP tunneling supports Inner Fragmentation. The inner packet MAY be fragmented if DF=0, otherwise the packet would have been dropped if too big, as noted earlier. The tunnel MUST NOT fragment at the outer header if DF=1 is set, i.e., this tunnel protocol assumes the network honors the DF bit (note that some tunnels, as well as some network devices, do not honor the DF bit). Further, if the DF bit is set in the inner header, it MUST be set in the outer; if not, it MAY be set in the outer. 4.1.3. Signaling IP-in-IP tunnels MAY relay ICMPs from inside the tunnel to the source, i.e., at the ingress. They SHOULD relay network and host unreachable messages, and MUST relay "message too big" messages; these reflect network conditions that the source should be informed about. They MUST NOT relay port unreachable messages, because these are meaningless for encapsulated packets, and thus reflect internal link conditions that the source should not care about at all. They MUST NOT relay and SHOULD handle locally messages that affect the ingress as if it were a host, e.g., source quench and router errors. Most notably, IP-in-IP notes that the tunnel SHOULD keep sufficient soft state to assist with relaying. Such state may involve keeping copies of recently sent packets, to have sufficient context to relay when lacking in the received ICMP message. 4.2. IPsec The Internet network security standard, IPsec, incorporates IP-in-IP encapsulation as part of its tunnel mode of operation [RFC4301]. Although IP-in-IP packets can be secured via IPsec transport mode, resulting in identical packets [RFC3884], the rules affecting IPsec tunnel mode MTU discovery, fragmentation, and signaling mode are specified by IPsec, rather than IP-in-IP. Touch, Townsley Expires September 26, 2010 [Page 14] Internet-Draft Tunnels in the Internet March 2010 4.2.1. MTU discovery Tunnel mode IPsec MTU discovery supports ICMP-based path MTU discovery (PMTUD), but only as a SHOULD. If an IPv4 packet arrives with DF=1, or an IPv6 packet arrives, and either is too large for the tunnel, the ingress SHOULD discard and send an ICMP to the source. If IPv4 and DF=0, the ingress SHOULD perform Outer Fragmentation, and SHOULD NOT send an ICMP to the source. 4.2.2. Fragmentation IPsec performs only Outer Fragmentation; this distinguishes it from IP-in-IP, which performs only Inner Fragmentation. It requires that implementations of tunnel mode allow the security policy to decide how the IPv4 DF bit should propagate from the inner to the outer header. It may be copied, cleared, or set, again, differing from IP-in-IP which allows only copy or set. 4.2.3. Signaling IPsec, like IP-in-IP, relays ICMP "message to big" signals from the ingress back to the source. The size indicated is adjusted to take into account for the space for both encapsulation and security information. Further, it allows that any ICMP message may be blocked, on a per-security association basis; this filtering is for security reasons, but also can directly result in "black holing". 5. Issues As has been shown in only two examples, even similar mechanisms for encapsulation can result in very different approaches to tunneling. Although these approaches result in different MTU discovery, fragmentation, and signaling mechanisms, they result from different architectural perspectives on the role of tunnels in the Internet. This section discusses these more fundamental perspectives, and their impact on the mechanisms. 5.1. Tunnel model The Internet architecture is composed of hosts, gateways (i.e., routers), and links [Cl88]. A host is a source or sink of network packet traffic, a router redirects packets from one set of links to another, and links interconnect hosts and routers. Although originally described for the Internet's network layer, this architecture, with a bit of renaming (e.g., routers become bridges), applies equally well for link layers. Touch, Townsley Expires September 26, 2010 [Page 15] Internet-Draft Tunnels in the Internet March 2010 Tunnels could, in principle, be related to this basic model in one of three ways: o Tunnel as a link o Tunnel as a router/bridge o Tunnel as invisible Tunnels require distinct ingress and egress addresses, to use during encapsulation, and to direct encapsulated traffic from the ingress to the egress. As a result, a tunnel is most usefully considered a link in the architecture in which they are deployed. As a result, tunnel designers should consider and apply link design issues [RFC3819]. This also implies that operating systems designers should represent tunnels as links; this may be conveniently represented as virtual interfaces. [this includes tunnel as point-point vs. tunnel as multipoint] 5.2. Parties participating The description of a tunnel focuses on the functions of the ingress and egress, but not all functions need be located at one of these two points. Recall inner fragmentation, in which fragment reassembly occurs at the destination, not the egress - this imposes load on the destination as a result of behavior of the ingress. Containing all tunnel functions solely inside the tunnel endpoints, as with outer fragmentation, is architecturally clean. It also obeys the 'clean up your own mess' principle; the impact of encapsulation and fragmentation caused by the ingress is then handled by the egress, without imposing load on the destination. Distributing tunnel functions across both egress and destination, as with inner fragmentation, can be more efficient. The impact of the limited IPv4 IP ID space is more prominent in the outer header, due to aggregation of traffic at the ingress. Using the inner header for fragmentation allows use of a larger effective IP ID space because of the additional IP source/destination addresses present there. Reassembly can be distributed among a large number of destinations (where present), and the impact of reassembly can be isolated to only affected destinations. Further, fragmenting once at the ingress can avoid repeated fragmentation/reassembly steps when packets traverse multiple tunnels in succession. Touch, Townsley Expires September 26, 2010 [Page 16] Internet-Draft Tunnels in the Internet March 2010 The primary case in favor of distributed tunnel functions, and thus inner encapsulation is that high speed ingress devices can be implemented, but that corresponding high speed egresses are difficult or costly. Unfortunately, network operators cannot always know in advance that high-speed ingresses are being deployed where the destination traffic is sufficiently diffuse; deploying such a device where the traffic focuses on a single destination puts an undue burden on that destination. 6. Potential Ways Forward There are a number of issues which may benefit from a coordinated review. These include unification of various tunneling standards, and revision of tunnel standards to address: o Relation of inner/outer headers (i.e., which fields are copied, derived, etc.) o MTU discovery o Fragmentation o Signaling This revision may suggest the utility of a single, configurable tunnel mechanism that includes various solutions as alternatives, rather than developing custom tunnel solutions on-demand. It may also suggest the development of new solutions, such as: o The use of PLPMTUD for tunnels o Addressing the IP ID issue and fragmentation o New ICMP signals o Optimization solutions, such as packing SEAL addresses a few of these issues, notably the first two [RFC5320]. It adds an active signal exchange between ingress and egress for intra-tunnel MTU discovery, and an extension to the IP ID space to detect collisions. Tunnels are further evidence that the current requirements for IPv4 ID uniqueness may need revision. In particular, it is clear that even moderate speed transport connections already violate these requirements. We recommend revisiting the requirements as suggested in [To10]. Touch, Townsley Expires September 26, 2010 [Page 17] Internet-Draft Tunnels in the Internet March 2010 Note that this document does not argue for a single, generic tunneling protocol or mechanism. Such a mechanism is no more likely to be useful than would a 'one size fits all' transport protocol. It does argue, however, for consistency in tunnel design, and abstraction and reuse of mechanism where possible. 7. Notes for future updates [This area includes notes for future updates which have been reported but not yet fully included - it represents a holding area for comments, and should not appear in the final document.] tunnel as virtualization - Stewart Bryant (SB) tunnel as endpoint only, not on-path (not MPLS, e.g.) - JT/coauthor gigE packing like PWE3 ATM packing - SB PPP chopping and coalescing - MT/coauthor end sec 2 "we need large seq num and to frag at the tunnel" / maybe, but do we want recommendations? - SB security should add addr management and ACLs (?) - SB MTU as part of BGP? - SB (Will this even work - JT) section 2 it says: "The IPv6 fragment header is present only when a packet has been fragmented", but I know of at least one effort in MANET that is proposing to include the fragment header even for unfragmented IPv6 packets. That would seem to bend the rules set forth in RFC2460, but I just thought it might be worth pointing out that some people are considering bending them. - Fred Templin NATs - i.e., One other thought; where the IP ID problem becomes truly pathological is for tunnels that traverse IPv4 NATs. First, the NATs could rewrite the ID to something the ingress tunnel endpoint never intended. Secondly, multiple ingress tunnel endpoints that traverse the same NAT could have IP ID "collisions" from the perspective of the outside world. This may deserve a section unto itself? - FT NAT as half-tunnel - JT tunnel endpoint as following host rules - JT (as with ECN in CAPWAP, per Magnus' email of 10/10/08) the need for larger min MTU - FT (see SEAL) Touch, Townsley Expires September 26, 2010 [Page 18] Internet-Draft Tunnels in the Internet March 2010 describe relationship to [Ho08] - JT (as per INTAREA meeting notes, don't cover Teredo-specific issues in Ho08, but include generic issues here) 8. Security Considerations Tunnels may introduce vulnerabilities, or add to the potential for receiver overload and thus DOS attacks. These issues are primarily related to the fact that a tunnel is a link that traverses a network path, and to fragmentation and reassembly. Regarding ICMP signals, tunnels have similar security issues to routers, in that they SHOULD throttle ICMPs sent to a given source, and SHOULD send ICMPs that correspond to events inside the tunnel. Such ICMPs MUST have the tunnel ingress IP address as the source IP, because IP addresses inside a tunnel path may have no meaning outside the tunnel. Tunnels traverse multiple hops of a network path from ingress to egress. Traffic along such tunnels may be susceptible to on-path and off-path attacks, including fragment injection, reassembly buffer overload, and ICMP attacks. Some of these attacks may not be as visible to the endpoints of the architecture into which tunnels are deployed, and may result in these attacks being more difficult to detect. Inner fragmentation can present an undue burden on destinations where traffic is not sufficiently diffuse; tunnels SHOULD NOT employ inner fragmentation except where such diffusion is confirmed either by the tunnel mechanism or network designer. All tunnel fragmentation - inner and outer - MUST obey all existing fragmentation requirements, i.e., IPv6 tunnels MUST NOT employ inner fragmentation, and IPv4 tunnels MUST NOT use inner fragmentation where the inner header DF=1. Tunnels MUST obey all existing IP requirements, such as the uniqueness of the IP ID field, until otherwise exceptioned or revoked. Failure to either limit encapsulation traffic, or use additional ingress/egress IP addresses, can result in high speed traffic fragments being incorrectly reassembled. 9. IANA Considerations This document has no IANA considerations. The RFC Editor should remove this section prior to publication. Touch, Townsley Expires September 26, 2010 [Page 19] Internet-Draft Tunnels in the Internet March 2010 10. References 10.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 10.2. Informative References [Cl88] Clark, D., "The design philosophy of the DARPA internet protocols," Proc. Sigcomm 1988, p.106-114, 1988. [Er94] Eriksson, H., "MBone: The Multicast Backbone," Communications of the ACM, Aug. 1994, pp.54-60. [Fa10] Farinacci, D., V. Fuller, D. Meyer, D. Lewis, "Locator/ID Separation Protocol (LISP)," (work in progress), draft- ietf-lisp-06, Jan. 2010. [Ho08] Hoagland, J., S. Krishnan, D. Thaler, "Security Concerns With IP Tunneling," (work in progress), draft-ietf-v6ops- tunnel-security-concerns-01, Oct. 2008. [Pe10] Perlman, R., D. Eastlake, D. Dutt, S. Gai, A. Ghanwani, "RBridges: Base Protocol Specification," (work in progress), trill draft-ietf-trill-rbridge-protocol-15, Jan. 2010. [RFC791] Postel, J., "Internet Protocol," RFC 791 / STD 5, September 1981. [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - Communication Layers," RFC 1122 / STD 3, October 1989. [RFC1191] Mogul, J., S. Deering, "Path MTU discovery," RFC 1191, November 1990. [RFC2003] Perkins, C., "IP Encapsulation within IP," RFC 2003, October 1996. [RFC2923] Lahey, K., "TCP Problems with Path MTU Discovery," RFC 2923, September 2000. [RFC3344] Perkins, C., Ed., "IP Mobility Support for IPv4," RFC 3344, August 2002. Touch, Townsley Expires September 26, 2010 [Page 20] Internet-Draft Tunnels in the Internet March 2010 [RFC3819] Karn, P., Ed., C. Bormann, G. Fairhurst, D. Grossman, R. Ludwig, J. Mahdavi, G. Montenegro, J. Touch, L. Wood, "Advice for Internet Subnetwork Designers," RFC 3819 / BCP 89, July 2004. [RFC3884] Touch, J., L. Eggert, Y. Wang, "Use of IPsec Transport Mode for Dynamic Routing," RFC 3884, September 2004. [RFC3931] Lau, J., Ed., M. Townsley, Ed., I. Goyret, Ed., "Layer Two Tunneling Protocol - Version 3 (L2TPv3)," RFC 3931, March 2005. [RFC4176] El Mghazli, Y., Ed., T. Nadeau, M. Boucadair, K. Chan, A. Gonguet, "Framework for Layer 3 Virtual Private Networks (L3VPN) Operations and Management," RFC 4176, October 2005. [RFC4301] Kent, S., and K. Seo, "Security Architecture for the Internet Protocol," RFC 4301, December 2005. [RFC4459] Savola, P., "MTU and Fragmentation Issues with In-the- Network Tunneling," RFC 4459, April 2006. [RFC4664] Andersson, L., Ed., E. Rosen, Ed., "Framework for Layer 2 Virtual Private Networks (L2VPNs)," RFC 4664, September 2006. [RFC4821] Mathis, M., J. Heffner, "Packetization Layer Path MTU Discovery," RFC 4821, March 2007. [RFC4963] Heffner, J., M. Mathis, B. Chandler, "IPv4 Reassembly Errors at High Data Rates," RFC 4963, July 2007. [RFC5320] Templin, F., Ed., "The Subnetwork Encapsulation and Adaptation Layer (SEAL)," RFC 5320, Feb. 2010. [RFC5556] Touch, J., R. Perlman, "Transparently Interconnecting Lots of Links (TRILL): Problem and Applicability Statement," RFC 5556, May 2009. [To01] Touch, J., "Dynamic Internet Overlay Deployment and Management Using the X-Bone," Computer Networks, July 2001, pp. 117-135. [To10] Touch, J., "Updated Specification of the IPv4 ID Field," (work in progress), draft-touch-intarea-ipv4-id-update, Feb. 2010. Touch, Townsley Expires September 26, 2010 [Page 21] Internet-Draft Tunnels in the Internet March 2010 11. Acknowledgments This document originated as the result of numerous discussions among the authors, Jari Arkko, Stuart Bryant, Lars Eggert, Dino Farinacci, Matt Mathis, and Fred Templin, as well as members participating in the Internet Area Working Group. This document was prepared using 2-Word-v2.0.template.dot. Authors' Addresses Joe Touch USC/ISI 4676 Admiralty Way Marina del Rey, CA 90292-6695 U.S.A. Phone: +1 (310) 448-9151 Email: touch@isi.edu W. Mark Townsley Cisco L'Atlantis, 11, Rue Camille Desmoulins Issy Les Moulineaux, ILE DE FRANCE 92782 Email: townsley@cisco.com Touch, Townsley Expires September 26, 2010 [Page 22]