IETF B. Carpenter Internet Draft February 1999 Internet Transparency Copyright Notice Placeholder for ISOC copyright. Abstract draft-carpenter-transparency-00.txt This document describes the current state of the Internet from the architectural viewpoint, concentrating on issues of end-to-end connectivity and transparency. It concludes with a summary of the major architectural alternatives facing the Internet network layer. This document has been discussed by the IAB, but is not yet an official IAB draft. Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Carpenter Expires August 1999 [Page 1] Internet Draft Internet Transparency Feb 1999 Table of Contents: Status of this Memo.............................................1 1. Introduction.................................................3 2. Aspects of end-to-end connectivity...........................3 2.1 The end-to-end argument.....................................3 2.2 End-to-end performance......................................4 2.3 End-to-end address transparency.............................4 3. Multiple causes of loss of transparency......................5 3.1 The Intranet concept........................................5 3.2 Dynamic address allocation..................................6 3.2.1 SLIP and PPP..............................................6 3.2.2 DHCP......................................................6 3.3 Private addresses...........................................6 3.4 Firewalls...................................................6 3.4.1 Basic firewalls...........................................6 3.4.2 SOCKS.....................................................7 3.5 Network address translators.................................7 3.6 Application gateways, relays, proxies, and caches...........7 3.7 Split DNS...................................................8 3.8 Various load-sharing tricks.................................8 4. Summary of current status and impact.........................8 5. Possible future directions...................................9 5.1 Successful migration to IPv6................................9 5.2 Complete failure of IPv6....................................9 5.2.1 Recycling of addresses....................................9 5.2.2 Exhaustion...............................................10 5.3 Partial deployment of IPv6.................................10 6. Security considerations.....................................10 Acknowledgements...............................................10 References.....................................................11 Authors' Addresses.............................................11 Intellectual Property..........................................12 Full Copyright Statement.......................................12 Carpenter Expires August 1999 [Page 2] Internet Draft Internet Transparency Feb 1999 1. Introduction The Internet is experiencing growing pains which are often referred to as "the end-to-end problem". This document attempts to analyse those growing pains by reviewing the current state of the network layer, especially its progressive loss of transparency. The causes of this loss of transparency are partly artefacts of the limited address space available to IPv4, and partly the result of broader issues resulting from the widespread use of TCP/IP technology by businesses and consumers. For example, network address translation is an artefact, but Intranets are not. Thus the way forward must recognise the fundamental changes in the usage of TCP/IP that are driving current Internet growth. In one scenario, a complete migration to IPv6 allows the restoration of address transparency, but without removing firewalls and proxies from the picture. At the other extreme, a total failure of IPv6 leads to complete fragmentation of the network layer, with global connectivity depending on endless patchwork. 2. Aspects of end-to-end connectivity The phrase "end to end", often abbreviated as "e2e", is widely used in architectural discussions of the Internet. For the purposes of this paper, we first present three distinct aspects of end-to- endness. 2.1 The end-to-end argument This is an argument first described in [Saltzer] and reviewed in [RFC 1958], from which an extended quotation follows: "The basic argument is that, as a first principle, certain required end- to-end functions can only be performed correctly by the end-systems themselves. A specific case is that any network, however carefully designed, will be subject to failures of transmission at some statistically determined rate. The best way to cope with this is to accept it, and give responsibility for the integrity of communication to the end systems. Another specific case is end-to-end security. "To quote from [Saltzer], "The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the endpoints of the communication system. Therefore, providing that questioned function as a feature of the communication system itself is not possible. (Sometimes an incomplete version of the function provided by the communication system may be useful as a performance enhancement.") "This principle has important consequences if we require applications Carpenter Expires August 1999 [Page 3] Internet Draft Internet Transparency Feb 1999 to survive partial network failures. An end-to-end protocol design should not rely on the maintenance of state (i.e. information about the state of the end-to-end communication) inside the network. Such state should be maintained only in the endpoints, in such a way that the state can only be destroyed when the endpoint itself breaks (known as fate-sharing). An immediate consequence of this is that datagrams are better than classical virtual circuits. The network's job is to transmit datagrams as efficiently and flexibly as possible. Everything else should be done at the fringes." Thus this first aspect of end-to-endness limits what the network is expected to do, and clarifies what the end-system is expected to do. The end-to-end argument underlies the rest of this document. 2.2 End-to-end performance Another aspect, in which the behaviour of the network and that of the end-systems interact in a complex way, is performance, in a generalised sense. This is not a primary focus of the present document, but it is mentioned briefly for the sake of completeness. Much work has been done over many years to improve and optimise the performance of TCP. Interestingly, this has led to comparatively minor changes to TCP itself; [RFC 792] is still valid apart from minor additions [RFC 1323, RFC 2001, RFC 2018]. However a great deal of knowledge about good practice in TCP implementations has built up, and the queuing and discard mechanisms in routers have been fine- tuned to provide the best possible system performance in congested conditions. Unfortunately all this experience in TCP performance and congestion control does not apply to transport protocols that do not exhibit TCP-like response to congestion [RFC 2309]. Also, the requirement for specified quality of service for different applications and/or customers has led to much new development, especially the Integrated Services [RFC 2210] and Differentiated Services [RFC 2475] models. At the same time new transport protocols have appeared [RFC 1889, RFC 2326] or are in discussion. Thus, despite the fact that performance and congestion issues for TCP are now quite well understood, the arrival of QOS mechanisms and of new transport protocols raise new questions about end-to-end performance. 2.3 End-to-end address transparency When the catenet concept (a network of networks) was first introduced by Cerf and Kahn [CATENET], a clear assumption was that a single logical address space would cover the whole catenet (or Internet as we now know it). This applied not only to the early TCP/IP Internet, but also to the Xerox PARC XNS design, the OSI connectionless network Carpenter Expires August 1999 [Page 4] Internet Draft Internet Transparency Feb 1999 design, and numerous proprietary network architectures. This concept had two clear consequences - packets could flow unaltered throughout the network, and their source and destination address were certain to be unique. The first of these consequences was, and is, a requirement for the end-to-end principle of Section 2.1 to be applied. Of course, minor changes to a packet such as setting type of service bits, or even fragmenting it, do not affect the end-to-end principle, but major changes to the packet by the network are not allowed. The second consequence, the uniqueness of addresses, was in a sense a side-effect of the catenet concept. However, it was a side-effect that came to be highly significant. The uniqueness and durability of addresses have been exploited in many ways, for example by building them into transport checksums, cryptogragphic signatures, Web documents, and proprietary software licence servers. [RFC 2101] explores this topic in some detail. Its main conclusion is that IPv4 addresses can no longer be assumed to be either globally unique or invariant, and any protocol or applications design that assumes these properties will fail unpredictably. Work in the IAB and the NAT working group [NAT-ARCH] has analysed the impact of one specific cause of non-uniqueness and non-invariance, i.e., network address translators. Again the conclusion is that many applications will fail, unless they are specifically adapted to avoid the assumption of address transparency. One form of adaptation is the insertion of some form of application level gateway, and another form is for the NAT to modify packets on the fly, but in either case the adaptation is application-specific. Non-transparency of addresses is part of a more general phenomenon often referred to as "Balkanisation" of the Internet. Without wishing to insult our colleagues in the Balkans, we have to recognise that the Internet has lost end-to-end transparency, and this requires further analysis. 3. Multiple causes of loss of transparency This section describes the various recent inventions that have led to the loss of end-to-end transparency in the Internet. 3.1 The Intranet concept Underlying a number of the specific developments mentioned below is the concept of an "Intranet", loosely defined as a private corporate network using TCP/IP technology, and connected to the Internet at large in a carefully controlled manner. The Intranet is presumed to be used by corporate employees for business purposes, and to interconnect hosts that carry sensitive or confidential information. It is also held to a higher standard of operational availability than Carpenter Expires August 1999 [Page 5] Internet Draft Internet Transparency Feb 1999 the Internet at large. The emergence of the Intranet concept has had a profound effect on the notion of applications transparency. Many corporate network managers feel it is for them alone to determine which applications can traverse the Internet/Intranet boundary. In this world view, address transparency may seem to be an unimportant consideration. 3.2 Dynamic address allocation 3.2.1 SLIP and PPP It is to be noted that with the advent of vast numbers of dial-up Internet users, whose addresses are allocated at dial-up time, and whose traffic may be tunelled back to their home ISP, the actual IP addresses such users use becomes a matter of only transient significance. 3.2.2 DHCP Similarly, LAN-based users of the Internet today frequently use DHCP to acquire a new address at system restart, so here again the actual value of the address has only transient siginficance. 3.3 Private addresses When the threat of IPv4 address exhaustion first arose, and in some cases user sites were known to be "pirating" addresses for private use, a set of official private addresses were hurriedly allocated [RFC 1597] and later more carefully defined [RFC 1918]. The legitimate existence of such an address allocation became a self- fulfilling prophecy, so Intranets with large numbers of non-global addresses came into existence. 3.4 Firewalls 3.4.1 Basic firewalls Intranet managers have a major concern about security: non-employees must be kept out of the Intranet at all costs. This concern led directly to the firewall concept (a system that intercepts all traffic between the Internet and the Intranet, and only lets through selected traffic, usually belonging to a very limited set of Carpenter Expires August 1999 [Page 6] Internet Draft Internet Transparency Feb 1999 applications). Firewalls, by their nature, fundamentally limit transparency. 3.4.2 SOCKS A footnote to the effect of firewalls is the SOCKS mechanism [RFC 1928] by which untrusted applications such as telnet and ftp can punch through a firewall. SOCKS requires a shim library in the Intranet client, and a server in the firewall which is essentially an application level relay. As a result, the remote server does not see the real client; it believes that the firewall is the client. 3.5 Network address translators These are an almost inevitable consequence of the existence of Intranets using private addresses yet needing to communicate with the Internet at large. Their architectural implications are discussed at length in [NAT-ARCH], the fundamental point being that address translation on the fly utterly destroys end-to-end address transparency and any middleware or applications that depend on it. In practice NATs only work in conjunction with application level gateways. 3.6 Application gateways, relays, proxies, and caches At certain critical topological points, specially the Intranet/Internet boundary, it is reasonable to position these functions. For example if an Intranet is foolish enough not to use SMTP as its mail protocol, an SMTP gateway is needed. Even in the normal case, an SMTP relay is common, and can perform useful mail routing functions, spam filtering, etc. (It may be observed that spam filtering is in some ways a firewall function, but the store-and- forward nature of electronic mail and the availability of MX records allow it to be a distinct and separate function.) Similarly, for a protocol such as HTTP with a well-defined voluntary proxy mechanism, application proxies also serving as caches are very useful. Although these devices interfere with transparency, they do so in an architecturally sound way. However, there appear to be cases of "involuntary" applications level devices such as proxies that grab and modify HTTP traffic without using the appropriate mechanisms, or mail relays that purport to remove undesirable words. These devices are not properly architected and may have totally unforeseeable side effects. (A possible conclusion is that even for non-store-and-forward protocols, a generic diversion mechanism analagous to the MX record would be of benefit.) Carpenter Expires August 1999 [Page 7] Internet Draft Internet Transparency Feb 1999 3.7 Split DNS Another consequence of the Intranet/Internet split is "split DNS" or "two faced DNS", where a corporate network serves up partly or completely different DNS inside and outside its firewall. There are many possible variants on this; the basic point is that the correspondence between a given FQDN and a given IPv4 address is no longer universal. 3.8 Various load-sharing tricks Unlike IPv6, IPv4 does not have a designed mechanism for anycast, so there is no natural approach to load-sharing when one server cannot do the job. Various tricks have been used to resolve this (multicast to find a free server, the DNS returns different addresses for the same FQDN in a round-robin, a router actually routes packets sent to the same address automatically to different servers). While these tricks are not particularly harmful in the overall picture, they again interfere with name or address transparency. 4. Summary of current status and impact It is impossible to estimate with any numerical reliability how widely the above inventions have been deployed. Since many of them preserve the illusion of transparency while actually interfering with it, they are impossible to measure. However it is certain that all the mechanisms just described are in very widespread use; they are not a marginal phenomenon and in corporate networks, many of them are the norm. Some of them (firewalls, relays, proxies and caches) clearly have intrinsic value given the Intranet concept. The others are largely artefacts and pragmatic responses to various pressures in the operational Internet, and they must be costing the industry very dearly. In particular, the various lacks of transparency are having a severe effect on deployment of end-to-end IP security. The Internet security model [SECMECH] calls for security at several levels (roughly, network, applications, and object levels). The current network level security model [RFC 2401] was constructed prior to the recognition that end-to-end address transparency was under severe threat. Although alternative proposals have begun to emerge [HIP] the current reality is that IPSEC cannot be deployed end-to-end in the general case. Tunnel-mode IPSEC can be deployed between corporate gateways or firewalls. Transport-mode IPSEC can be deployed within a corporate network in some cases, but it cannot span from Intranet to Internet and back to another Intranet if there is any chance of a NAT along the way. Even without address translation, the key management implications of such a span are horrendous. Carpenter Expires August 1999 [Page 8] Internet Draft Internet Transparency Feb 1999 5. Possible future directions 5.1 Successful migration to IPv6 In this scenario, IPv6 becomes fully implemented on all hosts and routers, including the adaptation of middleware, applications, and management systems. Since the address space then becomes big enough for all conceivable needs, address transparency is restored. Transport-mode IPSEC can in principle deploy. However, the Intranet/firewall model will certainly still apply. Clearly this is any case a very long term scenario, since it assumes that all IPv4 devices have ceased to communicate. Thus, a viable version of Scenario 5.3 is a prerequisite for Scenario 5.1. 5.2 Complete failure of IPv6 In this scenario, IPv6 fails to reach any significant level of operational deployment, IPv4 addressing is the only available mechanism, and address transparency cannot be restored. IPSEC cannot be deployed globally in its current form. Eventually, the pool of globally unique IPv4 addresses will be totally allocated and no new addresses will be available for any purpose. At least two sub- scenarios are possible: 5.2.1 Recycling of addresses A mechanism is created continuously to recover and re-allocate addresses. A single global address space is maintained, with all sites progressively moving to an Intranet private address model, with global addresses being assigned temporarily. 5.2.1.1 A sub-sub-scenario of this is generalised use of NAT and NAPT (NAT with port number translation). See [NAT-ARCH]. 5.2.1.2 Another sub-sub-scenario is the use of realm-specific IP addressing implemented at the host rather than by a NAT box. See [RSIP]. (Note - this was known as Host NAT until recently.) 5.2.1.3 A final sub-sub-scenario is the "map and encapsulate" model in which address translation is replaced by systematic encapsulation of all packets for wide-area transport, which could also be viewed as a generic alternative to virtual private networks (virtual public networks?). This model has never been fully developed. Carpenter Expires August 1999 [Page 9] Internet Draft Internet Transparency Feb 1999 5.2.2 Exhaustion No mechanism is created to recover addresses (except perhaps black or open market trading). Sites with large address blocks keep them. All the scenarios of 5.2.1 appear but are insufficient. Eventually the global address space is no longer adequate. Nightmare scenario - NATs appear within the "global" address space, for example at ISP-ISP boundaries. 5.3 Partial deployment of IPv6 In this scenario, IPv6 is completely implemented but only deploys in certain segments of the network (e.g. in certain countries, or in certain areas of application such as mobile hand-held devices). Instead of being transitional in nature, some of the IPv6 transition techniques become permanent features of the landscape. Sometimes addresses are 32 bits, sometimes they are 128 bits. DNS lookups may return either or both. In the 32 bit world, the scenarios 5.2.1 and 5.2.2 may occur. IPSEC can only deploy partially. 6. Security considerations The loss of transparency is both a bug and a feature from the security viewpoint. To the extent that it prevents the end-to-end deployment of IPSEC, it damages security and creates vulnerabilities. For example, if a NAT box is in the path, then the best that can be done is to decrypt and re-encrypt IP traffic in the NAT. The traffic will therefore be momentarily in clear text and potentially vulnerable. If this is unacceptable, encryption must be applied above the network layer instead, of course with no usage whatever of IP addresses in the cryptographic process. See section 4 for further discussion. The loss of transparency at the Intranet/Internet boundary may be considered a security feature, since it provides a well defined point at which to apply restrictions. This form of security is subject to the "crunchy outside, soft inside" risk, whereby any successful penetration of the boundary exposes the entire Intranet to trivial attack. Acknowledgements This document has been discussed by the IAB, but is not yet an official IAB draft. Carpenter Expires August 1999 [Page 10] Internet Draft Internet Transparency Feb 1999 References [Saltzer] End-To-End Arguments in System Design, J.H. Saltzer, D.P.Reed, D.D.Clark, ACM TOCS, Vol 2, Number 4, November 1984, pp 277-288. [CATENET] need a reference here [RFC 792] [RFC 1597] [RFC 1889] [RFC 1918] [RFC 1928] [RFC 1958] [RFC 2001] [RFC 2018] [RFC 2101] [RFC 2210] [RFC 2309] [RFC 2326] [RFC 2401] [RFC 2475] [NAT-ARCH] draft-iab-nat-implications-03.txt [SECMECH] draft-iab-secmech-00.txt [RSIP] draft-ietf-nat-rsip-framework-00.txt [HIP] R. Moskowitz, private communication Authors' Addresses Brian E. Carpenter IBM United Kingdom Laboratories MP 185, Hursley Park Winchester, Hampshire SO21 2JN, UK Email: brian@hursley.ibm.com Carpenter Expires August 1999 [Page 11] Internet Draft Internet Transparency Feb 1999 Intellectual Property PLACEHOLDER for full IETF IPR Statement if needed. Full Copyright Statement PLACEHOLDER for full ISOC copyright Statement if needed. Carpenter Expires August 1999 [Page 12]