James Kempf Internet Draft Rob Austein Document: draft-iab-e2e-futures-00.txt IAB Expires: July 2003 January 2003 The Rise of the Middle and the Future of End to End: Reflections on the Evolution of the Internet Architecture Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract The end to end principle is the core architectural principle of the Internet. In this document, we briefly examine the development of the end to end principle as it has been applied to the Internet architecture over the years. We discuss current trends in the evolution of the Internet architecture in relation to the end to end principle, and try to draw some conclusion about the evolution of the end to end principle, and thus for the Internet architecture which it supports, in light of these current trends. Table of Contents 1.0 Introduction.................................................2 2.0 A Brief History of End to End................................2 3.0 Trends Opposing to End to End................................5 4.0 Whither End to End?..........................................7 5.0 Internet Standards as an Arena for Conflict..................9 6.0 Conclusions..................................................9 7.0 Acknowledgements............................................10 Kempf and Austein Expires July 2003 [Page 1] Internet Draft Future of End to End November, 2002 8.0 References..................................................10 9.0 Security Considerations.....................................11 10.0 IANA Considerations........................................11 11.0 Author Information.........................................11 12.0 Full Copyright Statement...................................11 1.0 Introduction One of the key architectural principles of the Internet is the end to end principle [1][2]. The end to end principle was originally articulated as a question of where best not to put functions in a communication system. Yet, in the ensuing years, it has evolved to address concerns of maintaining openness, increasing reliability and robustness, and preserving the properties of user choice and ease of new service development [3]; concerns that were not part of the original articulation of the end to end principle. In this document, we examine how the interpretation of the end to end principle has evolved over the years, and where it stands currently. We examine trends in the development of the Internet that have lead to pressure to define services in the network, a topic that has already received some amount of attention from the IAB [4]. We describe some considerations about how the end to end principle might evolve in light of these trends. 2.0 A Brief History of End to End 2.1 In the Beginnning... The end to end principle was originally fairly broadly articulated as a question of where best not to put functions in a communication system: The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the end points of the communication system. Therefore, providing that questioned function as a feature of the communication system itself is not possible. (Sometimes an incomplete version of the function provided by the communication system may be useful as a performance enhancement.) [1]. However, the specific examples given in [1] and other references at the time [2] primarily involve transmission of data packets: data integrity, delivery guarantees, duplicate message suppression, per packet encryption, and transaction management. From the viewpoint of today's Internet architecture, we would view most of these as transport layer functions (data integrity, delivery guarantees, duplicate message suppression, and perhaps transaction management), others as network layer functions with support at other layers where necessary (for example, packet encryption). The authors of [1] cite a few other application areas, outside of the specific area of communication system design, where the end to end principle seems to apply, including RISC architectures. Kempf and Austein Expires July 2003 [Page 2] Internet Draft Future of End to End November, 2002 Interestingly, the expression of the end to end principle cited above is phrased as a negative: what should *not* be provided in a communication system rather than what should be provided. Much of the wider applicability later attributed to the end to end principle, outside of the original application to the transport of packets, derives from this phrasing. The end to end principle itself only provides the designer with guidance about what not to do. If a function is not supplied in the communication system, then it must be supplied in the application. In order to avoid duplicating functions in the application, the primitives in the communication system must provide the right set of non- overlapping functions (orthogonal primitives) upon which the application can be based. Though the principle of orthogonal primitives is not explicitly cited in [1], its influence is apparent throughout the examples. 2.2 ...In the Middle... As the Internet developed, the end to end principle gradually widened to concerns about where best to put services in the Internet: in the network or at end nodes. The best example is the description in RFC 1958: This principle has important consequences if we require applications to survive partial network failures. An end-to-end protocol design should not rely on the maintenance of state (i.e. information about the state of the end-to-end communication) inside the network. Such state should be maintained only in the endpoints, in such a way that the state can only be destroyed when the endpoint itself breaks (known as fate-sharing). An immediate consequence of this is that datagrams are better than classical virtual circuits. The network's job is to transmit datagrams as efficiently and flexibly as possible. Everything else should be done at the fringes.[5] The original broad articulation of the end to end principle as where not to define functions in a communication system took a while to percolate through the engineering community, and had evolved by this point to an explicit architectural statement (but still quite broad) about what belongs in the network and what doesn't. Functions that require maintaining application state belong in the applications. "Fate-sharing" describes this quite clearly: the fate of a conversation between two applications is only shared between the two applications; the fate does not depend on anything in the network, except for the network's ability to get packets from one application to the other. The end to end principle in this formulation is specifically about what kind of state is maintained where: To perform its services, the network maintains some state information: routes, QoS guarantees that it makes, session information where that is used in header compression, compression histories for data compression, and the like. This state must be self-healing; adaptive procedures or protocols must exist to derive Kempf and Austein Expires July 2003 [Page 3] Internet Draft Future of End to End November, 2002 and maintain that state, and change it when the topology or activity of the network changes. The volume of this state must be minimized, and the loss of the state must not result in more than a temporary denial of service given that connectivity exists. Manually configured state must be kept to an absolute minimum.[5] In this formulation of the end to end principle, state involved in getting packets from one end of the network to the other is maintained in the network. The state is "soft state," in sense that it can be quickly dropped and reconstructed (or even required to be periodically renewed) as the network topology changes due to routers and switches going on and off line. In summary, the general awareness both of the principle itself and of its implications for how unavoidable state should be handled grew over time to become a (if not the) foundation principle of the Internet architecture. 2.3 ...And Now. An interesting example of how end to end continues to influence the technical debate in the Internet community is route optimization in Mobile IPv6 [6]. This is a specific example in which the end to end principle has been applied to something that is the quintessential network function, namely routing. Route optimization in Mobile IPv6 requires a moving mobile node to send routing updates to its correspondents when the mobile node moves from one subnet to another. These routing updates allow the correspondent to match the mobile node's home network address to its current care-of address. Packets originally addressed to the home address by the transport layer can be routed by the correspondent's Mobile IP stack directly to the mobile node's care-of address, avoiding the overhead and latency of tunneling through the home agent. A consequence of this design is that any IPv6 node in the Internet (not just a mobile node) that wants to participate in route optimization must maintain a host route table for those mobile nodes with which it is in conversation. This design extends host-maintained routing from end to end communication in the subnet to end to end communication across the Internet. The vision emerging out of the IETF working groups developing standards for mobile networking is of a largely autonomous mobile node with multiple wireless link options among which the mobile node picks and chooses, a kind of innovative application of end to end that derives from the same basic considerations of reliability and robustness (wireless link integrity, changes in connectivity and service availability with movement, etc.) which motivated the original development of end to end. Route optimization is one example. The mobile node's autonomy is supported by having the mobile node itself take care of link selection and routing, expanding the number of multi- homed hosts by several orders of magnitude. Whether this vision will provide adequate performance without more network support is currently an open question (and see Section 5.0); but as the last line in the Kempf and Austein Expires July 2003 [Page 4] Internet Draft Future of End to End November, 2002 original formulation of the end to end principle states, sometimes performance enhancements within the network may be necessary. This new application of the end to end principle suggests that the principle is still a relevant consideration in the design of new technical applications. 3.0 Trends Opposing to End to End While the situation with mobile computing looks encouraging, the specific application of end to end described in RFC 1958 has increasingly come into question from people who want to locate services, together with their state, in the network. In particular the recently published IAB opinion on Open Pluggable Edge Services (OPES) [4] was intended to assess the architectural desirability of defining services in the network and to raise questions about how such services might result in compromises of the end to end principle. The topic of service definition the network is also taken up in [7] and [8]. Perhaps the best review of the forces militating against end to end is [3]. The authors make the point that the Internet originally developed among a community of like-minded technical professionals who trusted each other, and was administered by academic and government institutions who enforced a policy of no commercial use. The major stakeholders in the Internet are quite different today. As a consequence, new requirements have evolved over the last decade. Examples of these requirements are discussed in the following subsections. Other discussions about pressures on the end to end principle in today's Internet can be found in [9] and [10]. 3.1 Lack of Trust Perhaps the single most important change from the Internet of 15 years ago is the lack of trust between end nodes. Because the end users in the Internet of 15 years ago were few, and were largely dedicated to using the Internet as a tool for computer science research and for communicating research results, trust between end users (and thus between the end nodes that they use) was simply not an issue in general. Today, the motivations of some individuals using the Internet are not always entirely ethical, and, even if they are, the assumption that end nodes will always co-operate to achieve some mutually beneficial action, as implied by the end to end principle, is not always accurate. One of the most common examples of network elements interposing between end hosts are those dedicated to security: firewalls, VPN tunnel endpoints, certificate servers, etc. These intermediaries are designed to either protect the network from unimpeded attack or to allow two end nodes that may have no inherent reason to trust each other to achieve some level of trust. Trusted intermediaries are a major example of services defined in the network. 3.2 New Service Models New service models inspired by new applications require achieving the proper performance level as a fundamental part of the delivered service. These service models are a significant change from the Kempf and Austein Expires July 2003 [Page 5] Internet Draft Future of End to End November, 2002 original best effort service model. Email, file transfer, and even Web access aren't perceived as failing if performance degrades, though the user may become frustrated at the time required to complete the transaction. However, for streaming audio and video, to say nothing of real time bidirectional voice and video, achieving the proper performance level is part of delivering the service, and a customer contracting for the service has a right to expect the level of performance for which they have contracted. Designs for achieving additional performance in the Internet often require network elements that maintain some kind of state involving the real time flow, for example, caching streaming media content locally so it can be delivered more quickly. 3.3 Rise of the Third Party The Internet of 10 years ago was run by academic and government institutions. These institutions did not expect to make a profit from their investment in networking technology. In contrast, the network operator with which most Internet users deal today is the ISP. ISPs run their networks as a business, and expect to make a profit (or at least not lose much) on their investment in the network. While this radical change in business model is not an excuse for modifying an architectural principle that has exhibited its value over time, it does put a certain amount of pressure on the end to end principle. In particular, because an ISP delivers a commodity service, the profit margins on basic bandwidth provision for a best effort service bit pipe, together with the email and Web access services that are typically bundled with bit pipe service, are fairly low. As a result, ISPs would like to differentiate themselves by providing some services within their networks. This desire is being met by new hardware that allows routers to perform line speed examination of IP flows, primarily for purposes of billing, but the capability is also available for defining other services. An example is enhanced content delivery performance. Many ISPs today use caching services to increase the performance of Web page delivery, and caching services for streaming media are also under discussion. These services are typically deployed so that they are only accessible within the ISPs network, and as a result, they do not contribute to open, end to end service. ISPs are not the only third party intermediary that has appeared within the last 10 years. Unlike the previous involvement of corporations and governments in running the Internet, corporate network administrators, and governmental officials have become increasingly demanding of opportunities to interpose between two parties in an end to end conversation. A benign motivation for this involvement is to mitigate the lack of trust, so the third party acts as a trust anchor or enforcer of good behavior between the two ends. A less benign motivation is for the third party to insert policy for their own reasons, perhaps taxation or even censorship. The requirements of third parties often have little or nothing to do with technical concerns, but rather derive from particular social and legal considerations. Kempf and Austein Expires July 2003 [Page 6] Internet Draft Future of End to End November, 2002 3.4 The Consumer as the Primary User The original users of the Internet were technologists who understood how it worked and had no reservations about tinkering with the software or hardware on their end hosts. The users of today are primarily consumers who buy packaged hardware and software from vendors and contract with ISPs for service. They expect their Internet service to function smoothly like any other product they buy, without much involvement on their part in keeping the product functional. This development matches closely the development of other technologies, for example the automobile, that have become mainstream. This pressure to simplify the user experience has resulted in a corresponding pressure to reduce the amount of installation, configuration, maintenance, and upgrade on end nodes. Requiring user involvement in the deployment of new software or hardware on the end nodes, in order to deploy new services, runs directly counter to this trend. One response has been the tendency to move deployment of new services to servers running an existing protocol, such as HTTP, or downloadable code, such as Java or browser plug-ins, which don't require any user involvement to install. Another response has been network intermediaries to provide the service. Typically, these intermediaries don't interpose on a flow between a client and a server, but they may act more like DNS, in that the intermediary is required in order to get access to the service. A further development of this trend would be to move much of the context and configuration for a user into a node in the network, where it can be upgraded without any user involvement. This development would remove the end host as the definitive location for the application and spread it out between the network and the end host. 4.0 Whither End to End? Given the pressures on end to end discussed in the previous section, a question arises about the future of end to end. Does end to end have a future in the Internet architecture or not? If it does have a future, how should it be applied? Clearly, an unproductive approach to answering this question is to insist upon end to end as a fundamentalist principle that allows no compromise. The pressures described above are real and powerful, and if the current Internet technical community chooses to ignore these pressures, the likely result is that a market opportunity will be created for a new technical community that does not ignore these pressures but which may not understand the implications of their design choices. A more productive approach is to return to first principles and re-examine what end to end is trying to accomplish, and then update our definition and exposition of the end to end principle given the complexities of the Internet today. In the next two subsections, we consider the two basic motivations for end to end: protecting innovation and providing reliability and robustness. Kempf and Austein Expires July 2003 [Page 7] Internet Draft Future of End to End November, 2002 4.1 Protecting Innovation One motivation underlying continued application of the end to end principle is to protect innovation. Requiring modification in the network in order to deploy new services is still typically more difficult than modifying end nodes. The counterargument in Section 3.4 - that many end nodes are now essentially closed boxes that are not updatable and that most users don't want to update them anyway - does not apply to all nodes and all users. Many end nodes are still user configurable and a sizable percentage of users are "early adopters," who are willing to put up with a certain amount of technological grief in order to try out a new idea. Requiring someone with a new idea for a service to convince a bunch of ISPs to modify their networks is much more difficult than simply putting up a Web page with some downloadable software implementing the service. End to end design thus remains a viable way of fostering the service innovation that has been so important in making the Internet a useful tool in people's lives. At the same time, the pressure for lots of different parties to get involved in existing, successful applications will be irresistible [7]. Application service providers, ISPs, corporate networks, and others will want to provide special performance, reliability, and add-on services. Such pressure is fueling the desire for OPES as an adjunct to existing Web services. The result of all this additional complexity in the network is likely to move the services away from their original simplicity, and foster interoperability and other problems. About the best one can hope for is to provide protocol and application designers with guidance in maintaining end user choice, so that the basic end to end principle is maintained. To a large extent, the IAB OPES [4] work is directed at providing such guidance. 4.2 Reliability and Robustness The second motivation for continued application of the end to end principle is to increase the reliability and robustness of the exchange between the two parties in the conversation. During the early development of the Internet, the basic reliability of the hardware and software was fairly low, so involving additional network elements between the two ends could radically decrease the reliability of the overall connection. Technical reliability has improved considerably, but reliability due to involvement of network elements is still a concern [4]. Of more concern today, however, is the decrease in reliability and robustness that results from deliberate, active attacks on the network infrastructure and end nodes. While the original developers of the Internet were concerned by large scale system failures, attacks of the subtlety and variety that the Internet experiences today were not a problem during the original development of the Internet. By and large, the end to end principle was not addressed to the decrease in reliability resulting from subtlety engineered attacks. These attacks are part of the larger issue of the trust breakdown discussed in Section 3.1. Thus, the issue of the trust breakdown can be considered Kempf and Austein Expires July 2003 [Page 8] Internet Draft Future of End to End November, 2002 another forcing function on the Internet architecture, similar to the issue of reliability and robustness due to technical. The immediate reaction to this trust breakdown has been to try to back fit security into existing protocols. While this effort is necessary, it is not enough. The issue of trust must become as firm an architectural principle in protocol design for the future as the end to end principle is today. Trust isn't simply a matter of adding some cryptographic protection to a protocol after it is designed. Rather, prior to designing the protocol, the trust relationships between the network elements involved in the protocol must be defined, and boundaries must be drawn between those network elements that share a trust relationship. The trust boundaries should be used to determine what type of signaling occurs between the network elements involved in the protocol and which network elements signal each other. When signaling occurs across a trust boundary, cryptographic or other security protection of some sort may be necessary. Additional measures may be necessary to secure the protocol when communicating network elements do not share a trust relationship. For example, a protocol may need to minimize state in the recipient prior to establishing the validity of the credentials from the sender in order to avoid a memory depletion DoS attack. The end to end principle is insufficient to cover this case, because the recipient is an end node. 5.0 Internet Standards as an Arena for Conflict Internet standards have increasingly become an arena for conflict [7]. ISPs have particular business concerns, and their concerns drive some of the pressure for defining services in the network, as described in Section 3.3. Businesses and government have other concerns, and vendors of networking hardware and software still others. For example, as discussed in Section 2.3, the trend in the IETF mobile networking standards is to apply the end to end principle quite radically, to enable an autonomous mobile host. This trend strengthens the important properties of choice and user empowerment. Yet, the business pressure for mobile network operators is to hold on to the customer as long as possible, in order to derive maximum revenue from the customer. The result is a conflict between the desire of the ISP to obtain revenue to run their business and the desires of the users for choice. These conflicts will inevitably be reflected in the Internet architecture going forward. Some of these conflicts are impossible to resolve on a technical level, nor would it even be desirable, because they involve social and legal choices that the IETF is not empowered to make (for a counter argument in the area of privacy, see [11]). But for those conflicts that do involve technical choices, the important properties of user choice and empowerment, reliability and integrity of end to end service, supporting trust and "good network citizen behavior"," and fostering innovation in services should be the basis upon which resolution is made. The conflict will then play out on the field of the resulting architecture. 6.0 Conclusions Kempf and Austein Expires July 2003 [Page 9] Internet Draft Future of End to End November, 2002 The end to end principle continues to guide technical development of Internet standards, and remains as important today for the Internet architecture as in the past. While the end to end principle originated as a focused argument about where best not to put functions in a communication system, particular properties developed by the Internet as a result of the end to end principle have come to be recognized as being as important, if not more so, than the principle itself. Protection of innovation, end user choice and empowerment, reliability, integrity of service, support for trust, and "good network citizen behavior" are all properties that have developed as a consequence of the end to end principle. Recognizing these properties in a particular proposal for modifications to the Internet has become more important than before as the pressures to incorporate services into the network have increased. Any proposal to incorporate services in the network should be weighed against these properties before proceeding. 7.0 Acknowledgements Many of the ideas presented here originally appeared in the works of Dave Clark, John Wroclawski, Bob Braden, Karen Sollins, Marjory Blumenthal, and Dave Reed on forces currently influencing the evolution of the Internet. The authors would particularly like to single out the work of Dave Clark, who was the original articulator of the end to end principle and who continues to inspire and guide the evolution of the Internet architecture, and John Wroclawski, with whom conversations during the development of this paper helped to clarify issues involving tussle and the Internet. 8.0 References [1] Saltzer, J.H., Reed, D.P., and Clark, D.D., "End to End Arguments in System Design," Communications Policy in Transition: The Internet and Beyond, B. Compaine and S. Greenstein, eds. MIT Press, September 2001. [2] Clark, D., "The Design Philosophy of the DARPA Internet Protocols," Proc SIGCOMM 88, ACM CCR Vol 18, Number 4, August 1988, pp. 106-114. [3] Blumenthal, M., Clark, D.D., "Rethinking the design of the Internet: The end to end arguments vs. the brave new world", ACM Transactions on Internet Technology, Vol. 1, No. 1, August 2001, pp 70-109. [4] Floyd, S., and Daigle, L., "IAB Architectural and Policy Considerations for Open Pluggable Edge Services", RFC 3238, January 2002. [5] Carpenter, B., "Architectural Principles of the Internet," RFC 1958, June, 1996. [6] Johnson, D., Perkins, C., and Arkko, J., "Mobility Support in IPv6," draft-ietf-mobileip-ipv6-17.txt, a work in progress. [7] Clark, D.D., Wroclawski, J., Sollins, K., and Braden, B., "Tussle in Cyberspace: Defining Tommorow's Internet", Proceedings of Sigcomm 2002. [8] Carpenter, B., and Brim, S., "Middleboxes: Taxonomy and Issues," RFC 3234, February, 2002. Kempf and Austein Expires July 2003 [Page 10] Internet Draft Future of End to End November, 2002 [9] Reed, D., "The End of the End-to-End Argument?", http://www.reed.com/dprframeweb/dprframe.asp?section=paper&fn=e ndofendtoend.html, April, 2000. [10] Moors, T., "A Critical Review of End-to-end Arguments in System Design," Proc. 2000 IEEE International Conference on Communications, pp. 1214-1219, April, 2002. [11] Goldberg, I., Wagner, D., and Brewer, E., "Privacy-enhancing technologies for the Internet," Proceedings of IEEE COMPCON 97, pp. 103-109, 1997. 9.0 Security Considerations This document does not propose any new protocols, and therefore does not involve any security considerations in that sense. However, throughout this document there are discussions of the privacy and integrity issues and the architectural requirements created by those issues. 10.0 IANA Considerations There are no IANA considerations regarding this document. 11.0 Author Information Internet Architecture Board EMail: iab@iab.org IAB Membership at time this document was completed: Harald Alvestrand Ran Atkinson Rob Austein Fred Baker Leslie Daigle Steve Deering Sally Floyd Ted Hardie Geoff Huston Charlie Kaufman James Kempf Eric Rescorla Mike St. Johns This draft was created in January 2003. 12.0 Full Copyright Statement Copyright (C) The Internet Society (2001). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any Kempf and Austein Expires July 2003 [Page 11] Internet Draft Future of End to End November, 2002 kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Kempf and Austein Expires July 2003 [Page 12]