Internet Draft B. Cain Cereva Networks O. Spatscheck AT&T Labs M. May Activia Networks A. Barbir Nortel Networks Expires July 2001 Request Routing Requirements for Content Internetworking draft-cain-request-routing-req-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) The Internet Society (2000). All Rights Reserved. Abstract Content Delivery Networks (CDN) require Request Routing Systems to direct user requests to an available copy of an object based on one or more metrics. As CDNs begin to interconnect, it is necessary for these systems to exchange information so that requests can be routed between domains. This document describes the requirements for the interconnection of Request Routing Systems. 1. Introduction Content Delivery Networks (CDN) require Request Routing Systems to direct user requests to an available copy of an object based on one Cain, et. al. Expires July 2001 ^L[Page 1] Internet-Draft January 2001 or more metrics. As CDNs begin to interconnect, it is necessary for these systems to exchange information so that requests can be routed between domains. This document describes the requirements for the interconnection of Request Routing Systems and makes use of terminology from [MODEL]. 1.1 Overview of Request Routing Systems Request Routing Systems (RRS) direct requests to a suitable surrogate which can service a request and provide "best" service to the user. This decision is based on a set of metrics which may include for example network proximity and server load. The request routing process has also been referred to as content routing in certain contexts. Most CDNs make use of multiple metrics in their request routing systems. The fundamental problems that a request routing system attempts to solve are: 1. Direct users to a surrogate which can best serve the request (per a set of metrics) 2. Direct users to a surrogate which (per a set of metrics) is "close" to the user In order to show how the requirements contained in this document map into [MODEL] [ARCH] we now reiterate serveral important assumptions from [ARCH] [MODEL]: 1. Each content delivery network is a "black box" to other networks to which it is connected (or peered). 2. Content is served by surrogates which act on behalf of an origin server which contain the "master" copy of content objects 3. Each content delivery network may have its own internal (or intra-domain) request routing system which is not exposed to other interconnected networks 1.2 Two Request Routing System Types Request routing systems may be architected in several ways, making their decisions based on a request which may be: 1. A DNS lookup 2. A Layer-7 request (e.g. HTTP, RTSP) received by a proxy, an origin server (using a redirect) or layer-7 router Cain, et. al. Expires July 2001 ^L[Page 2] Internet-Draft January 2001 It is important to make this distinction because the request visibility has a high impact on the request routing decision and therefore on the requirements and architecture for interconnecting these systems. We now quickly reiterate the issues here; a better description is in [ARCH] [KNOWN MECH]. 1.2.1 DNS Based When DNS is used a a basis for the request routing system, only aggregate sets of content may be "directed" because a domain name (e.g. images.blah.com) almost always represents a larger set of content. A DNS request routing system works well in scenarios where many surrogates share large sets of content. It is important to note that individual content objects (e.g. full URI, URL) are not visible to a DNS request routing system. Furthermore, since most DNS systems use time-to-live (TTL) values in replies, every request is not seen by the request routing system. 1.2.2 Layer-7 Based When a Layer-7 router or Proxy is situated close to a user, it may be used to route requests based on individual content requests. This is possible because layer-7 information (e.g. HTTP headers) are exposed to the layer-7 router or proxy. In this type of request routing system, a surrogate can be chosen based on for example, a full URI path. Another example where layer-7 information is visible is when an origin server or other proxy performs a layer-7 redirection. Because individual layer-7 requests are being received, each request may be directed individually (as opposed to DNS, see above). 1.2.3 Comparison The two request routing system types presented above have slightly different requirements with respect to information exchange between interconnected networks. Interconnected DNS request routing systems for example, may only exchange information about the aggregate networks. Interconnected Proxy based request routing systems for example, may exchange specific information as to where a particular content object resides. In summary, interconnected request routing systems need to exchange two basic types of information: 1. Information related to aggregate network capacity, network health, and other network related metrics 2. Information specific to a content object or set of content objects. This information is per URL information may include: content type, distribution model, authoritative delivery root, etc Cain, et. al. Expires July 2001 ^L[Page 3] Internet-Draft January 2001 1.2.4 Examples We now present two examples follow which illustrate the differences between these two types of request routing systems. Both examples use a simple scenario of CDN A (with CPG-A) interconnected to CDN B (with CPG-B). 1.2.4.1 DNS Example CDN A is authoritative for http://images.blah.com (or CNAMEs are used to ultimately force a resolution of this name to CDN A). When CPG-A receives the DNS request for images.blah.com, it makes a request routing decision. This decision may be to direct the request to a its own surrogates (assuming CDN A has it own distribution network) or to direct the request to another distribution network. CPG-A may decide based on "area" advertisements and other associated metrics to direct the request to CDN B. This decision will be based on: 1. Information contained in area advertisements which have been received from CPG-B 2. The ability of CDN-B to support the content type of the request 3. Comparison of CDN-B's area metrics vs. other directly interconnected CDNs (none in the example). 1.2.4.2 Layer-7 Example Assume X uses proxy P which (internally) communicates with its own CPG-B. [note: we consider the communication from P to CPG-B to be "internal" CDN communication -- of course P may be CPG-B or P may use a similar protocol to CGP-B] When a request arrives from X to P, it uses the request routing information communicated from CPG-B to make a decision as to where to direct X's request. Assume that CPG-A has advertised http://images.blah.com/logo.gif to CPG-B, and CPG-B has communicated this to proxy P; proxy P can make a request routing decision for that content. If X has requested http://images.blah.com/logo.gif, proxy P will make a routing decision based on: 1. Information contained in content advertisements which have been received from CPG-A (and communicated to P through CPG-B). These metrics are per content unit or aggregate content set. 2. Comparison of CDN A's content metrics vs. other directly interconnected CDNs (none in the example). 1.3 Important Issues to Consider This document attempts to gather requirements for the interconnection of request routing systems. The following are several of the Cain, et. al. Expires July 2001 ^L[Page 4] Internet-Draft January 2001 difficult issues with respect to interconnecting request routing systems: 1. Not all content delivery networks are able to handle the same types of content. 2. Content delivery networks are overlay networks which makes request routing decisions difficult. Furthermore, content delivery networks have differing requirements for service levels which complicates the decision process. 3. Multiple types of request routing systems must be supported (see section 1.2) 4. The scalability of exchanging content specific information may be difficult to achieve. 2. General Requirements for Request Routing Systems and Protocols In this section we present general requirements for the exchange of inter-domain request-routing information. 2.1 Overview of General Requirements The overall goal of request routing is to route a client request to an appropriate surrogate. To support this goal each piece of content MUST have one authoritative request routing system which has the ultimate control over request routing. This authoritative request routing system MAY make the entire routing decision or use an iterative or recursive model to determine the surrogate. If an iterative process is used the request might be forwarded to another request routing system to be resolved. If a recursive process is used the authoritative request routing system might make a query to another request routing system. The request routing system MUST scale to the size of the Internet. To achieve this goal the request routing system follows the proven path of separating the Internet into autonomous systems (called CDNs) which interoperate. To support this model the CDN Request Routing Systems MUST treat other CDN networks as "black boxes"; that is, a given CDN A does not have direct visibility into another peered CDN B. Since in such an environment nobody posses a global few of all CDNs the request routing system MUST also rely on a peer to peer model in which each request routing system is only aware of its direct neighbor. Note: A direct neighbor of the request routing systems does not have to be a direct neighbor on layer 3. 2.2 Type of Request Routing Cain, et. al. Expires July 2001 ^L[Page 5] Internet-Draft January 2001 In different environments different types of request routing might be optimal according to a given metrics like delay, cost or convenience. Therefore, one goal of the request routing system is to support as many types of request routing as possible. The request routing protocols MUST support at least the types of request routing discussed in the following subsections which are the currently prevalent types of request routing. In addition it MUST be possible to utilize more than one type of request routing for a single piece of content. However, this document does not require that all request routing systems support all the methods defined in the protocol. In this case peering MUST be restricted to request routing systems which support a compatible set of methods. 2.2.1 DNS Based Request Routing CNAME based DNS request routing MUST be supported by the request routing system. In a CNAME based DNS request routing system a clients DNS resolution is redirected using a DNS CNAME record to another DNS based request routing system until a surrogate is found which is optimal (according to some definition of optimal) to serve the content. The drawbacks of CNAME based request routing are discussed in [KNOWN MECH]. One problem with DNS based request routing is that only the DNS name is utilized for request routing. Therefore, DNS based request routing MUST use DNS names which allow to make a useful routing decision based on a DNS name alone. For example, if images are served from different surrogates then streaming media it has to be possible to identify the type of content by the DNS name alone. One of the problems with DNS request routing is being able to identify the content type of the request. This may require some standardization of DNS names in order to properly identify this information (for content distributed on CDNs). 2.2.2 Layer-7 Based Request Routing Both HTTP and RTSP allow the redirection of clients on the application layer. The protocols introduced by the request routing system MUST support the option of using this methods and other L7 request routing methods to redirect the client to a surrogate. 2.2.3 Interception Based Request Routing Interception proxies are widely deployed in todays Internet and MUST be supported by the request routing protocol. To support interception proxies the protocol MUST provide enough information so that an interception element can decide if a particular request should be intercepted. 2.3 Policy Requirements Cain, et. al. Expires July 2001 ^L[Page 6] Internet-Draft January 2001 In this section we specify the minimum requirements of a policy engine which will be used to drive the request routing system. The policy which a particular request routing system enforces is most likely more complex, however, only the minimum policy requires standardization. As a minimum the policy engine of the request routing system MUST guarantee that a peer CDN is capable to serve the content with high probability before redirecting clients to the peer. 2.4 Feedback In order to exchange any content specific information, a standardized method for identifying an atomic unit of content MUST be defined. Using this method of identification it will be necessary to gather additional information to support realistic policies. This information SHOULD be retrieved as much as possible from the accounting and distribution system. As a minimum the information exchanged MUST be sufficient to prevent request routing loops, however, more complex information is most likely needed. Therefore, the request routing protocols MUST support feedback with extendible metrics. To increase the likelihood of interoperability of multiple request routing system the protocol SHOULD define a minimum set of metrics which are required to conform to the protocol specification. 2.5 Impact on End User The goal of the request routing systems is to improve the performance and scalability of the Internet without inconvenience to the end user. Therefore, the request routing system MUST be transparent to the end-user and avoid violation of the end-to-end principles of the Internet architecture. This does not mean that an end user can not discover the presence of a request routing system. However, the end user should not be forced to change its current behavior. 2.6 Summary of Requirements - Request Routing Systems and protocols MUST support arbitrary direction topologies; this means "peer-to-peer" design. - Request Routing Protocols MUST support both DNS based direction as well as layer-7 routing and proxy direction. - Request Routing Protocols MUST support feedback with extensible metrics. Cain, et. al. Expires July 2001 ^L[Page 7] Internet-Draft January 2001 - Peered CDN Request Routing Systems MUST treat other CDN networks as "black boxes"; that is, a given CDN A does not have direct visibility into another peered CDN B. - In order to exchange content specific information between Request Routing Systems, a standardized method for identifying an atomic unit of content MUST be defined. - When exchanging content unit information, the authoritative redirection system MUST be identified for each content unit exchanged. - Request Routing Systems MUST be able to interface to associated Accounting and Distribution Systems. - Request Routing Systems SHOULD verify that peered CDNs have the ability to delivery a given type of content before exchanging information or directing requests regarding that content type. - Request Routing Systems MUST be able to respond to a direction request for content for which it is authoritative. A given Request Routing System MAY direct the request to another Request Routing System. - Request Routing Systems MAY use multiple metrics for the direction decision. Example metrics for peered CDNs may include: load, coverage, content types, proximity, etc. - Request Routing Systems and protocols MUST provide methods to avoid "direction loops". - Request Routing Systems MUST be transparent to the end-user and avoid violation of the end-to-end principles of the Internet architecture. - Request Routing System protocols MUST support DNS direction through the use of CNAME records (e.g. CDN B has to provide a CNAME to redirect to for each unique DNS name which might be redirected to CDN B) - Common formats for content identification must exist such that DNS direction may occur based on content type (e.g. streaming content vs. static content has to be identifiable using the DNS name alone). This may require standardization of DNS names in URI/URLs (for content distributed on CDNs) such that content types can be identified. 3. Requirements for Request Routing System Information Exchange 3.1 Overview of Information Exchange Cain, et. al. Expires July 2001 ^L[Page 8] Internet-Draft January 2001 This section provides an overview for Request Routing System (RRS) requirements for content routing information exchanged between peered CDN networks. Each CDN participating in the peering must have at least an RRS system that communicates with one or multiple RRS's supporting peered CDN networks. A typical Request Routing System consists of the following components: Content Toplogy Exchange, Content Topology Data Base and Route computation. (ref Figure 1) ____________________________________________ | ___________ ________ ________ | | |Routing | |Content | |Content | | MI<->| |Computation| |Topology|<->|Topology| |<->IEP | |___________| |DataBase| |Exchange| | | |________| |________| | |__________________________________________| Figure 1. A brief description of the components of Figure 1 is provided below: 1. Management Interface (MI): This interface provides the CDN provider the ability to configure the RRS system. This is specific to individual implementations. 2. Routing Computation: Responsible for computing the content route based on information stored in the Content Topology Data Base, route computing algorithm and policies. This decision may be based on a variety of internal or external metrics. 3. Content Topology Database: The topology database includes detailed topology information about its peers and associated metrics exchanged during Content Information Exchange. 4. Content Information Exchange: This functional block is responsible for implementating the Information Exchange protocol. 5. Information Exchange Protocol (IEP): Protocol used to negotiate mode of operation, and capabilities that the RRS system will use to communicate with each other. The protocol is also used to exchange advertisement and other messages between the peered RRS. 3.2 Overview of Request-Routing System Information Exchange In general, the RRS system communicate using a two phase approach. The details of the phases are provided in the subsequent sections. 3.2.1 Phase A: Capability Advertisement or Negotiation Cain, et. al. Expires July 2001 ^L[Page 9] Internet-Draft January 2001 - A Request Routing System MUST be able to advertise which Request Routing system types it supports (i.e. DNS vs. Layer-7 Routing/Proxy). - A Request Routing System MUST be able to advertise which content types is supports (e.g. streaming vs. static). 3.2.2 Phase B: Advertisment - A Request Routing System MUST be able to associated multiple (and optional) metrics with advertisement information. - A Request Routing System MUST support both the exchange of AREA (e.g.IP prefixes) and CONTENT UNITS (e.g. URIs) independently to support one or more types of Request Routing system types. - A Request Routing System MUST be able to advertise aggregrate metrics per region and content type (e.g. X Mbps, Y CIDR blocks, Z static http). 3.2.2.1 Area Advertisments - A Request Routing System MUST be able to advertise layer-3 (e.g. IP prefix coverage) for its associated distribution system. - Area advertisement information MUST be able to be aggregated for the exchange process. 3.2.2.2 Content Unit Advertisements - A Request Routing System MUST be able to advertise that it is authoritative for a unit(s) of content. - A Request Routing System MUST be able to advertise content units (e.g. Content type, URIs) with an associated set of metrics. [note: proxy/L7 case] - Content unit advertisements must support some form of aggregation for the exchange process. - A Request Routing System MUST be able to advertise which content types (e.g. streaming, static) it supports. - A Request Routing System MUST be able to advertise the availability of content per particular distribution methods (e.g. push). Cain, et. al. Expires July 2001 ^L[Page 10] Internet-Draft January 2001 3.3 Overview of Request-Routing System Information Exchange For the discussion it is assumed that the RRS represents a logical node on a network that could be reached by another RRS node. For the purposes of this draft, each RRS must be able to be identified by a unique identifier (RRS node identity). The peered RRS exchange content routing information using the IEP protocol in the form of Topology State Elements (TSE), whereby, each TSE contains the identity of the RRS node and other information pertaining to the status of the current content. The TSEs are the smallest collection of information that is exchanged as a unit among peered RR nodes. Within an RRS node, there is a content topology database that consists of a collection of all TSEs that have been received from a peered RRS. In particular the content topology database provides all the information required to compute a content route to a peered CDN. At a high level, the IEP process requires two phases. The phases are described below: 1. Setup Stage This stage includes the "Hello" step at the beginning of communication. It also includes the negotiation or advertisement (see below) of the capabilities that the two RRS nodes will support. Examples include: * DNS vs layer 7/proxy * metric update frequency * Data encoding type * Agree on set of metrics including optional ones * etc Basically, the negotiation of capabailities will follow a defined state machine. At the end of the negotiation phase, the two RRS will start communicating if the negotiation is successful, otherwise, the process terminates. [note: It is assumed that RR process will be carried over a secure channel such as SSL or IPSec, that may also include an authentication step.] Negotation processes may follow either one of two designs. The first process is one where strict negotation takes place between peers. Only if all requirements are satisfied does the peering continue and with only the particular negotationed items [note: this is similar to PPP protocol negotitation]. A second process is much simplified where peers advertise their capabilities; if certain capabilities do not match then the protocol does not proceed and logs an error [note: this is similar to BGP capability advertisement]. At a minimum the exchange protocol must support advertisement; it is unclear at this time whether Cain, et. al. Expires July 2001 ^L[Page 11] Internet-Draft January 2001 full blown negotation needs to be supported (i.e. it is an open issue). 2. Exchange of Advertisement Phase This phase consists of exchanging of advertisements through a set of messages. The exchange of advertisements could be done on regular intervals as agreed upon during the setup phase or as required in the event of a change in some metrics or content. This is highly dependent on the protocol design (e.g. hard state vs. soft state). 4. Specific Protocol Requirements for Request-Routing Systems This section describes several specific protocol requirements which have been identified that for an inter-domain request-routing exchange protocol. 4.1 Overview of Protocol Requirements The following section presents lower level protocol requirements for exchange of information between request routing systems. Request routing system information exchange MUST include transit information that allows to construct a graph of the connected CDNs. This graph information MUST be used to prevent request routing loops. The RRS SHOULD produce a resolution which is "transparent" to the end user. The Request Routing Protocol (RRP) is an inter-CDN request routing system protocol. It runs over a reliable transport level protocol. Hence, it does not require retransmission, acknowledgement, and sequencing. The protocol MUST be secure through the use of IPSec or similar security measures. An authentication scheme MUST/SHOULD/MAY be used. An authentication code specifies the type of authentication that will be used and determines the form and the meaning of the authentication data. Two CDN peers perform an initial exchange after their first connection. Once a connection is established, the peering partners (RRP stations) will start exchanging "updates". After each change in the request routing tables, a corresponding update message is propagated to all neighbors. Cain, et. al. Expires July 2001 ^L[Page 12] Internet-Draft January 2001 If a RRP station receives a ill-formated message, or if it fails to receive any message, it will report the error to its peer by sending a notification message. This notification message supports error codes. 4.2 Protocol Requirements 4.2.1 Functional Requirements - Protocols for exchange of information between Request Routing Systems MUST be connection oriented. [possible disagreement] - Protocols for exchange of information between Request Routing Systems SHOULD support peer discovery (this may be at odds with above). - Protocols for exchange of information between Request Routing Systems MUST have capabilities to prevent looping of information. - Protocols for exchange of information between Request Routing Systems MUST support error code exchange for debugging. - Protocols for exchange of information between Request Routing Systems MUST support extensible metrics and attributes. 4.2.2 Identification Authentification - Protocols MUST provide services to identify peers before granting access to other protocol services - Protocol MUST provide services to authenticate clients before granting access to other protocol services - Protocols for exchange of information between Request Routing Systems MUST be secure through the use of IPSec or other security measures. 4.2.3 Protocol Extensibility Extensibility is a measure of the extent to which a protocol can be adapted for future uses that were not readily evident when the protocol was originally designed. The request routing protocol SHOULD provide features that at a minimum allow for the management of for example new metrics and attributes without requiring revisions to the protocol itself. Cain, et. al. Expires July 2001 ^L[Page 13] Internet-Draft January 2001 4.2.4 Other Protocol Issues - Protocols for exchange of information between Request Routing Systems MUST scale to large sets of content meta data as well as other CDN specific information (e.g IP prefixes). An example is hard state vs. soft state protocol design. - Protocols for exchange of information between Request Routing Systems MUST support (at minimal) a simple capability advertisement. - Protocols for exchange of information between Request Routing Systems MUST be able to accomodate implementation specific policies. - Protocols for exchange of information between Request Routing Systems SHOULD NOT exchange policy information. 5. Open and Interesting Issues In this section we present several open issues with respect to the exchange protocol. Note that many of these are implementation specific. However, it is useful to think about these issues with respect to request routing. - How should data be represented in the exchange protocol? Example: XML vs. byte encoding - How should the protocol support multiple languages (per DNS and URIs)? - How can large amounts of content specific information be exchanged in a scalable fashion? - Should compression techniques be used? - Should hashing techniques be used? 6. Security Considerations TBD 7. References Cain, et. al. Expires July 2001 ^L[Page 14] Internet-Draft January 2001 [MODEL] Day, M., Cain, B. and G. Tomlinson, "A Model for CDN Peering", draft-day-cdnp-model-03.txt (work in progress), November 2000. [KNOWN MECH] Cain, B., Douglis, F., Green, M., Hofmann, M., Nair, R., Potter, D. and O. Spatscheck, "Known CDN Request-Routing Mechanisms", draft-cain-cdnp-known-req-route-00.txt (work in progress), November 2000. [ARCH] Green, M., Cain, B. and G. Tomlinson, "CDN Peering Architectural Overview", draft-green-cdnp-gen-arch-02.txt (work in progress), November 2000. 8. Author's Address: Brad Cain Cereva Networks bcain@cereva.com Oliver Spatscheck AT&T Labs spatsch@research.att.com Martin May Activia Networks Martin.May@activia.net Abbie Barbir Nortel Networks abbieb@nortelnetworks.com 9. Acknowledgements Thanks to the following people for their contributions: John Martin, Nalin Mistry, Mark Day, and Stephen Thomas, Hillary Orman. Copyright Statement "Copyright (C) The Internet Society (date). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Cain, et. al. Expires July 2001 ^L[Page 15] Internet-Draft January 2001 Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into Cain, et. al. Expires July 2001 ^L[Page 16]