Network Working Group M. Green Internet-Draft Entera Expires: March 30, 2001 B. Cain Mirror Image Internet G. Tomlinson Entera September 29, 2000 CDN Peering Architectural Overview draft-green-cdnp-gen-arch-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on March 30, 2001. Copyright Notice Copyright (C) The Internet Society (2000). All Rights Reserved. Abstract This memo presents the general architecture and core building blocks used in the peering of content distribution networks (CDNs). This involves the interconnection of CDNs to create larger virtual CDNs with greater reach, while still retaining the same simple interface to both content providers and viewers. The scope of this work is limited to external interconnections between CDNs and does not address internal mechanisms used within CDNs, which for the purpose of the document are considered to be black boxes. This work is intended to establish an abstract architectural framework to be used Green, et. al. Expires March 30, 2001 [Page 1] Internet-Draft CDNP Architecture September 2000 in the development of protocols, interfaces and system models for standardized, interoperable, peering among CDNs. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Meta Level CDN Peering Architecture . . . . . . . . . . . . 6 3. Redirection Peering System . . . . . . . . . . . . . . . . . 9 3.1 Redirection Overview . . . . . . . . . . . . . . . . . . . . 9 3.2 Request Redirection . . . . . . . . . . . . . . . . . . . . 11 3.3 Redirection Problems to Solve . . . . . . . . . . . . . . . 11 3.4 Requirements . . . . . . . . . . . . . . . . . . . . . . . . 12 3.5 Example . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.5.1 Modified DNS Redirection Model . . . . . . . . . . . . . . 13 3.5.2 DNS Redirection Model using NS records . . . . . . . . . . . 13 3.5.3 DNS Redirection Model using CNAME records . . . . . . . . . 14 3.5.4 Hybrid DNS & Content Aware Redirection Model . . . . . . . . 14 4. Distribution Peering System . . . . . . . . . . . . . . . . 16 4.1 Distribution Overview . . . . . . . . . . . . . . . . . . . 16 4.2 Distribution Models . . . . . . . . . . . . . . . . . . . . 17 4.3 Distribution Components . . . . . . . . . . . . . . . . . . 18 4.4 Distribution Problems to Solve . . . . . . . . . . . . . . . 18 4.4.1 Replication Problems . . . . . . . . . . . . . . . . . . . . 19 4.4.2 Signaling Problems . . . . . . . . . . . . . . . . . . . . . 19 4.4.3 Advertising Problems . . . . . . . . . . . . . . . . . . . . 19 4.5 Distribution Requirements . . . . . . . . . . . . . . . . . 20 4.5.1 Replication Requirements . . . . . . . . . . . . . . . . . . 20 4.5.2 Signaling Requirements . . . . . . . . . . . . . . . . . . . 20 4.5.3 Advertising Requirements . . . . . . . . . . . . . . . . . . 20 5. Accounting Peering System . . . . . . . . . . . . . . . . . 22 5.1 Accounting Overview . . . . . . . . . . . . . . . . . . . . 22 5.2 Accounting Data Types . . . . . . . . . . . . . . . . . . . 23 5.3 Accounting Models . . . . . . . . . . . . . . . . . . . . . 24 5.4 Accounting Problems to Solve . . . . . . . . . . . . . . . . 24 5.5 Accounting Requirements . . . . . . . . . . . . . . . . . . 25 6. Security Considerations . . . . . . . . . . . . . . . . . . 26 6.1 CDN Peering Trust Model . . . . . . . . . . . . . . . . . . 26 6.2 Man in the middle DNS attacks . . . . . . . . . . . . . . . 26 6.3 Logs and legal implications . . . . . . . . . . . . . . . . 26 6.4 Denial of service . . . . . . . . . . . . . . . . . . . . . 26 6.5 Application level access . . . . . . . . . . . . . . . . . . 27 7. Impact on the Internet Architecture . . . . . . . . . . . . 28 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 29 References . . . . . . . . . . . . . . . . . . . . . . . . . 30 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 31 Full Copyright Statement . . . . . . . . . . . . . . . . . . 33 Green, et. al. Expires March 30, 2001 [Page 2] Internet-Draft CDNP Architecture September 2000 1. Introduction In a typical (non-peered) CDN [11], a single service provider operates the REDIRECTION SYSTEM and the DISTRIBUTION SYSTEM. In addition, that service provider has the commercial relationship with the content source (operating the origin server). Typically, the value that this CDN presents to a PUBLISHER is based on the scale and reach of its combined systems. There are practical limits to the scale and reach of any single network. Increasing either scale or reach is ultimately limited by the cost of equipment, the space available for deploying equipment, and/or the demand for that scale/reach of infrastructure. Sometimes a particular audience is tied to a single service provider or a small set of providers by constraints of technology, economics, or law. CDN peering allows different CDNs to share resources so as to provide larger scale and/or reach to each participant than they could otherwise achieve. Although this peering is similar in concept to layer 3 peering between autonomous systems [1], it differs rather fundamentally in that it involves the peering of content delivery according to semantically rich application policies. This memo describes the overall architectural structure and the fundamental building blocks used in the composition of CDN peering. Consult [11] for a description of, and the vocabulary used in, this application domain. A key requirement of the architecture itself, is the it be able to address each of the CDN peering scenarios enumerated in [12]. The scope of this work is limited to external interconnections between CDNs (i.e. INTER-CDN) and does not address internal mechanisms used within CDNs (i.e. INTRA-CDN), which for the purpose of the document are considered to be black boxes. This work is intended to establish an abstract architectural framework to be used in the development of protocols, interfaces and system models for standardized, interoperable peering among CDNs. At the core of CDN peering are three principle architectural elements which constitute the building blocks of the CDN peering system. These elements are REDIRECTION PEERING SYSTEM, DISTRIBUTION PEERING SYSTEM, and ACCOUNTING PEERING SYSTEM. Collectively, they control selection of the delivery CDN, content distribution between peering CDNs, and usage accounting, including billing settlement among the peering CDNs. This work takes into consideration relevant IETF RFCs and IETF works-in-progress. In particular, it is mindful of the end-to-end nature [5][8] of the Internet, the current taxonomy of web replication and caching [9], and the accounting, authorization and Green, et. al. Expires March 30, 2001 [Page 3] Internet-Draft CDNP Architecture September 2000 authentication framework [10] Terms in ALL CAPS are defined in [11]. Green, et. al. Expires March 30, 2001 [Page 4] Internet-Draft CDNP Architecture September 2000 2. Meta Level CDN Peering Architecture The macro architecture revolves around the general premise that individual CDNs are wholly contained within an administrative domain [2] that are composed of either autonomous systems [1] or overlay networks. Although this premise is not essential to the integrity of the macro architecture, it has great bearing on the design center utilized for it. With this in mind, CDN peering involves the interconnection of these administrative domains through layer 5 exterior gateway protocols and machinery. The notion of exterior gateway protocols and machinery has precedence in the IETF in the form of the Border Gateway Protocol (BGP) [4], which serves as a baseline this architecture is modeled upon. In essence, this architecture is a layer 5 analog to the layer 3 architecture established by BGP. Figure 1 contains a system architecture diagram of the core elements involved in CDN peering. +--------------+ /--------------| REDIRECTION | / | PEERING | / /-------------->| SYSTEM* | / / +--------------+ / / ^ / / | / / | / / +--------------+ | | | DISTRIBUTION | V | __| PEERING | +--------+ +-----------+ / | SYSTEM* |<-\ +---------+ | |<---| |<-/ +--------------+ \_| | | CLIENT | | SURROGATE | | ORIGIN | | |--->| |-\ +--------------+ /->| | +--------+ +-----------+ \ | ACCOUNTING |-/ +---------+ -->| PEERING | | SYSTEM* |-\ +---------+ +--------------+ \ | BILLING | ->| ORG. | | | +---------+ Note: * represents core elements of CDN peering Figure 1 System Architecture Elements of a CDN Peering System The System Architecture is comprised of 7 major elements, 3 of which Green, et. al. Expires March 30, 2001 [Page 5] Internet-Draft CDNP Architecture September 2000 constitute the CDN peering system itself. The peering elements are REDIRECTION PEERING SYSTEM, DISTRIBUTION PEERING SYSTEM, and ACCOUNTING PEERING SYSTEM. Correspondingly, the system architecture is a system of systems: 1. The ORIGIN publishes content into the DISTRIBUTION SYSTEM. 2. The DISTRIBUTION PEERING SYSTEM moves content among CDN DISTRIBUTION SYSTEMS. Additionally this system interacts with the REDIRECTION PEERING SYSTEM via feedback ADVERTISEMENTs to assist in the peered CDN selection process for CLIENT requests. 3. The CLIENT requests content from the DISTRIBUTION SYSTEM. 4. The REDIRECTION PEERING SYSTEM directs a REQUEST from a CLIENT to a suitable SURROGATE in a peering CDN. REDIRECTION PEERING SYSTEMs interact with one another via feedback ADVERTISEMENTs in order to keep request routing tables current. 5. The selected SURROGATE delivers the requested content to the CLIENT. Additionally, the SURROGATE sends accounting information for delivered content to the ACCOUNTING PEERING SYSTEM. 6. The ACCOUNTING PEERING SYSTEM aggregates and distills the accounting information into statistics and content detail records for use by the ORIGIN and BILLING ORGANIZATION. 7. The BILLING ORGANIZATION uses the content detail records to settle with each of the parties involved in the content distribution and delivery process. This process has been described in its simplest form in order to present the CDN peering architecture in the most abstract way possible. In reality, this process is more complex when applied to policies, business models and service level agreements that span multiple peering CDNs. The orthogonal core peering systems are discussed in greater depth in Section 3, Section 4 and Section 5. It is important to note that the REDIRECTION PEERING SYSTEM is the only mandatory element for CDN peering to function. A DISTRIBUTION PEERING SYSTEM is needed when the PUBLISHER does not have a NEGOTIATED RELATIONSHIP with every peering CDN. Additionally, an ACCOUNTING PEERING SYSTEM is needed when statistical and usage information is needed in order to satisfy PUBLISHER and/or BILLING ORGANIZATION requirements. Green, et. al. Expires March 30, 2001 [Page 6] Internet-Draft CDNP Architecture September 2000 Additionally, it is important to note that the core elements can be provided by independent administrative domains [2] so long as they have authorized peering relationships (i.e. affiliations) between themselves. Green, et. al. Expires March 30, 2001 [Page 7] Internet-Draft CDNP Architecture September 2000 3. Redirection Peering System The REDIRECTION PEERING SYSTEM represents the request routing function of the CDN peering system. It is responsible for binding CLIENTs to peered CDNs for the delivery of content. It has a dependency upon the DISTRIBUTION PEERING SYSTEM for content location information within the peered CDNs. 3.1 Redirection Overview REDIRECTION SYSTEMs direct CLIENT REQUESTs to a suitable SURROGATE, which is able to service a client request. Many redirection systems direct users to a surrogate that is "closest" to the user on the "least loaded" surrogate. The only requirement of the redirection system is that it direct users to a surrogate which can serve the requested content. Redirection is accomplished through a variety of connection hand-off mechanisms including but not limited to DNS [3] and HTTP [7]/RTSP [6]/etc redirection. REDIRECTION PEERING is the interconnection of two or more REDIRECTION SYSTEMs so as to increase the number of REACHABLE SURROGATEs for at least one of the interconnected systems. In order for a PUBLISHER's CONTENT to be delivered by multiple peering CDNs, it is necessary to federate the CDN REDIRECTION SYSTEMs under the DNS name space of the PUBLISHER object. This federation is accomplished by first delegating the PUBLISHER DNS name space to an AUTHORITATIVE REDIRECTION SYSTEM. The AUTHORITATIVE REDIRECTION SYSTEM subsequently splices each peering CDN REDIRECTION SYSTEM into this DNS name space. Figure 2 contains a architecture diagram of the entities involved in the REDIRECTION PEERING SYSTEM. Green, et. al. Expires March 30, 2001 [Page 8] Internet-Draft CDNP Architecture September 2000 +---------------+ | CLIENT | +---------------+ | | (Redirection Tree Root) +---------------+ | AUTHORITATIVE | | REDIRECTION | | SYSTEM | +---------------+ | | INTER-CDN Redirection /----------------/ \-----------------\ | | (1st Level) +--------------+ +--------------+ .........| CDN |.......... .........| |.......... . | REDIRECTION | . . | *********** | . . | CPG | . . | ***** | . . +--------------+ . . +--------------+ . . INTRA-CDN | | | Redirection . . | | . . /----/ | \-----\ . . /------/ \-----\ . . | | | . . | | . . +-----------+ | +------------+ . . +------------+ +-----------+ . ..| SURROGATE |.|..| SURROGATEs |.. ..| ********** |....| ********* |.. +-----------+ | +------------+ +------------+ +-----------+ | | INTER-CDN Redirection (2nd Level) +--------------+ .........| CDN |.......... . | REDIRECTION | . . | CPG | . . +--------------+ . . INTRA-CDN | | Redirection . . /----/ \-----\ . . | | . . +-----------+ +------------+ . ..| SURROGATE |....| SURROGATEs |.. +-----------+ +------------+ Figure 2 REDIRECTION PEERING SYSTEM Architecture The REDIRECTION PEERING SYSTEM is hierarchical in nature. There exists exactly one redirection tree for each PUBLISHER object. The AUTHORITATIVE REDIRECTION SYSTEM is the root of the redirection tree. There may be only one AUTHORITATIVE REDIRECTION SYSTEM for the DNS name space of a PUBLISHER object. Subordinate to the AUTHORITATIVE REDIRECTION SYSTEM are the REDIRECTION SYSTEMs of the first level peering CDNs. There may exist recursive subordinate REDIRECTION SYSTEMs of additional level peering CDNs. Green, et. al. Expires March 30, 2001 [Page 9] Internet-Draft CDNP Architecture September 2000 This architecture mandates DNS as the only allowable redirection mechanism used between the CLIENT and the AUTHORITATIVE REDIRECTION SYSTEM. This constraint is imposed, since by definition, this system is authoritative for the DNS name space of PUBLISHER objects being distributed by peering CDNs. There are no redirection constraints imposed on subordinate INTER-CDN hand offs, except that the peering REDIRECTION SYSTEMs agree on the mechanism employed. 3.2 Request Redirection The actual "routing" of a client request is through REDIRECTION CPGs. The AUTHORITATIVE REDIRECTION CPG receives the initial client request and redirects the request to an appropriate DISTRIBUTING CDN. This process of INTER-CDN redirection may occur multiple times in a recursive manner between REDIRECTION CPGs until the REDIRECTION SYSTEM arrives at an appropriate DISTRIBUTING CDN to deliver the content. Redirection systems explicitly peer but do not have "interior" knowledge of surrogates from other CDNs. Each CDN operates its own redirection system internally. In this manner, redirection systems peer very much like IP network layer peering. 3.3 Redirection Problems to Solve Specific problems in request redirection needing further investigation include: 1. How do DNS redirection systems redirect a request? If a given CDN is peered with many other CDNs, what are the criteria by which a request is redirected to another CDN? 2. What is the normalized name space for redirection? Because redirection is performed by DNS, it is necessary to have agreed upon standards for the encoding of DNS names. There are many potential elements which may be encoded. Some of these elements are: authoritative agent domain, publisher domain, content type, content length, etc. 3. How are policies communicated between the REDIRECTION SYSTEM and the DISTRIBUTION ADVERTISEMENT system? A given CDN may wish to serve only a given content type or a particular set of users. These types of policies must be communicated between CDNs. 4. What are the redirection protocols in DNS? When a request is routed to a particular REDIRECTION CPG, a clear set of DNS rules and policies must be followed in order to have a workable and predictable system. Green, et. al. Expires March 30, 2001 [Page 10] Internet-Draft CDNP Architecture September 2000 5. How do we protect the REDIRECTION SYSTEM against denial of service attacks? 3.4 Requirements REDIRECTION SYSTEMs require some coupling to DISTRIBUTION SYSTEMs. For example, a CDN may have a REDIRECTION SYSTEM which makes use of its own DISTRIBUTION SYSTEM. The REDIRECTION SYSTEMs may also communicate some information about the DISTRIBUTION SYSTEMs for which they are performing redirection. We assume that there is a peering relationship between REDIRECTION CPGs. This peering relationship at a minimum must exchange a set of CLIENT IP addresses that can be serviced, and a set of information about the DISTRIBUTION SYSTEMs, for which they are performing redirection. Redirection Requirements 1. Single AUTHORITATIVE REDIRECTION SYSTEM for PUBLISHER object DNS name space. 2. Use of DNS redirection mechanism between CLIENT and AUTHORITATIVE REDIRECTION SYSTEM. 3. Assure the redirection tree does not become a cyclic redirection graph. 4. Assure that adjacent redirection systems from different administrative domains (different CDNs) use a compatible redirection mechanism. 5. Assure that adjacent redirection systems from different administrative domains (different CDNs) agree to redirect requests for the CONTENT in question. 3.5 Example In order to provide a greater understanding of the REDIRECTION PEERING SYSTEM, the following four examples are described. While these don't represent all implementations, they are considered to be representative of the most common implementations deployed today. It is important to remember the INTRA-CDN REDIRECTION SYSTEM is opaque to the CDN peering architecture, since it is within the CDN black box. The following examples contain known INTRA-CDN implementations in order to present the reader with a complete scenario of REDIRECTION PEERING. Green, et. al. Expires March 30, 2001 [Page 11] Internet-Draft CDNP Architecture September 2000 3.5.1 Modified DNS Redirection Model This example describes a DNS redirection model that utilizes protocol extensions for proxies between peered REDIRECTION CPGs. DNS is utilized exclusively by the AUTHORITATIVE REDIRECTION SYSTEM for INTER-CDN redirection and exclusively by the CDN REDIRECTION SYSTEM for INTRA-CDN redirection. We assume in this example that the CDN REDIRECTION SYSTEM R2 has a INTER-CDN peering relationship with the AUTHORITATIVE SYSTEM R1 and has informed R1 via a peering protocol, similar to BGP but modified for content routing information, and that some set of addresses including the address of the CLIENT, is in the "redirection set" of R2. We also assume the URL being REQUESTED is contained within a name space authoritatively serviced by R1. When the CLIENT contacts the authoritative DNS server R1 to resolve the URL domain name, R1 determines that the peering R2 needs to perform the INTRA-CDN redirection to one of its SURROGATES for service of the forthcoming CLIENT REQUEST. Since, R2 can not return the NS record, R1 proxies R2 with an DNS protocol extension carrying both the CLIENT address and the domain name of the URI. At this point the redirection process has been delegated to the proper peering CDN for INTRA-CDN redirection. R2 runs its request routing computation as though the CLIENT had directly contacted it, and returns the result of the selected SURROGATE to R1, which in turn passes it on to the CLIENT. At this point the CLIENT has the correct SURROGATE to connect with for DELIVERY of the CONTENT. 3.5.2 DNS Redirection Model using NS records This example describes a pure DNS redirection model. DNS is utilized exclusively by the AUTHORITATIVE REDIRECTION SYSTEM for INTER-CDN redirection and exclusively by the CDN REDIRECTION SYSTEM for INTRA-CDN redirection. We assume in this example that the CDN REDIRECTION SYSTEM R2 has a INTER-CDN peering relationship with the AUTHORITATIVE SYSTEM R1. We also assume that the DNS name used by the PUBLISHER contains at least as many name levels as the INTER-CDN redirection tree is deep. In our case, for example, using only R1 and R2 the name could be foo2.foo1.com . When the CLIENT request's a URL, the DNS resolution request will contain the domain name foo2.foo1.com. To resolve this domain name the client site DNS server will first contact the REDIRECTION SYSTEM of R1 since it is authoritative for the domain foo1.com. The REDIRECTION SYSTEM R1 has to decide now if it wants to serve the content from one of its own SURROGATEs or if the content should be Green, et. al. Expires March 30, 2001 [Page 12] Internet-Draft CDNP Architecture September 2000 served from the CDN with REDIRECTION SYSTEM R2. If R1 wants to serve the content it returns an A record with the IP address of the appropriate SURROGATE. If R1 decides R2 should serve the content, it returns a NS record to the client site DNS server, denying a recursive resolution and pointing the client site DNS server to the REDIRECTION SYSTEM of R2. R2 can now decide which SURROGATE to use and returns an appropriate A record. At this point, the CLIENT has the correct SURROGATE to connect with for DELIVERY of the CONTENT. 3.5.3 DNS Redirection Model using CNAME records This example describes a pure DNS redirection model. DNS is utilized exclusively by the AUTHORITATIVE REDIRECTION SYSTEM for INTER-CDN redirection and exclusively by the CDN REDIRECTION SYSTEM for INTRA-CDN redirection. We assume in this example that the CDN REDIRECTION SYSTEM R2 has a INTER-CDN peering relationship with the AUTHORITATIVE SYSTEM R1. When the CLIENT request's a URL, the DNS resolution request will first be made to REDIRECTION SYSTEM R1 since it is the authoritative REDIRECTION SYSTEM. The REDIRECTION SYSTEM R1 has to decide now if it wants to serve the content from one of its own SURROGATEs or if the content should be served from the CDN with REDIRECTION SYSTEM R2. If R1 wants to serve the content it returns an A record with the IP address of the appropriate SURROGATE. If R1 decides R2 should serve the content it returns a CNAME record to the CLIENT site DNS server denying a recursive resolution. In this case the AUTHORITATIVE REDIRECTION SYSTEM for the CNAME (not the original name requested by the CLIENT) has to be R2. R2 can now decide which SURROGATE to use and returns an appropriate A record. At this point, the CLIENT has the correct SURROGATE to connect with for DELIVERY of the CONTENT. This scheme allows for an easier introduction of additional redirection levels as the NS scheme described in Section 3.5.2. 3.5.4 Hybrid DNS & Content Aware Redirection Model This example describes a hybrid DNS/Content-Aware redirection model. DNS is utilized exclusively by the AUTHORITATIVE REDIRECTION SYSTEM. DNS is used in the initial SURROGATE selection process by the CDN REDIRECTION SYSTEM, while Content-Aware redirection is employed to further redirect the CLIENT REQUEST to a better SURROGATE based upon the content contained within the CLIENT REQUEST itself. Since there is more semantic information contained within the CLIENT REQUEST than was present in the DNS lookup up, it is possible to more finely target the redirection to a suitable SURROGATE. Green, et. al. Expires March 30, 2001 [Page 13] Internet-Draft CDNP Architecture September 2000 We assume in this example the same R1 and R2 relationship that was present in the previous Section 3.5.1. We also assume the same process that caused R1 to determine R2 needs to perform the INTRA-CDN redirection. However, in this case, R2 doesn't perform the request routing computation, but rather selects a virtual SURROGATE that is in fact a Content-Aware redirection network element. R2 returns the result of the virtual SURROGATE to R1, which in turn passes it on to the CLIENT. The CLIENT connects to the virtual SURROGATE and sends the CLIENT REQUEST. The virtual SURROGATE perform a Content-Aware request routing computation and returns an application level redirect such as a HTTP [7] or RTSP [6] 307 reply to the CLIENT. At this point, the CLIENT has the correct SURROGATE to connect with for DELIVERY of the CONTENT. Green, et. al. Expires March 30, 2001 [Page 14] Internet-Draft CDNP Architecture September 2000 4. Distribution Peering System The DISTRIBUTION PEERING SYSTEM represents the content distribution function of the CDN peering system. It is responsible for moving content from one DISTRIBUTION CPG to another DISTRIBUTION CPG and for supplying content location information to the REDIRECTION PEERING SYSTEM. 4.1 Distribution Overview One goal of the CDN peering system is to move content closer to the client. Typically this is accomplished by replicating content from ORIGIN servers to SURROGATEs which are then used to deliver the content directly to the CLIENT. For example this content replication path may traverse links internal to a content provider's network, then external links to reach the CDN and then links internal to the CDN's network to finally arrive at the surrogate. For the purposes of the CDN peering system we consider only the path between the two networks. In the above example the last server on the content provider's network in the path, and the first server on the CDN's network in the path, must contain DISTRIBUTION CPGs which communicate directly with each other. The DISTRIBUTION CPGs could be located in the ORIGIN server and the SURROGATE server. Thus in the simplest form the ORIGIN server is in direct contact with the SURROGATE. However the DISTRIBUTION CPG in the content provider's network could aggregate content from multiple ORIGIN servers and the DISTRIBUTION CPG in the CDN's network could represent multiple SURROGATEs. These DISTRIBUTION CPGs could then be co-located in an exchange facility. Figure 3 contains a architecture diagram of the entities involved in the DISTRIBUTION PEERING SYSTEM. Green, et. al. Expires March 30, 2001 [Page 15] Internet-Draft CDNP Architecture September 2000 .................................. .................. . Peering CDN . . Peering CDN . +-------+ . +----------+ +------------+ . . +------------+ . +------+ |CLIENT |---|SURROGATE |----|DISTRIBUTION|------|DISTRIBUTION|---|ORIGIN| +-------+ . +----------+ /--| CPG | . /--| CPG | . +------+ . | +------------+ . |. +------------+ . +-------+ . +----------+ | . |.................. |CLIENTs|---|SURROGATEs|-/ . | +-------+ . +----------+ . | . . | .................................. | | .................................. | . Peering CDN . | +-------+ . +----------+ +------------+ . | |CLIENT |---|SURROGATE |----|DISTRIBUTION|---/ +-------+ . +----------+ /--| CPG | . . | +------------+ . +-------+ . +----------+ | . |CLIENTs|---|SURROGATEs|-/ . +-------+ . +----------+ . . . .................................. Figure 3 DISTRIBUTION PEERING SYSTEM Architecture 4.2 Distribution Models Replication advertisement may take place in a layer 5 model similar to the way BGP is used today at layer 3. DISTRIBUTION CPGs could take care of exterior content replication between content providers and CDNs, while at the same time performing content replication interior to their networks in an independent manner. If this model is used then the internal structure of the networks is hidden and the only knowledge of other networks are the locations of DISTRIBUTION CPGs. Replication of content may take place using a push model, or a pull model, or a combination of both. Hierarchical caching, where SURROGATEs, upon getting a cache miss, retrieve CONTENT from a cache higher up the chain, represents the pull model. Replication of CONTENT from ORIGIN servers to replica origin servers represents the push model. Replication of CONTENT from ORIGIN servers to SURROGATEs, in order to pre-populate the caches, also represents the push model. A combination of the two models would be a cache hierarchy which has a replica origin server as its root. DISTRIBUTION CPGs may be located at various points in these models depending on the topologies of the networks involved. Green, et. al. Expires March 30, 2001 [Page 16] Internet-Draft CDNP Architecture September 2000 With CDN peering it may be necessary to replicate content through a network which has no internal SURROGATEs. On one hand it may be possible to do this transparently with no DISTRIBUTION CPGs on the transit network. On the other hand it may be desirable for the transit network to have DISTRIBUTION CPGs. For example add a transit network between the content provider network and the CDN network to the example above. The transit network could have a DISTRIBUTION CPG co-located with the content provider's DISTRIBUTION CPG which acts as a proxy for the CDN. The transit network could also have a DISTRIBUTION CPG co-located with the CDN's DISTRIBUTION CPG which acts as a proxy for the content provider. In a simpler example the transit network could have a single DISTRIBUTION CPG which acts as a proxy for both the content provider and the CDN. Replication of CONTINUOUS MEDIA takes place in a different model from content which has a fixed length CONTENT DATA UNIT, especially in the case of live streaming data. Replication in this case typically takes the form of splitting the live streaming data at various points in the network. In the CDN peering system DISTRIBUTION CPGs could perform this function. In this sense the collection of DISTRIBUTION CPGs would constitute an application layer multicast overlay network. 4.3 Distribution Components The three main components of distribution are replication, signaling and advertising. Each of these is utilized between DISTRIBUTION CPGs belonging to content providers and CDNs. They may also be used between CDNs. The final goal of replication involves moving the content from an ORIGIN server to SURROGATE delivery servers. The immediate goal in CDN peering is moving the content between DISTRIBUTION CPGs. The second component of content distribution is content signaling. Content signaling is the propagation of content meta-data. This meta-data may include such information such as the immediate expiration of content or a change in the expiration time of a given CONTENT DATA UNIT. The third component of content distribution is content advertising. Content providers must be able to advertise content that can be distributed by CDNs and its associated terms. It is important that the advertising of content must be able to aggregate content information. 4.4 Distribution Problems to Solve Some of the problems in distribution revolve around supporting both Green, et. al. Expires March 30, 2001 [Page 17] Internet-Draft CDNP Architecture September 2000 a push model and a pull model for replication of content in that they are not symmetric. The push model is used for pre-loading of content and the pull model is used for on-demand fetching and pre-fetching of content. These models are not symmetric in that the amount of available resources in which to place the content on the target server must be known. In the fetching cases the server that pulls the content knows the available resources on the target server, itself. In the pre-loading case the server that pushes the content must find out the available resources from the target server before pushing the data. 4.4.1 Replication Problems Specific problems in replication needing further investigation include: 1. How do replication systems direct a request? 2. How are policies communicated between the replication systems? 3. What are the replication protocols? 4. Does replication only take place between CPGs? 4.4.2 Signaling Problems Specific problems in content signaling needing further investigation include: 1. How do we represent a content signal? 2. What protocols should be used for content signals? 3. What is a scalable manner for delivering content signals? 4. Do content signals need a virtual distribution system of their own? 4.4.3 Advertising Problems Specific problems in content advertisement needing further investigation include: 1. How do we represent a collection of meta-data in a concise and compressed manner? 2. How do we represent aggregates of meta-data? 3. What protocols to use for the aggregation of this data? Green, et. al. Expires March 30, 2001 [Page 18] Internet-Draft CDNP Architecture September 2000 4. How distributed of an approach should be used for this problem? 5. How do we prevent looping? 4.5 Distribution Requirements Replication systems must have a peering relationship. This peering relationship must exchange sets of aggregated content and its meta-data. Meta-data may change over time independently of the content data and must be exchanged independently as well. Replication systems may require some coupling to redirection systems. It is possible that when fetching content as opposed to pushing content that sessions between replication peering systems may be directed by the redirection system. 4.5.1 Replication Requirements The specific requirements in content replication are: 1. A common protocol for the replication of content. 2. A common format for the actual content data in the protocol. 3. A common format for the content meta-data in the protocol. 4. Security mechanisms. 5. Scalable distribution of the content. 4.5.2 Signaling Requirements The specific requirements in content signaling are: 1. Minimum support for a "flush" and an "expiration time update" signal. 2. Security mechanisms. 3. Scalable distribution of the signals on a large scale. 4.5.3 Advertising Requirements The specific requirements in content advertisement are: 1. A common protocol for the advertisement of content. 2. A common format for the actual advertisements in the protocol. Green, et. al. Expires March 30, 2001 [Page 19] Internet-Draft CDNP Architecture September 2000 3. A well-known state machine. 4. Use of TCP or SCTP (because soft-state protocols will not scale). 5. Well-known error codes to diagnose protocols between different networks. 6. Capability negotiation. 7. Ability to represent policy. Green, et. al. Expires March 30, 2001 [Page 20] Internet-Draft CDNP Architecture September 2000 5. Accounting Peering System The ACCOUNTING PEERING SYSTEM represents the accounting data collection function of the CDN peering system. It is responsible for moving accounting data from one ACCOUNTING CPG to another ACCOUNTING CPG. 5.1 Accounting Overview CDN peering must provide the ability for the content provider to collect data from surrogates which are delivering their content directly to clients. ACCOUNTING CPGs retrieve the data from SURROGATEs which collect and store the data locally. This interior data may be collected from the SURROGATEs by ACCOUNTING CPGs using SNMP or FTP, for example. ACCOUNTING CPGs transfer the data to exterior neighboring ACCOUNTING CPGs on request or in an asynchronous manner. This architecture is only concerned with the latter exchange. Accounting data may also be aggregated before it is transferred. Figure 4 contains a architecture diagram of the entities involved in the ACCOUNTING PEERING SYSTEM. Green, et. al. Expires March 30, 2001 [Page 21] Internet-Draft CDNP Architecture September 2000 ..................................... .................... . Peering CDN . . Peering CDN . . +-----------+ +--------------+ . . +--------------+ . +---------+ . | SURROGATE |----| ACCOUNTING |-------| ACCOUNTING |-----| ORIGIN | . +-----------+ /--| CPG | . ---| CPG |---\ +---------+ . | +--------------+ . / . +--------------+ . | . +-----------+ | . | . . | +---------+ . | SURROGATEs|-/ . | .................... \-| BILLING | . +-----------+ . | | ORG. | . . | +---------+ ..................................... | | ..................................... | . Peering CDN . | . +-----------+ +--------------+ . | . | SURROGATE |----| ACCOUNTING |--/ . +-----------+ /--| CPG | . . | +--------------+ . . +-----------+ | . . | SURROGATEs|-/ . . +-----------+ . . . ..................................... Figure 4 ACCOUNTING PEERING SYSTEM Architecture In addition information needs to be exchanged between ACCOUNTING CPGs in order for SURROGATEs to be able to provide authentication, authorization, and policy enforcement as specified by the content provider. It is possible that this meta-data could be included with the content replicated by the DISTRIBUTION PEERING SYSTEM. However these types of data are grouped together in the work of the Authentication, Authorization and Accounting (AAA) working group in the IETF. This work as well as the work of the Authentication Authorization Accounting Architecture (AAAARCH) research group in the IRTF should be examined to determine their applicability to CDN peering accounting. 5.2 Accounting Data Types Accounting data generally falls into the network or system management, policy, and settlement categories which are gathered to collect information about the use of the content. Network and system management data allows for the monitoring of resources in order to perform load balancing and to observe the conformance to Service Level Agreements (SLAs). Policy information allows for the enforcement of authentication and authorization. Data needed for settlement includes statistics such as proxy hits and misses, Green, et. al. Expires March 30, 2001 [Page 22] Internet-Draft CDNP Architecture September 2000 information from log files, and session detail records for CONTINUOUS MEDIA. 5.3 Accounting Models In one model a third-party BILLING ORGANIZATION must be able to receive the information necessary to bill the appropriate party. In another model this function may be performed by the PUBLISHER or AUTHORITATIVE REDIRECTION SYSTEM. Consult [13] for detailed information on CDN peering billing models. Accounting data may be requested by an ACCOUNTING CPG or supplied asynchronously by another ACCOUNTING CPG. Asynchronous data may be subscribed to or sent in an solicited manner. Guidelines should be set on the amount of accounting data traffic which should be allowed in proportion to the content data and how aggregation of accounting data is performed. Some accounting data may be sensitive to time. Four categories of time sensitive management data have been identified. The first is real-time data that consists of events that require immediate action or attention. The second is data that is needed within 5 seconds of generation such as resource loading or diagnostic data. The third is data that is needed in 5-minute intervals such as statistics. The fourth is data that is needed on a 24 hour or less basis such as logs or billing data. 5.4 Accounting Problems to Solve There are several problems with data retrieval that need to be solved. These include latency, overhead, and large data size. The core set of facilities must provide solutions to these and must consist of collection of data on the server, controlled access to the data, and the aggregation and archival of the data on ACCOUNTING CPGs. Specific problems in accounting data exchange needing further investigation include: 1. How do we represent accounting info for a given object? 2. How do we represent accounting for many media types? 3. How do we aggregate this information? 4. How do we signal upload place or type? 5. How do we aggregate this information "hop-by-hop" back to the BILLING ORGANIZATION? Green, et. al. Expires March 30, 2001 [Page 23] Internet-Draft CDNP Architecture September 2000 5.5 Accounting Requirements The complexity of CDN peering requires that a method be created for the exchange of accounting information. This information must be accurately logged, aggregated and ultimately collected at the BILLING entity for each PUBLISHER. The specific requirements for accounting data exchange are: 1. Simple methods for representing accounting information. 2. Simple methods for aggregating this accounting information. 3. Agreed upon protocols for the uploading and distribution of this information. 4. Agreed upon standardized accounting records. Green, et. al. Expires March 30, 2001 [Page 24] Internet-Draft CDNP Architecture September 2000 6. Security Considerations 6.1 CDN Peering Trust Model The trust model utilized with CDN peering is predicated largely on transitive trust between the ORIGIN, REDIRECTION PEERING SYSTEM, DISTRIBUTION PEERING SYSTEM, ACCOUNTING PEERING SYSTEM and SURROGATES. While it is not within the control of the REDIRECTION PEERING SYSTEM to establish security relationships with CLIENTs, it is possible and necessary to do so for communication between the CPGs. At a minimum the DISTRIBUTION PEERING SYSTEM connections should be protected to the same degree that the IP routing system is secured by BGP. As for the ACCOUNTING PEERING SYSTEM, this is the most sensitive system and thus needs the greatest degree of security. It should be secured commensurate with the components within IETF AAA. In general, the trust model employed within CDN peering is based upon the model adopted by IETF AAA. This model has addressed most, if not all the requirements likely to be faced by CDN peering. 6.2 Man in the middle DNS attacks CDN Peering aggregates many ORIGIN servers into an overlay network. This aggregation exacerbates the man in the middle attacks on its REDIRECTION PEERING SYSTEM. The AUTHORITATIVE PEERING SYSTEM is the most vulnerable as it serves the public at large and must service DNS requests that are not normally protected by a secure channel. This is a common security problem, but one that is potentially amplified due to the degree of name space aggregation being performed. Other components are less susceptible since they are within the controlled system, however care must taken to secure them properly as network elements. 6.3 Logs and legal implications Logs from SURROGATEs should be kept secure, as they are likely more sensitive than a web server, since they may provide information on user patterns for multiple ORIGIN servers. Also, transporting logs across borders may have legal implications. Log handling is restricted by law in some countries. 6.4 Denial of service Any redirection of traffic is susceptible to denial of service attacks at the redirect point. The REDIRECTION PEERING SYSTEM performs traffic redirection and is therefore susceptible to denial of service. The strategies used to counter denial of service are Green, et. al. Expires March 30, 2001 [Page 25] Internet-Draft CDNP Architecture September 2000 dependent upon the redirection mechanism being employed. Careful consideration must be taken to harden these architectural elements to reduce this threat. 6.5 Application level access SURROGATEs, DISTRIBUTION CPGs and ACCOUNTING CPGs are application level components in the traffic flow path, and may give intruders access to information that was previously only available at the network level in an intermediary-free world. Some network level equipment may have required physical access to get sensitive information. Introduction of application level components may require additional system security. Green, et. al. Expires March 30, 2001 [Page 26] Internet-Draft CDNP Architecture September 2000 7. Impact on the Internet Architecture On the face of it, the architectural framework proposed in this paper for adding intermediate DISTRIBUTION SYSTEMs to Internet infrastructure looks like a major change in the end-to-end model [5] that has been so successful in the Internet. However, in this model, the SURROGATEs are delegated by ORIGIN servers to act in their behalf, and thus are the terminating servers for CLIENTs in the end-to-end model. Conceptually, there are multiple end-to-end relationships introduced by peering CDNs, each being linked together by intermediary REDIRECTION SYSTEMs, DISTRIBUTION SYSTEMs, and ACCOUNTING SYSTEMs, duly authorized by the parties involved to provide intermediate services. This model has precedence in the Internet, with the Domain Name Service [3] being a classic example. The value of the end to end model is that the network is simple and transparent [8], so it is easy to add services, and easy to diagnose problems when they occur. With the end to end model, there are only two active entities that count, the CLIENT and the ORIGIN (or requesting peer and replying peer in peer-to-peer services). As previously stated, the end to end model is still in effect for the client and server relationship, thus, preserving transparency in the rendering of services by REDIRECTION SYSTEMs and SURROGATEs to CLIENTs. Green, et. al. Expires March 30, 2001 [Page 27] Internet-Draft CDNP Architecture September 2000 8. Acknowledgements The authors would like to acknowledge the contributions and comments of Mark Day (Cisco), Fred Douglis (AT&T), Don Gilletti (Entera), John Martin (Network Appliance), Raj Nair (Cisco), Doug Potter (Cisco), John Scharber (Entera) and Oliver Spatscheck (AT&T). Green, et. al. Expires March 30, 2001 [Page 28] Internet-Draft CDNP Architecture September 2000 References [1] Hawkinson, J. and T. Bates, "Guidelines for creation, selection, and registration of an Autonomous System (AS)", BCP 6, March 1996, . [2] Hares, S. and D. Katz, "Administrative Domains and Routing Domains A Model for Routing in the Internet", RFC 1136, December 1989, . [3] Postel, J., "Domain Name Structure and Delegation", RFC 1591, March 1994, . [4] Rekhter, Y. and T. Li, "A Border Gateway Protocol 4 (BGP-4)", RFC 1771, March 1995, . [5] Carpenter, B., "Architecture Principles of the Internet", RFC 1958, June 1996, . [6] Schulzrinne, H., Rao, A. and R. Lanphier, "Real Time Streaming Protocol", RFC 2326, April 1998, . [7] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999, . [8] Carpenter, B., "Internet Transparency", RFC 2775, February 2000, . [9] Cooper, I., Melve, I. and G. Tomlinson, "Internet Web Replication and Caching Taxonomy", draft-ietf-wrec-taxonomy-04.txt (work in progress), June 2000, . [10] Volbrecht, J., Calhoun, P., Farrell, S., Gommans, L., Gross, G., de Bruijn, B., de Laat, C., Holdrege, M. and D. Spence, "AAA Authorization Framework", draft-ietf-aaa-authz-arch-00.txt (work in progress), October 1999, . [11] Day, M., Cain, B. and G. Tomlinson, "A Model for CDN Peering", draft-day-cdnp-model-00.txt (work in progress), September 2000, . [12] Day, M. and D. Gilletti, "Content Distribution Network Peering Scenarios", draft-day-cdnp-scenarios-00.txt (work in progress), September 2000, . [13] Gilletti, D., Nair, R. and J. Scharber, "Accounting Models for CDN Peering", draft-gilletti-cdnp-accounting-models-00.txt (work in progress), September 2000, . Authors' Addresses Mark Green Entera, Inc. 40971 Encyclopedia Circle Fremont, CA 94538 US Phone: +1 510 770 5268 EMail: markg@entera.com Brad Cain Mirror Image Internet 49 Dragon Court Woburn, MA 01801 US Phone: +1 781 276 1904 EMail: brad.cain@mirror-image.com Green, et. al. Expires March 30, 2001 [Page 30] Internet-Draft CDNP Architecture September 2000 Gary Tomlinson Entera, Inc. 40971 Encyclopedia Circle Fremont, CA 94538 US Phone: +1 510 580 3726 EMail: garyt@entera.com Green, et. al. Expires March 30, 2001 [Page 31] Internet-Draft CDNP Architecture September 2000 Full Copyright Statement Copyright (C) The Internet Society (2000). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC editor function is currently provided by the Internet Society. Green, et. al. Expires March 30, 2001 [Page 32]