Internet Draft Document: draft-jl-pcdp-01.txt Expires: August 2001 February 2001 J. Lu Department007.com Propagated Content Delivery Protocol Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This memo defines a Propagated Content Delivery Protocol (PCDP) for heavy traffic web sites. The objective of this standard is to: (1) Reduce unnecessary Internet traffic; (2) Increase content delivery speed; (3) Provide extra accessibility of Internet contents in the event of a Denial of Service (DoS) attack. Table of Contents Status of this Memo 1 Abstract 1 Conventions used in this document 2 Existing practice 2 More sensible way 4 Changes required 4 Changes in DNS 5 Changes in content hosting 5 PCDP coordinator 6 Security Considerations 7 References 8 Author's Addresses 8 Overview Existing popular Internet content delivery (HTTP and FTP content) methods do not take network routing into consideration. In the past, HTTP and FTP server producers have mostly focused their attention on optimizing CPU and memory usage of one end - the server itself. Efficient usage of routing and Internet infrastructure, which is a big part of delivery, were often neglected. Although the interweaving Internet routing MAY not have caused much concern in the past, a more and more crowded Internet requires better and better traffic conducting. New design for web servers and new methods of content delivery, with network efficiency in mind, seem more useful than ever especially for large Internet content providers with global audiences. The Propagated Content Delivery Protocol (PCDP) discussed here is trying to facilitate better content delivery with propagation and improve traffic management via changes in DNS practice. Coordination of this new managed delivery method also provides extra security in the events of denial of DoS. Conventions used in this document PCDP (Propagated Content Delivery Protocol) Content delivery server, web server (a server which host content for Internet users via HTTP, FTP or other protocols) PCDP content delivery unit (a content delivery server and all the DNS servers which directs traffic to it) Internet Service Provider (ISP) The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [ ]. Existing practice Because HTTP (and FTP) server designs were separated from network management (DNS and routing) considerations, most existing Internet contents were delivered in an inefficient way (figure 1). Client (http browser or ftp client) gets a DNS resolution (steps 1 and 2) from authoritative DNS server. Then goes through many hops in different networks to get the content (1, 2, 3, 4 represents sequence of events). |--------| | Client | ---|--------| ---------- | | | Backbone | | |--. . .--| Router 1 | | |----------| | | | | |-----| <-- 1 | | DNS |---. . .------------| <--3 |----------| |-----| 2 --> |- -. . .-| Backbone | | 4 --> | Router 2 | | ---------- | |-------------| | Content | | Server | |-------------| Figure 1. An extreme case of inefficient content delivery Large content providers try to improve the situation by subscribing to different Internet Service Providers (ISP) to get better delivery coverage, but because of the separation between designs of web server and that of DNS software, best existing practice is multiple content delivery routes via round-robin (figure 2). |--------| | Client | ---|--------| ---------- | | | Backbone | | |--. . .--| Router 1 |----|-... | |----------| | | | | | | | |-----| <-- 1 | | DNS |---. . .--------------| <--3 |----------| |----------| |-----| Answer of multiple |- -. . .-| Backbone | | Backbone | IP Address 2 - -> | 4 --> | Router 2 | | Router 3 | | ---------- ---------- | | |-------------| | | Content | ....| | Server | |-------------| Figure 2. Improved content delivery for large web sites. More sensible way In short, an Internet client (browser or FTP client) should always go to its closest server to get content (Figure 3). To achieve this ideal content delivery scenario, DNS should return one unambiguous answer for the client to go to, which should be the closest content server. Also, because contents are to be delivered in a propagated fashion, it is better for all the content synchronization to be standardized to allow propagation between servers. To ensure contents are really available in the backend, another component to coordinate the DNS and content propagation needs to be in place. |-----------------| | A slave content | | Server | |-----------------| | |--------| 3 <--> 4 | | Client |--. . . . -- ---|--------| ---------- | | | Backbone | | |--. . .--| Router 1 |----|-... | |----------| | | | | | | | |-----| <-- 1 | | DNS |---. . .--------------| |----------| |----------| |-----| Answer of one distinct |- -. . .-| Backbone | | Backbone | IP Address 2 - -> | | Router 2 | | Router 3 | | ---------- ---------- | | |-------------| | | Content | ....| | Server | |-------------| Figure 3. An efficient way to deliver content Changes required For PCDP to work, changes are required in current DNS implementations. Also, content servers will need to be able to propagate or synchronize. Finally, a PCDP coordinator will coordinate content updates and periodically check network conditions to ensure DNS return accessible routes. Therefore PCDP will require three new components: 1) Revised DNS implementation; 2) PCDP propagator; 3) PCDP coordinator. Changes in DNS Existing DNS does round robin for load balancing. This "load balancing" scheme MAY sometimes direct a web browser or FTP client to remote networks unnecessarily. PCDP require "A" records for the HTTP or FTP servers to be returned based on the order of network "closeness". When a "close" server is not accessible, then it fails back to the old way of round robin. Other existing DNS functions are unchanged to ensure total compatibility with DNS servers who do not need PCDP. This component requires modification of DNS software source code, such as BIND (or complex configuration of "sortlist"). Also the format of DNS configuration files need slight change. For a domain "foo.com", a current (round robin) record will be in the format of: www IN A x.x.x.x www IN A y.y.y.y www IN A z.z.z.z A PCDP enabled record should be in the format of: ; If client from xx.0.0.0 or xy.0.0.0, then return xx.xy.xx.xx www IN A xx.xy.xx.xx {xx.0.0.0, xy.0.0.0} ; If client from yy.yx.0.0 or yx.0.0.0, then return yy.yx.yy.yy www IN A yy.yx.yy.yy {yy.yx.0.0, yx.0.0.0} ; If client from z.0.0.0 or zx.z.0.0, then return z.z.z.z www IN A z.z.z.z {z.0.0.0 or zx.z.0.0} ; For clients that did not appear in {}, return xx.yy.xx.xx ; then yy.yx.yy.yy and z.z.z.z Such a DNS server when queried, should always try to compare the incoming IP against the entries in {}. If a close match is found, return one single record, else return records according to round robin algorithm. When zone transfer happens, if the slave DNS server is PCDP compliant, then transfer the DNS entries with the information in {}. Otherwise, transfer the DNS entries without information in {}. Changes in content hosting PCDP tries to minimize the disturbance to the existing web contents. Therefore a propagator can be designed as a separate piece from existing HTTP or FTP server. However, the propagator is an integral part of the content hosting function. It depends on server configurations from co-existing HTTP or FTP servers to determine contents that need to be propagated. These contents usually include files and database. Configuration of propagators should define at least the following items: 1) Master or slave: Master propagator's functions include monitoring updates (both files and database) and propagate to slaves. Slaves receive files and updates and buffer write to the co-existing HTTP or FTP servers' content volume. Both master and slave have the responsibility of ensuring they are communicating to the correct server in a secured fashion; 2) Propagation method: The methods of propagation can vary, with the overriding concern being the accuracy and security of the propagated content. Following are a few recommended practices: a. NFS sharing via secured private network; b. Encrypted (secure) FTP when above is not available; c. Synchronization of database via secured connections and ports. Security of propagation should be implemented at application layer. This approach allows multiple means of securing propagated content, for instance, encryption of information, denial and permission of certain IP addresses based on ports. PCDP coordinator The purpose for PCDP coordinator is to coordinate DNS master/slave servers and content propagator master/slave servers. The key to PCDP is to view DNS and web server (HTTP or FTP) as a big content delivering unit. Changes of delivery conditions that have taken place in the master/slave, content or DNS server needs to be coordinated. When network conditions are normal, PCDP enabled servers should just deliver content based on network closeness. PCDP coordinator should just be idling and monitoring. When network outage or DoS happens, they can be 3 scenarios in terms of an PCDP delivering unit (a DNS/Content server "pair" - this is a pair in logic, reality might be 3 DNS server all pointing to one content server for one set of client IP). DNS down, content server down; DNS up, content server down; DNS down, content server up. Under each condition, PCDP coordinator in other backbone providers' network should be able to use existing live delivering units and reroute traffic to them. The functions for PCDP coordinator includes: 1) Monitor activities of other DNS and content servers; 2) Make DNS changes and reload the new DNS configuration. To accomplish 2), the most secure way is for PCDP coordinator resides on the same server as DNS server, running as the same user as DNS user. It is not necessary for each content server to have a PCDP coordinator, but each DNS server should have a PDCP coordinator as companion. Security Considerations Security about DNS PCDP should have equal level of security as DNS implementations such as BIND, since it does not interact with operating system more than existing DNS implementation. Security about propagation Based on different method of propagation, there will be different level of security. Security benefit under DoS Under a DoS attack, content will still be available for Internet users, unless multiple content/DNS servers are all compromised in different ISP's networks. References RFC 2119 Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 2 DNS and BIND, Cricket Liu, Paul Albitz, Mike Loukides 482 pages 3rd edition (September 1998) O'Reilly & Associates; ISBN: 1565925122 Author's Addresses John Lu Department007.com 8476 164A ST Phone: 1-604-575-2620 Surrey, BC Canada Email: johnjlu@yahoo.com