Internet Draft
Document: draft-jl-pcdp-00.txt
Expires: August 2001
February 2001
J. Lu
Department007.com
Propagated Content Delivery Protocol
Status of this Memo
This document is an Internet-Draft and is in full conformance
with all provisions of Section 10 of RFC2026.
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Abstract
This memo defines a Propagated Content Delivery Protocol (PCDP) for
heavy traffic web sites. The objective of this standard is to:
(1) Reduce unnecessary Internet traffic;
(2) Increase content delivery speed;
(3) Provide extra accessibility of Internet contents in the event
of a Denial of Service (DoS) attack.
Table of Contents
Status of this Memo 1
Abstract 1
Conventions used in this document 2
Existing practice 2
More sensible way 4
Changes required 4
Changes in DNS 5
Changes in content hosting 5
PCDP coordinator 6
Security Considerations 7
References 8
Author's Addresses 8
Overview
Existing popular Internet content delivery (HTTP and FTP content)
methods do not take network routing into consideration. In the past,
HTTP and FTP server producers have mostly focused their attention on
optimizing CPU and memory usage of one end - the server itself.
Efficient usage of routing and Internet infrastructure, which is a
big part of delivery, were often neglected. Although the
interweaving Internet routing MAY not have caused much concern in
the past, a more and more crowded Internet requires better and
better traffic conducting. New design for web servers and new
methods of content delivery, with network efficiency in mind, seem
more useful than ever especially for large Internet content
providers with global audiences. The Propagated Content Delivery
Protocol (PCDP) discussed here is trying to facilitate better
content delivery with propagation and improve traffic management via
changes in DNS practice. Coordination of this new managed delivery
method also provides extra security in the events of denial of DoS.
Conventions used in this document
PCDP (Propagated Content Delivery Protocol)
Content delivery server, web server (a server which host content for
Internet users via HTTP, FTP or other protocols)
PCDP content delivery unit (a content delivery server and all the
DNS servers which directs traffic to it)
Internet Service Provider (ISP)
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in
this document are to be interpreted as described in RFC-2119 [ ].
Existing practice
Because HTTP (and FTP) server designs were separated from network
management (DNS and routing) considerations, most existing Internet
contents were delivered in an inefficient way (figure 1). Client
(http browser or ftp client) gets a DNS resolution (steps 1 and 2)
from authoritative DNS server. Then goes through many hops in
different networks to get the content (1, 2, 3, 4 represents
sequence of events).
|--------|
| Client |
---|--------| ----------
| | | Backbone |
| |--. . .--| Router 1 |
| |----------|
| |
| |
|-----| <-- 1 |
| DNS |---. . .------------| <--3 |----------|
|-----| 2 --> |- -. . .-| Backbone |
| 4 --> | Router 2 |
| ----------
|
|-------------|
| Content |
| Server |
|-------------|
Figure 1. An extreme case of inefficient content delivery
Large content providers try to improve the situation by subscribing
to different Internet Service Providers (ISP) to get better delivery
coverage, but because of the separation between designs of web
server and that of DNS software, best existing practice is multiple
content delivery routes via round-robin (figure 2).
|--------|
| Client |
---|--------| ----------
| | | Backbone |
| |--. . .--| Router 1 |----|-...
| |----------| |
| | |
| | |
|-----| <-- 1 |
| DNS |---. . .--------------| <--3 |----------| |----------|
|-----| Answer of multiple |- -. . .-| Backbone | | Backbone |
IP Address 2 - -> | 4 --> | Router 2 | | Router 3 |
| ---------- ----------
| |
|-------------| |
| Content | ....|
| Server |
|-------------|
Figure 2. Improved content delivery for large web sites.
More sensible way
In short, an Internet client (browser or FTP client) should always
go to its closest server to get content (Figure 3). To achieve this
ideal content delivery scenario, DNS should return one unambiguous
answer for the client to go to, which should be the closest content
server. Also, because contents are to be delivered in a propagated
fashion, it is better for all the content synchronization to be
standardized to allow propagation between servers. To ensure
contents are really available in the backend, another component to
coordinate the DNS and content propagation needs to be in place.
|-----------------|
| A slave content |
| Server |
|-----------------|
|
|--------| 3 <--> 4 |
| Client |--. . . . --
---|--------| ----------
| | | Backbone |
| |--. . .--| Router 1 |----|-...
| |----------| |
| | |
| | |
|-----| <-- 1 |
| DNS |---. . .--------------| |----------| |----------|
|-----| Answer of one distinct |- -. . .-| Backbone | | Backbone |
IP Address 2 - -> | | Router 2 | | Router 3 |
| ---------- ----------
| |
|-------------| |
| Content | ....|
| Server |
|-------------|
Figure 3. An efficient way to deliver content
Changes required
For PCDP to work, changes are required in current DNS
implementations. Also, content servers will need to be able to
propagate or synchronize. Finally, a PCDP coordinator will
coordinate content updates and periodically check network conditions
to ensure DNS return accessible routes. Therefore PCDP will require
three new components:
1) Revised DNS implementation;
2) PCDP propagator;
3) PCDP coordinator.
Changes in DNS
Existing DNS does round robin for load balancing. This "load
balancing" scheme MAY sometimes direct a web browser or FTP client
to remote networks unnecessarily. PCDP require "A" records for the
HTTP or FTP servers to be returned based on the order of network
"closeness". When a "close" server is not accessible, then it fails
back to the old way of round robin. Other existing DNS functions
are unchanged to ensure total compatibility with DNS servers who do
not need PCDP. This component requires modification of DNS software
source code, such as BIND. Also the format of DNS configuration
files will need slight change as well.
For a domain "foo.com", a current (round robin) record will be in
the format of:
www IN A x.x.x.x
www IN A y.y.y.y
www IN A z.z.z.z
A PCDP enabled record should be in the format of:
; If client from xx.0.0.0 or xy.0.0.0, then return xx.xy.xx.xx
www IN A xx.xy.xx.xx {xx.0.0.0, xy.0.0.0}
; If client from yy.yx.0.0 or yx.0.0.0, then return yy.yx.yy.yy
www IN A yy.yx.yy.yy {yy.yx.0.0, yx.0.0.0}
; If client from z.0.0.0 or zx.z.0.0, then return z.z.z.z
www IN A z.z.z.z {z.0.0.0 or zx.z.0.0}
; For clients that did not appear in {}, return xx.yy.xx.xx
; then yy.yx.yy.yy and z.z.z.z
Such a DNS server when queried, should always try to compare the
incoming IP against the entries in {}. If a close match is found,
return one single record, else return records according to round
robin algorithm.
When zone transfer happens, if the slave DNS server is PCDP
compliant, then transfer the DNS entries with the information in {}.
Otherwise, transfer the DNS entries without information in {}.
Changes in content hosting
PCDP tries to minimize the disturbance to the existing web contents.
Therefore a propagator can be designed as a separate piece from
existing HTTP or FTP server. However, the propagator is an integral
part of the content hosting function. It depends on server
configurations from co-existing HTTP or FTP servers to determine
contents that need to be propagated. These contents usually include
files and database. Configuration of propagators should define at
least the following items:
1) Master or slave: Master propagator's functions include monitoring
updates (both files and database) and propagate to slaves. Slaves
receive files and updates and buffer write to the co-existing
HTTP or FTP servers' content volume. Both master and slave have
the responsibility of ensuring they are communicating to the
correct server in a secured fashion;
2) Propagation method: The methods of propagation can vary, with the
overriding concern being the accuracy and security of the
propagated content. Following are a few recommended practices:
a. NFS sharing via secured private network;
b. Encrypted (secure) FTP when above is not available;
c. Synchronization of database via secured connections and ports.
Security of propagation should be implemented at application layer.
This approach allows multiple means of securing propagated content,
for instance, encryption of information, denial and permission of
certain IP addresses based on ports.
PCDP coordinator
The purpose for PCDP coordinator is to coordinate DNS master/slave
servers and content propagator master/slave servers. The key to
PCDP is to view DNS and web server (HTTP or FTP) as a big content
delivering unit. Changes of delivery conditions that have taken
place in the master/slave, content or DNS server needs to be
coordinated.
When network conditions are normal, PCDP enabled servers should just
deliver content based on network closeness. PCDP coordinator should
just be idling and monitoring.
When network outage or DoS happens, they can be 3 scenarios in terms
of an PCDP delivering unit (a DNS/Content server "pair" - this is a
pair in logic, reality might be 3 DNS server all pointing to one
content server for one set of client IP).
DNS down, Content server down;
DNS up, content server down;
DNS down, content server down.
Under each condition, PCDP coordinator in other backbone providers'
network should be able to use existing live delivering units and
reroute traffic to them.
The functions for PCDP coordinator includes:
1) Monitor activities of other DNS and content servers;
2) Make DNS changes and reload the new DNS configuration.
To accomplish 2), the most secure way is for PCDP coordinator
resides on the same server as DNS server, running as the same user
as DNS user. It is not necessary for each content server to have a
PCDP coordinator, but each DNS server should have a PDCP coordinator
as companion.
Security Considerations
Security about DNS
PCDP should have equal level of security as DNS implementations such
as BIND, since it does not interact with operating system more than
existing DNS implementation.
Security about propagation
Based on different method of propagation, there will be different
level of security.
Security benefit under DoS
Under a DoS attack, content will still be available for Internet
users, unless multiple content/DNS servers are all compromised in
different ISP's networks.
References
RFC 2119 Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997
2 DNS and BIND, Cricket Liu, Paul Albitz, Mike Loukides 482 pages
3rd edition (September 1998) O'Reilly & Associates; ISBN:
1565925122
Author's Addresses
John Lu
Department007.com
8476 164A ST Phone: 1-604-575-2620
Surrey, BC Canada Email: johnjlu@yahoo.com