Internet DRAFT - draft-yry-decade-replication
draft-yry-decade-replication
DECADE R. Yang
Internet-Draft Yale University
Intended status: Informational N. Zong
Expires: April 25, 2012 Huawei Technologies
October 23, 2011
DECADE Content Replication and Access
draft-yry-decade-replication-00
Abstract
The DECADE Working Group at IETF is working on introducing standard-
based, application-agnostic in-network storage for content
distribution to improve both network efficiency and application
performance. In this document, we introduce content replication
trees and then discuss approaches on how a replication tree can be
constructed in DECADE. In particular, we introduce concepts
including core-stateless content pull, and stateful, open content
forwarding table.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on April 25, 2012.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
Yang & Zong Expires April 25, 2012 [Page 1]
Internet-Draft DECADE REPLICATION October 2011
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
This document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November
10, 2008. The person(s) controlling the copyright in some of this
material may not have granted the IETF Trust the right to allow
modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of it may
not be created outside the IETF Standards Process, except to format
it for publication as an RFC or to translate it into languages other
than English.
Yang & Zong Expires April 25, 2012 [Page 2]
Internet-Draft DECADE REPLICATION October 2011
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Entities and Terms . . . . . . . . . . . . . . . . . . . . . . 4
3. DECADE Content Replication Tree Structure . . . . . . . . . . 5
3.1. Simple Replication and Access Tree . . . . . . . . . . . . 5
3.2. Recursive Data Replication Tree . . . . . . . . . . . . . 6
4. DECADE Content Replication Tree Construction . . . . . . . . . 7
4.1. Proactive Push vs On-Demand Pull . . . . . . . . . . . . . 8
4.2. Application Control vs DECADE Control . . . . . . . . . . 9
5. On-Demand Pull for DECADE . . . . . . . . . . . . . . . . . . 9
5.1. Core-Stateless: Request Pull Chain . . . . . . . . . . . . 10
5.2. Core-Stateful: Open Content Forwarding Table . . . . . . . 10
6. Hybrid Control Approaches . . . . . . . . . . . . . . . . . . 11
7. Security Considerations . . . . . . . . . . . . . . . . . . . 11
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 11
10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11
10.1. Normative References . . . . . . . . . . . . . . . . . . . 11
10.2. Informative References . . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12
Yang & Zong Expires April 25, 2012 [Page 3]
Internet-Draft DECADE REPLICATION October 2011
1. Introduction
The IETF DECADE architecture [I-D.ietf-decade-arch] enables
applications to control in-network storage (INS) to achieve efficient
content distribution. The DECADE architecture is motivated in the
context of P2P applications to save last-mile uplink bandwidth. But
DECADE may also be applied to a wider range of applications. See
[I-D.ietf-decade-reqs] for a definition of the target applications
supported by DECADE.
In particular, DECADE introduces capabilities where applications can
have explicit control over writing and reading at specified in-
network storage locations. A Standard Data Transport (SDT) protocol
[I-D.ietf-decade-arch] is used to write (upload) data objects to a
DECADE Server and to read (download) data objects from a DECADE
Server.
In this document, we focus on two issues: DECADE Content Replication
and DECADE Content Access. DECADE Content Replication is to
determine, which DECADE Server(s) should store a given piece of
content, when, and from where does a storing DECADE server get the
content? The DECADE Content Access is to determine, for a request
for content from a Content Consumer, which server should serve the
request?
The solution to Content Replication and Access depends on the
setting: the workload and the capability of the Content Distribution
Application that is utilizing DECADE. The effectiveness of a scheme
for in-network storage for content distribution depends on the
overhead of pushing content into in-network storage versus the
benefit of having the content available into in-network storage.
Since this document is a first step, we consider a basic setting
where a single DECADE Service Provider offers to a Content
Distribution Application multiple DECADE Servers distributed across a
network. We focus on a single piece of content, which can be
ingested into DECADE Servers from a Content Provider. Some Content
Consumers will consume the piece of content.
2. Entities and Terms
This document uses entities defined in [I-D.ietf-decade-arch],
including DECADE Server, DECADE Service Provider, DECADE Content
Provider, DECADE Content Consumer, Content Distribution Application,
etc.
This document also defines additional terminology:
Yang & Zong Expires April 25, 2012 [Page 4]
Internet-Draft DECADE REPLICATION October 2011
Application Control Logic: We use this term to refer to the generic
data-flow control logic implemented by a Content Distribution
Application to control which DECADE Server(s) store the piece of data
content, and which DECADE Server serves a request for a piece of
data. This logic can be centralized or distributed.
DECADE Control Logic: This is similar to Application Control Logic,
but implemented by the DECADE Service Provider.
3. DECADE Content Replication Tree Structure
Since we focus on the data flow of a specific single piece of
content, we consider that the Replication and Access data flow forms
a tree.
3.1. Simple Replication and Access Tree
We first show a simple Replication and Access structure, as shown in
below Figure 1.
We identify three types of content data flow links in the tree:
Content Provider -> Content Consumer;
Content Provider -> DECADE Server;
DECADE Server -> Content Consumer.
The sub-tree formed by omitting the Content Consumer nodes is the
Replication tree. Since there are no data flow links among DECADE
Servers, the Replication tree is one level.
Yang & Zong Expires April 25, 2012 [Page 5]
Internet-Draft DECADE REPLICATION October 2011
+------------------+
| Content Provider |
+------------------+
| | |
----------------- | -------
| | |
V V V
+------------+ +-----------+ +-----------+
| Content | | DECADE | | DECADE |
| Consumer | | Server | | Server |
+------------+ +-----------+ +-----------+
| | |
------------- ----------- |
| | |
V V V
+----------+ +----------+ +----------+
| Content | | Content | | Content |
| Consumer | | Consumer | | Consumer |
+----------+ +----------+ +----------+
Figure 1: A Simple Replication and Access Tree.
3.2. Recursive Data Replication Tree
A one-level Replication Tree may not be scalable. For example, if
there is a flash-crowd arrival of Content Consumers, there may not be
enough capacity at the Content Provider to replicate simultaneously
to many DECADE Servers to serve many Content Consumers. What this
implies is that DECADE must support inter-DECADE server replication,
i.e., the capability to replicate the content from one DECADE server
to another DECADE server. We refer to a Replication Tree with inter-
DECADE Server data flow links as a Recursive Replication Tree.
Figure 2 below illustrates an example.
We identify four types of data flow links in the tree:
Content Provider -> Content Consumer;
Content Provider -> DECADE Server;
DECADE Server -> DECADE Server;
DECADE Server -> Content Consumer.
Yang & Zong Expires April 25, 2012 [Page 6]
Internet-Draft DECADE REPLICATION October 2011
+------------------+
| Content Provider |
+------------------+
| |
----------------- |
| |
V V
+----------+ +-----------+
| Content | | DECADE |
| Consumer | | Server |
+----------+ +-----------+
| | |
------------ | -------------
| | |
V V V
+-----------+ +-----------+ +----------+
| DECADE | | DECADE | | DECADE |
| Server | | Server | | Server |
+-----------+ +-----------+ +----------+
| | |
------------- ----------- |
| | |
V V V
+----------+ +----------+ +----------+
| Content | | Content | | Content |
| Consumer | | Consumer | | Consumer |
+----------+ +----------+ +----------+
Figure 2: A Recursive Data Replication and Access Tree.
We believe that DECADE should support the construction of Recursive
Replication trees.
4. DECADE Content Replication Tree Construction
There are multiple design dimensions in constructing a DECADE Content
Replication Tree:
Timing: a tree could be constructed proactively or on-demand;
Controller: Both the Application Control Logic (through a DECADE
API) and the DECADE Control Logic (through DECADE Service Provider
internal optimization) can be involved in constructing a
replication tree to achieve effective content distribution.
Below we discuss the two dimensions in more detail.
Yang & Zong Expires April 25, 2012 [Page 7]
Internet-Draft DECADE REPLICATION October 2011
4.1. Proactive Push vs On-Demand Pull
We identify the following scenarios where a replication link can be
created:
Proactive push: The sender pushes (write/post) a piece of content
to a DECADE Server before Consumer requests;
On-demand pull: A DECADE Server pulls content into itself upon
Consumer request.
Both Proactive Push and On-Demand Pull have their benefits and
issues.
Proactive Push: it may reduce content distribution latency. The push
operation can also be scheduled to utilize under-utilized resources.
An issue of Proactive Push, however, is that the resource waste ratio
can be higher, in particular, if the request patterns of Content
Consumers are difficult to predict. On the other hand, if the
request patterns can be known approximately, e.g., for patch
distribution, this may not be an issue.
On-demand Pull: it can have a much lower ratio of wasted resources.
In particular, if a DECADE Server is on or close to the distribution
path to a Content Consumer, then the replication into the DECADE
Server is essentially piggybacked on the service to the Content
Consumer.
To be more concrete, in Table 1 below, we look at how each entity may
initiate each given type of content replication link.
+------------+---------------+-------------+------------+------------+
| | Content | Content | DECADE | DECADE |
| | Provider -> | Provider -> | Server -> | Server -> |
| | Content | DECADE | DECADE | Content |
| | Consumer | Server | Server | Consumer |
+------------+---------------+-------------+------------+------------+
| Content | Push to the | Push to | Recursive | Recursive |
| Provider | Content | DECADE | push. | push. |
| | Consumer. | Server. | Issue: How | Issue: How |
| | Issue: How | Issue: How | does the | does the |
| | does the | does the | Content | Content |
| | Content | Content | Provider | Provider |
| | Provider know | Provider | select the | select the |
| | to push to | select the | link? | link? |
| | the Content | DECADE | | |
| | Consumer? | Server? | | |
| ========== | ========== | ========== | ========== | ========== |
Yang & Zong Expires April 25, 2012 [Page 8]
Internet-Draft DECADE REPLICATION October 2011
| Content | Pull from | Recursive | Recursive | Pull from |
| Consumer | Content | pull | pull | DECADE |
| | Provider | | | Server |
+------------+---------------+-------------+------------+------------+
Table 1: Proactive Push vs On-Demand Pull.
4.2. Application Control vs DECADE Control
From the below Table 2, one can identify extreme control cases: (1)
Application Control Logic conducts the full control; (2) DECADE
Control Logic provides the full control.
+-------------+------------+--------------+------------+------------+
| | Content | Content | DECADE | DECADE |
| | Provider | Provider -> | Server -> | Server -> |
| | -> Content | DECADE | DECADE | Content |
| | Consumer | Server | Server | Consumer |
+-------------+------------+--------------+------------+------------+
| Application | Send | See left. | See left. | See left. |
| Control | command to | Issue: How | | |
| Logic | sender to | much should | | |
| | push or to | Application | | |
| | receiver | knows about | | |
| | to pull. | DECADE | | |
| | | servers and | | |
| | | request | | |
| | | patterns? | | |
| ========== | ========== | ========== | ========== | ========== |
| DECADE | Send | See left. | See left. | See left. |
| Control | command to | | | |
| Logic | sender to | | | |
| | push or | | | |
| | receiver | | | |
| | to pull. | | | |
+-------------+------------+--------------+------------+------------+
Table 2: Application Control vs DECADE Control.
5. On-Demand Pull for DECADE
The current DECADE architecture document [I-D.ietf-decade-arch]
introduces the method of DECADE GET with a REMOTE_SERVER parameter to
obtain data from a remote DECADE Server. One can consider this
method as a specific pull-based inter-DECADE Server replication
primitive. A challenge of DECADE design is how to support generic
pull-based replication.
Yang & Zong Expires April 25, 2012 [Page 9]
Internet-Draft DECADE REPLICATION October 2011
In this document, we introduce two techniques.
5.1. Core-Stateless: Request Pull Chain
In this design, we introduce the concept of a pull chain in DECADE
GET method. An example of such a pull chain is shown in Figure 3:
+-----------+------------+------------+-------------+
| Object ID | NextHop S1 | NextHop S2 | NextHop ... |
+-----------+------------+------------+-------------+
Figure 3: Pull Chain in DECADE GET.
This pull chain can be used by a DECADE Client or Server to
instantiate a request to pull a data object with Object ID.
Specifically, if the DECADE Server receiving this request does not
have the data, it will pull from NextHop S1, who may pull from
NextHop S2 if it does not have, etc.
The construction of the pull chain is implemented by either the
Application Control Logic or the DECADE Control Logic.
5.2. Core-Stateful: Open Content Forwarding Table
In this design, we introduce the novel concept of DECADE Content
Forwarding Table at DECADE Servers. An example data plane structure
of a DECADE Server with Content Forwarding Table is shown in Figure
4:
+-----------+--------------+---------------+
| Object ID | NextHop | DECADE Data |
--------------------------------------------
| | | | |
----------------+---------------------------
| | | * | |
| | | * | |
| | | * | |
----------------+---------------------------
| | | | |
----------------+---------------------------
| | | |
----------------+-----------
| | | * |
| | | * |
| | V * |
----------------------------
| | |
----------------------------
Figure 4: DECADE Data Plane with Open Content Forwarding Table.
Yang & Zong Expires April 25, 2012 [Page 10]
Internet-Draft DECADE REPLICATION October 2011
This Open Content Forwarding Table is used by DECADE Server to
forward the content request based on looking-up the Next Hop of the
request. If this is the approach, then DECADE needs to:
GAP 1: Open an interface to write the Content Forwarding Table;
GAP 2: Consider the issue of hierarchical object ID space to allow
construction of compact content forwarding table.
6. Hybrid Control Approaches
One design option of DECADE is that a DECADE Provider partitions its
DECADE Servers into zones, where each zone has multiple DECADE
Servers. One hybrid control mechanism is that the Application
Control Logic controls the data replication at the zone level and
DECADE Control Logic controls intra zone replication.
7. Security Considerations
TBA
8. IANA Considerations
This document does not have any IANA considerations.
9. Acknowledgements
TBA
10. References
10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
10.2. Informative References
[RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter,
L., Leach, P., and T. Berners-Lee, "Hypertext Transfer Protocol --
HTTP/1.1", RFC 2616, June 1999.
[RFC3744] Clemm, G., Reschke, J., Sedlar, E., and J. Whitehead, "Web
Yang & Zong Expires April 25, 2012 [Page 11]
Internet-Draft DECADE REPLICATION October 2011
Distributed Authoring and Versioning (WebDAV) Access Control
Protocol", RFC 3744, May 2004.
[RFC4331] Korver, B. and L. Dusseault, "Quota and Size Properties for
Distributed Authoring and Versioning (DAV) Collections", RFC 4331,
February 2006.
[RFC4709] Reschke, J., "Mounting Web Distributed Authoring and
Versioning (WebDAV) Servers", RFC 4709, October 2006.
[RFC4918] Dusseault, L., "HTTP Extensions for Web Distributed
Authoring and Versioning (WebDAV)", RFC 4918, June 2007.
[I-D.ietf-decade-problem-statement] Song, H., Zong, N., Yang, Y., and
R. Alimi, "DECoupled Application Data Enroute (DECADE) Problem
Statement", draft-ietf-decade-problem-statement-04 (work in
progress), October 2011.
[I-D.ietf-decade-survey] Alimi, R., Rahman, A., and Y. Yang, "A
Survey of In- network Storage Systems", draft-ietf-decade-survey-06
(work in progress), August 2011.
[I-D.ietf-decade-reqs] Yingjie, G., Bryan, D., Yang, Y., and R.
Alimi, "DECADE Requirements", draft-ietf-decade-reqs-04 (work in
progress), September 2011.
[I-D.ietf-decade-arch] Alimi, R., Yang. Y., Rahman, A., Kutscher,
D., and H. Liu, "DECADE Architecture", draft-ietf-decade-arch-03
(work in progress), September 2011.
[GoogleStorageDevGuide] "Google Storage Developer Guide",
<http://code.google.com/ apis/storage/docs/developer-guide.html>.
Authors' Addresses
Richard Yang
Yale University
Email: yry@cs.yale.edu
Ning Zong
Huawei Technologies
Email: zongning@huawei.com
Yang & Zong Expires April 25, 2012 [Page 12]
