DECADE R. Yang Internet-Draft Yale University Intended status: Informational N. Zong Expires: April 25, 2012 Huawei Technologies October 23, 2011 DECADE Content Replication and Access draft-yry-decade-replication-00 Abstract The DECADE Working Group at IETF is working on introducing standard- based, application-agnostic in-network storage for content distribution to improve both network efficiency and application performance. In this document, we introduce content replication trees and then discuss approaches on how a replication tree can be constructed in DECADE. In particular, we introduce concepts including core-stateless content pull, and stateful, open content forwarding table. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on April 25, 2012. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect Yang & Zong Expires April 25, 2012 [Page 1] Internet-Draft DECADE REPLICATION October 2011 to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English. Yang & Zong Expires April 25, 2012 [Page 2] Internet-Draft DECADE REPLICATION October 2011 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Entities and Terms . . . . . . . . . . . . . . . . . . . . . . 4 3. DECADE Content Replication Tree Structure . . . . . . . . . . 5 3.1. Simple Replication and Access Tree . . . . . . . . . . . . 5 3.2. Recursive Data Replication Tree . . . . . . . . . . . . . 6 4. DECADE Content Replication Tree Construction . . . . . . . . . 7 4.1. Proactive Push vs On-Demand Pull . . . . . . . . . . . . . 8 4.2. Application Control vs DECADE Control . . . . . . . . . . 9 5. On-Demand Pull for DECADE . . . . . . . . . . . . . . . . . . 9 5.1. Core-Stateless: Request Pull Chain . . . . . . . . . . . . 10 5.2. Core-Stateful: Open Content Forwarding Table . . . . . . . 10 6. Hybrid Control Approaches . . . . . . . . . . . . . . . . . . 11 7. Security Considerations . . . . . . . . . . . . . . . . . . . 11 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 11 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11 10.1. Normative References . . . . . . . . . . . . . . . . . . . 11 10.2. Informative References . . . . . . . . . . . . . . . . . . 11 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12 Yang & Zong Expires April 25, 2012 [Page 3] Internet-Draft DECADE REPLICATION October 2011 1. Introduction The IETF DECADE architecture [I-D.ietf-decade-arch] enables applications to control in-network storage (INS) to achieve efficient content distribution. The DECADE architecture is motivated in the context of P2P applications to save last-mile uplink bandwidth. But DECADE may also be applied to a wider range of applications. See [I-D.ietf-decade-reqs] for a definition of the target applications supported by DECADE. In particular, DECADE introduces capabilities where applications can have explicit control over writing and reading at specified in- network storage locations. A Standard Data Transport (SDT) protocol [I-D.ietf-decade-arch] is used to write (upload) data objects to a DECADE Server and to read (download) data objects from a DECADE Server. In this document, we focus on two issues: DECADE Content Replication and DECADE Content Access. DECADE Content Replication is to determine, which DECADE Server(s) should store a given piece of content, when, and from where does a storing DECADE server get the content? The DECADE Content Access is to determine, for a request for content from a Content Consumer, which server should serve the request? The solution to Content Replication and Access depends on the setting: the workload and the capability of the Content Distribution Application that is utilizing DECADE. The effectiveness of a scheme for in-network storage for content distribution depends on the overhead of pushing content into in-network storage versus the benefit of having the content available into in-network storage. Since this document is a first step, we consider a basic setting where a single DECADE Service Provider offers to a Content Distribution Application multiple DECADE Servers distributed across a network. We focus on a single piece of content, which can be ingested into DECADE Servers from a Content Provider. Some Content Consumers will consume the piece of content. 2. Entities and Terms This document uses entities defined in [I-D.ietf-decade-arch], including DECADE Server, DECADE Service Provider, DECADE Content Provider, DECADE Content Consumer, Content Distribution Application, etc. This document also defines additional terminology: Yang & Zong Expires April 25, 2012 [Page 4] Internet-Draft DECADE REPLICATION October 2011 Application Control Logic: We use this term to refer to the generic data-flow control logic implemented by a Content Distribution Application to control which DECADE Server(s) store the piece of data content, and which DECADE Server serves a request for a piece of data. This logic can be centralized or distributed. DECADE Control Logic: This is similar to Application Control Logic, but implemented by the DECADE Service Provider. 3. DECADE Content Replication Tree Structure Since we focus on the data flow of a specific single piece of content, we consider that the Replication and Access data flow forms a tree. 3.1. Simple Replication and Access Tree We first show a simple Replication and Access structure, as shown in below Figure 1. We identify three types of content data flow links in the tree: Content Provider -> Content Consumer; Content Provider -> DECADE Server; DECADE Server -> Content Consumer. The sub-tree formed by omitting the Content Consumer nodes is the Replication tree. Since there are no data flow links among DECADE Servers, the Replication tree is one level. Yang & Zong Expires April 25, 2012 [Page 5] Internet-Draft DECADE REPLICATION October 2011 +------------------+ | Content Provider | +------------------+ | | | ----------------- | ------- | | | V V V +------------+ +-----------+ +-----------+ | Content | | DECADE | | DECADE | | Consumer | | Server | | Server | +------------+ +-----------+ +-----------+ | | | ------------- ----------- | | | | V V V +----------+ +----------+ +----------+ | Content | | Content | | Content | | Consumer | | Consumer | | Consumer | +----------+ +----------+ +----------+ Figure 1: A Simple Replication and Access Tree. 3.2. Recursive Data Replication Tree A one-level Replication Tree may not be scalable. For example, if there is a flash-crowd arrival of Content Consumers, there may not be enough capacity at the Content Provider to replicate simultaneously to many DECADE Servers to serve many Content Consumers. What this implies is that DECADE must support inter-DECADE server replication, i.e., the capability to replicate the content from one DECADE server to another DECADE server. We refer to a Replication Tree with inter- DECADE Server data flow links as a Recursive Replication Tree. Figure 2 below illustrates an example. We identify four types of data flow links in the tree: Content Provider -> Content Consumer; Content Provider -> DECADE Server; DECADE Server -> DECADE Server; DECADE Server -> Content Consumer. Yang & Zong Expires April 25, 2012 [Page 6] Internet-Draft DECADE REPLICATION October 2011 +------------------+ | Content Provider | +------------------+ | | ----------------- | | | V V +----------+ +-----------+ | Content | | DECADE | | Consumer | | Server | +----------+ +-----------+ | | | ------------ | ------------- | | | V V V +-----------+ +-----------+ +----------+ | DECADE | | DECADE | | DECADE | | Server | | Server | | Server | +-----------+ +-----------+ +----------+ | | | ------------- ----------- | | | | V V V +----------+ +----------+ +----------+ | Content | | Content | | Content | | Consumer | | Consumer | | Consumer | +----------+ +----------+ +----------+ Figure 2: A Recursive Data Replication and Access Tree. We believe that DECADE should support the construction of Recursive Replication trees. 4. DECADE Content Replication Tree Construction There are multiple design dimensions in constructing a DECADE Content Replication Tree: Timing: a tree could be constructed proactively or on-demand; Controller: Both the Application Control Logic (through a DECADE API) and the DECADE Control Logic (through DECADE Service Provider internal optimization) can be involved in constructing a replication tree to achieve effective content distribution. Below we discuss the two dimensions in more detail. Yang & Zong Expires April 25, 2012 [Page 7] Internet-Draft DECADE REPLICATION October 2011 4.1. Proactive Push vs On-Demand Pull We identify the following scenarios where a replication link can be created: Proactive push: The sender pushes (write/post) a piece of content to a DECADE Server before Consumer requests; On-demand pull: A DECADE Server pulls content into itself upon Consumer request. Both Proactive Push and On-Demand Pull have their benefits and issues. Proactive Push: it may reduce content distribution latency. The push operation can also be scheduled to utilize under-utilized resources. An issue of Proactive Push, however, is that the resource waste ratio can be higher, in particular, if the request patterns of Content Consumers are difficult to predict. On the other hand, if the request patterns can be known approximately, e.g., for patch distribution, this may not be an issue. On-demand Pull: it can have a much lower ratio of wasted resources. In particular, if a DECADE Server is on or close to the distribution path to a Content Consumer, then the replication into the DECADE Server is essentially piggybacked on the service to the Content Consumer. To be more concrete, in Table 1 below, we look at how each entity may initiate each given type of content replication link. +------------+---------------+-------------+------------+------------+ | | Content | Content | DECADE | DECADE | | | Provider -> | Provider -> | Server -> | Server -> | | | Content | DECADE | DECADE | Content | | | Consumer | Server | Server | Consumer | +------------+---------------+-------------+------------+------------+ | Content | Push to the | Push to | Recursive | Recursive | | Provider | Content | DECADE | push. | push. | | | Consumer. | Server. | Issue: How | Issue: How | | | Issue: How | Issue: How | does the | does the | | | does the | does the | Content | Content | | | Content | Content | Provider | Provider | | | Provider know | Provider | select the | select the | | | to push to | select the | link? | link? | | | the Content | DECADE | | | | | Consumer? | Server? | | | | ========== | ========== | ========== | ========== | ========== | Yang & Zong Expires April 25, 2012 [Page 8] Internet-Draft DECADE REPLICATION October 2011 | Content | Pull from | Recursive | Recursive | Pull from | | Consumer | Content | pull | pull | DECADE | | | Provider | | | Server | +------------+---------------+-------------+------------+------------+ Table 1: Proactive Push vs On-Demand Pull. 4.2. Application Control vs DECADE Control From the below Table 2, one can identify extreme control cases: (1) Application Control Logic conducts the full control; (2) DECADE Control Logic provides the full control. +-------------+------------+--------------+------------+------------+ | | Content | Content | DECADE | DECADE | | | Provider | Provider -> | Server -> | Server -> | | | -> Content | DECADE | DECADE | Content | | | Consumer | Server | Server | Consumer | +-------------+------------+--------------+------------+------------+ | Application | Send | See left. | See left. | See left. | | Control | command to | Issue: How | | | | Logic | sender to | much should | | | | | push or to | Application | | | | | receiver | knows about | | | | | to pull. | DECADE | | | | | | servers and | | | | | | request | | | | | | patterns? | | | | ========== | ========== | ========== | ========== | ========== | | DECADE | Send | See left. | See left. | See left. | | Control | command to | | | | | Logic | sender to | | | | | | push or | | | | | | receiver | | | | | | to pull. | | | | +-------------+------------+--------------+------------+------------+ Table 2: Application Control vs DECADE Control. 5. On-Demand Pull for DECADE The current DECADE architecture document [I-D.ietf-decade-arch] introduces the method of DECADE GET with a REMOTE_SERVER parameter to obtain data from a remote DECADE Server. One can consider this method as a specific pull-based inter-DECADE Server replication primitive. A challenge of DECADE design is how to support generic pull-based replication. Yang & Zong Expires April 25, 2012 [Page 9] Internet-Draft DECADE REPLICATION October 2011 In this document, we introduce two techniques. 5.1. Core-Stateless: Request Pull Chain In this design, we introduce the concept of a pull chain in DECADE GET method. An example of such a pull chain is shown in Figure 3: +-----------+------------+------------+-------------+ | Object ID | NextHop S1 | NextHop S2 | NextHop ... | +-----------+------------+------------+-------------+ Figure 3: Pull Chain in DECADE GET. This pull chain can be used by a DECADE Client or Server to instantiate a request to pull a data object with Object ID. Specifically, if the DECADE Server receiving this request does not have the data, it will pull from NextHop S1, who may pull from NextHop S2 if it does not have, etc. The construction of the pull chain is implemented by either the Application Control Logic or the DECADE Control Logic. 5.2. Core-Stateful: Open Content Forwarding Table In this design, we introduce the novel concept of DECADE Content Forwarding Table at DECADE Servers. An example data plane structure of a DECADE Server with Content Forwarding Table is shown in Figure 4: +-----------+--------------+---------------+ | Object ID | NextHop | DECADE Data | -------------------------------------------- | | | | | ----------------+--------------------------- | | | * | | | | | * | | | | | * | | ----------------+--------------------------- | | | | | ----------------+--------------------------- | | | | ----------------+----------- | | | * | | | | * | | | V * | ---------------------------- | | | ---------------------------- Figure 4: DECADE Data Plane with Open Content Forwarding Table. Yang & Zong Expires April 25, 2012 [Page 10] Internet-Draft DECADE REPLICATION October 2011 This Open Content Forwarding Table is used by DECADE Server to forward the content request based on looking-up the Next Hop of the request. If this is the approach, then DECADE needs to: GAP 1: Open an interface to write the Content Forwarding Table; GAP 2: Consider the issue of hierarchical object ID space to allow construction of compact content forwarding table. 6. Hybrid Control Approaches One design option of DECADE is that a DECADE Provider partitions its DECADE Servers into zones, where each zone has multiple DECADE Servers. One hybrid control mechanism is that the Application Control Logic controls the data replication at the zone level and DECADE Control Logic controls intra zone replication. 7. Security Considerations TBA 8. IANA Considerations This document does not have any IANA considerations. 9. Acknowledgements TBA 10. References 10.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 10.2. Informative References [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. [RFC3744] Clemm, G., Reschke, J., Sedlar, E., and J. Whitehead, "Web Yang & Zong Expires April 25, 2012 [Page 11] Internet-Draft DECADE REPLICATION October 2011 Distributed Authoring and Versioning (WebDAV) Access Control Protocol", RFC 3744, May 2004. [RFC4331] Korver, B. and L. Dusseault, "Quota and Size Properties for Distributed Authoring and Versioning (DAV) Collections", RFC 4331, February 2006. [RFC4709] Reschke, J., "Mounting Web Distributed Authoring and Versioning (WebDAV) Servers", RFC 4709, October 2006. [RFC4918] Dusseault, L., "HTTP Extensions for Web Distributed Authoring and Versioning (WebDAV)", RFC 4918, June 2007. [I-D.ietf-decade-problem-statement] Song, H., Zong, N., Yang, Y., and R. Alimi, "DECoupled Application Data Enroute (DECADE) Problem Statement", draft-ietf-decade-problem-statement-04 (work in progress), October 2011. [I-D.ietf-decade-survey] Alimi, R., Rahman, A., and Y. Yang, "A Survey of In- network Storage Systems", draft-ietf-decade-survey-06 (work in progress), August 2011. [I-D.ietf-decade-reqs] Yingjie, G., Bryan, D., Yang, Y., and R. Alimi, "DECADE Requirements", draft-ietf-decade-reqs-04 (work in progress), September 2011. [I-D.ietf-decade-arch] Alimi, R., Yang. Y., Rahman, A., Kutscher, D., and H. Liu, "DECADE Architecture", draft-ietf-decade-arch-03 (work in progress), September 2011. [GoogleStorageDevGuide] "Google Storage Developer Guide", . Authors' Addresses Richard Yang Yale University Email: yry@cs.yale.edu Ning Zong Huawei Technologies Email: zongning@huawei.com Yang & Zong Expires April 25, 2012 [Page 12]