DECADE Y. Gu Internet-Draft Huawei Intended status: Informational D. Bryan Expires: November 18, 2011 Cogent Force, LLC / Huawei Y. Yang Yale University R. Alimi Google May 17, 2011 DECADE Requirements draft-ietf-decade-reqs-02 Abstract The target of DECoupled Application Data Enroute (DECADE) is to provide an open and standard in-network storage system for applications, primarily P2P applications, to store, retrieve and manage their data. This draft enumerates and explains requirements, not only for store and retrieve, but also for data management, access control and resource control, that should be considered during the design and implementation of a DECADE system. These are requirements on the entire system; some of the requirements may eventually be implemented by an existing protocol with/without some extensions (e.g., the data transport level). A user of DECADE as a complete architecture would be guaranteed complete functionality. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." Gu, et al. Expires November 18, 2011 [Page 1] Internet-Draft DECADE Requirements May 2011 The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on November 18, 2011. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the BSD License. Gu, et al. Expires November 18, 2011 [Page 2] Internet-Draft DECADE Requirements May 2011 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 2. Terminology and Concepts . . . . . . . . . . . . . . . . . . . 5 3. Requirements Structure . . . . . . . . . . . . . . . . . . . . 6 4. Protocol Requirements . . . . . . . . . . . . . . . . . . . . 7 4.1. Requirements . . . . . . . . . . . . . . . . . . . . . . . 7 4.1.1. Overall Protocol Requirements . . . . . . . . . . . . 7 4.1.1.1. Cross-platform Access . . . . . . . . . . . . . . 7 4.1.1.2. Connectivity Concerns . . . . . . . . . . . . . . 7 4.1.1.2.1. NATs and Firewalls . . . . . . . . . . . . . . 7 4.1.1.2.2. Connections to Clients . . . . . . . . . . . . 7 4.1.1.3. Security . . . . . . . . . . . . . . . . . . . . . 8 4.1.1.3.1. Secure Transport . . . . . . . . . . . . . . . 8 4.1.1.3.2. Connections to Clients . . . . . . . . . . . . 8 4.1.1.4. Error and Failure Conditions . . . . . . . . . . . 8 4.1.1.4.1. Overload Condition . . . . . . . . . . . . . . 8 4.1.1.4.2. Insufficient Resources . . . . . . . . . . . . 8 4.1.1.4.3. Unavailable and Deleted Data . . . . . . . . . 9 4.1.1.4.4. Redirection . . . . . . . . . . . . . . . . . 9 4.1.2. Transfer and Latency Requirements . . . . . . . . . . 9 4.1.2.1. Low-Latency Access . . . . . . . . . . . . . . . . 9 4.1.2.2. Indirect Transfer . . . . . . . . . . . . . . . . 10 4.1.2.3. Data Object Size . . . . . . . . . . . . . . . . . 10 4.1.2.4. Communication among In-network Storage Elements . 10 4.1.3. Data Access Requirements . . . . . . . . . . . . . . . 11 4.1.3.1. Reading/Writing Own Storage . . . . . . . . . . . 11 4.1.3.2. Access by Other Users . . . . . . . . . . . . . . 11 4.1.3.3. Negotiable Data Protocol . . . . . . . . . . . . . 11 4.1.3.4. Separation of Data Operations from Application Control . . . . . . . . . . . . . . . . . . . . . 12 4.1.4. Data Management Requirements . . . . . . . . . . . . . 12 4.1.4.1. Agnostic of reliability . . . . . . . . . . . . . 12 4.1.4.2. Time-to-live for Stored Data . . . . . . . . . . . 13 4.1.4.3. Offline Usage . . . . . . . . . . . . . . . . . . 13 4.1.5. Data Naming Requirements . . . . . . . . . . . . . . . 13 4.1.5.1. Unique Names . . . . . . . . . . . . . . . . . . . 13 4.1.6. Resource Control . . . . . . . . . . . . . . . . . . . 14 4.1.6.1. Multiple Applications . . . . . . . . . . . . . . 14 4.1.6.2. Per-Remote-Client, Per-Data Control . . . . . . . 14 4.1.6.3. Server Involvement . . . . . . . . . . . . . . . . 15 4.1.7. Authorization . . . . . . . . . . . . . . . . . . . . 15 4.1.7.1. Per-Remote-Client, Per-Data Read Access . . . . . 15 4.1.7.2. Per-User Write Access . . . . . . . . . . . . . . 15 4.1.7.3. Authorization Checks . . . . . . . . . . . . . . . 16 4.1.7.4. Credentials Not IP-Based . . . . . . . . . . . . . 16 4.1.7.5. Server Involvement . . . . . . . . . . . . . . . . 16 5. Discovery Requirements . . . . . . . . . . . . . . . . . . . . 16 Gu, et al. Expires November 18, 2011 [Page 3] Internet-Draft DECADE Requirements May 2011 5.1. Requirements . . . . . . . . . . . . . . . . . . . . . . . 16 5.1.1. Locating DECADE Servers . . . . . . . . . . . . . . . 17 5.1.2. Support for Clients Behind NATs and Firewalls . . . . 17 5.1.3. Prefer Existing Protocols . . . . . . . . . . . . . . 17 6. Storage Requirements . . . . . . . . . . . . . . . . . . . . . 17 6.1. Requirements . . . . . . . . . . . . . . . . . . . . . . . 17 6.1.1. Explicit Deletion of Stored Data . . . . . . . . . . . 17 6.1.2. Multiple writing . . . . . . . . . . . . . . . . . . . 18 6.1.3. Multiple reading . . . . . . . . . . . . . . . . . . . 18 6.1.4. Reading before completely written . . . . . . . . . . 18 6.1.5. Hints concerning usage stored data . . . . . . . . . . 18 6.1.6. Writing model . . . . . . . . . . . . . . . . . . . . 19 6.1.7. Storage Status . . . . . . . . . . . . . . . . . . . . 19 6.2. Non-Requirements . . . . . . . . . . . . . . . . . . . . . 19 6.2.1. No ability to update . . . . . . . . . . . . . . . . . 20 7. Implementation Considerations . . . . . . . . . . . . . . . . 20 7.1. Resource Scheduling . . . . . . . . . . . . . . . . . . . 20 7.2. Removal of Duplicate Records . . . . . . . . . . . . . . . 20 8. Discussion and Open Issues . . . . . . . . . . . . . . . . . . 21 8.1. Discussion . . . . . . . . . . . . . . . . . . . . . . . . 21 8.2. Open Issues . . . . . . . . . . . . . . . . . . . . . . . 21 9. Security Considerations . . . . . . . . . . . . . . . . . . . 22 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 11.1. Normative References . . . . . . . . . . . . . . . . . . . 22 11.2. Informative References . . . . . . . . . . . . . . . . . . 22 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . . 22 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 Gu, et al. Expires November 18, 2011 [Page 4] Internet-Draft DECADE Requirements May 2011 1. Introduction The object of DECoupled Application Data Enroute (DECADE) is to provide an open and standard in-network storage system for applications, primarily applications that could be implemented using a content distribution paradigm, where data is broken in to one or more chunks and then distributed. This may already include many types of applications including P2P applications, IPTV, and VoD. Instead of always transferring data directly from a source/owner client to a requesting client, the source/owner client can store and manage its content on its in-network storage. The requesting client can get the address of the in-network storage pertaining to the source/owner client and retrieve data from the storage. This draft enumerates and explains the rationale behind SPECIFIC requirements on the protocol design and on any data store implementation that may be used to implement DECADE servers that should be considered during the design and implementation of a DECADE system. As such, it DOES NOT include general guiding principals. General design considerations, explanation of the problem being addressed, and enumeration of the types of applications to which DECADE may be suited is not considered in this document. For general information, please see the problem statement [I-D.ietf-decade-problem-statement] and architecture drafts. This document enumerates the requirements to enable target applications to utilize in-network storage. In this context, using storage resources includes not only basic capabilities such as storing and retrieving data, and managing data, but also (1) controlling access for particular remote clients with which it is sharing data and (2) controlling the resources used by remote clients when they access data. 2. Terminology and Concepts This document uses terms defined in [I-D.ietf-decade-problem-statement]. In particular, IAP refers to the In-network storage Access Protocol, which is the protocol used to communicate between a DECADE client and DECADE server (in-network storage) for access control and resource control. This document also defines additional terminology: Target Application: An application (typically installed at end-hosts) with the ability to explicitly control usage of network and/or storage resources to deliver contents to a large number of users. This includes scenarios where multiple applications or entities Gu, et al. Expires November 18, 2011 [Page 5] Internet-Draft DECADE Requirements May 2011 cooperate, such as with P2P/CDN hybrid architectures. Such applications distribute large contents (e.g., a large file, or video stream) by dividing the contents into smaller blocks for more flexible distribution (e.g., multipath). The distributed content is typically immutable (though it may be deleted). We use the term Target Application to refer to the type of applications that are explicitly (but not exclusively) supported by DECADE. 3. Requirements Structure The DECADE protocol is intended to sit between Target Applications and a back-end storage system. In the development of DECADE, it must be made clear that the intention is to NOT develop yet another storage system, but rather to create a protocol that enables Target Applications to make use of storage within the network, leaving specific storage system considerations to the implementation of the DECADE servers as much as possible. For this reason, we have divided the requirements into two categories: o Protocol Requirements: Protocol requirements for Target Applications to make use of in-network storage within their own data dissemination schemes. Development of these requirements is guided by a study of data access, search and management capabilities used by Target Applications. These requirements may be met by a new protocol to be defined within the DECADE Working Group. o Storage Requirements: Functional requirements necessary for the back-end storage system employed by the DECADE server. Development of these requirements is guided by a study of the data access patterns used by Target Applications. These requirements should be met by the underling data transport used by DECADE. It should also be made clear that the approach is to make DECADE a simple protocol, while still enabling its usage within many Target Applications. For this reason, and to further reinforce the distinction between DECADE and a storage system, in some cases we also highlight the non-requirements of the protocol. These non- requirements are intended to capture behaviors that will NOT be assumed to be needed by DECADE's Target Applications and hence not present in the DECADE protocol. Finally, some implementation considerations are provided, which while strictly are not requirements, are intended to provide guidance and highlight potential points of concern that need to be considered by the protocol developers, and later by implementors. Gu, et al. Expires November 18, 2011 [Page 6] Internet-Draft DECADE Requirements May 2011 4. Protocol Requirements 4.1. Requirements 4.1.1. Overall Protocol Requirements 4.1.1.1. Cross-platform Access REQUIREMENT(S): If DECADE supports the ability to store metadata associated with data objects, the DECADE protocol(s) MUST transmit any metadata using an operating system-independent and architecture-independent format. RATIONALE: If DECADE supports the possibility for storing metadata (e.g., a description, uploaded date, object size, or access control list), a possible use for the metadata is to help a DECADE client locate a desired data object. Data objects may be stored by DECADE clients running on various platforms. To enable metadata to be readable regardless of its source it must be transmitted to and from the DECADE server in a standard format. 4.1.1.2. Connectivity Concerns 4.1.1.2.1. NATs and Firewalls REQUIREMENT(S): DECADE SHOULD be usable across firewalls and NATs without requiring additional network support (e.g., Application- level Gateways). RATIONALE: Firewalls and NATs are widely used in the Internet today, both in ISP networks and within households. Deployment of DECADE must not require modifications to such devices (beyond, perhaps, reconfiguration). Note that this requirement applies to both any new protocol developed by the DECADE Working Group and any data transport used with DECADE. 4.1.1.2.2. Connections to Clients REQUIREMENT(S): DECADE SHOULD require that network connections be made from DECADE clients to DECADE servers (i.e., not to the DECADE client). RATIONALE: Many household networks and operating systems have firewalls and NATs configured by default. To ease deployment by avoiding configuration changes and help mitigate security risks, DECADE should not require clients to listen for any incoming network connections (beyond what is required by any other already-deployed application). Gu, et al. Expires November 18, 2011 [Page 7] Internet-Draft DECADE Requirements May 2011 4.1.1.3. Security 4.1.1.3.1. Secure Transport REQUIREMENT(S): DECADE MUST contain a mode in which all communication between a DECADE client and server is over a secure transport that provides confidentiality, integrity, and authentication. RATIONALE: Target Applications may wish to store sensitive data. To satisfy this use case, DECADE must provide a mode in which all communication between a DECADE client and server occurs over a secure transport protocol (e.g., SSL/TLS). 4.1.1.3.2. Connections to Clients REQUIREMENT(S): DECADE SHOULD require that network connections be made from DECADE clients to DECADE servers (i.e., not to the DECADE client). RATIONALE: Many household networks and operating systems have firewalls and NATs configured by default. To ease deployment by avoiding configuration changes and help mitigate security risks, DECADE should not require clients to listen for any incoming network connections (beyond what is required by any other already-deployed application). 4.1.1.4. Error and Failure Conditions 4.1.1.4.1. Overload Condition REQUIREMENT(S): In-network storage, which is operating close to its capacity limit (e.g., too busy servicing other requests), MUST be able to reject requests and not be required to generate responses to additional requests. RATIONALE: When in-network storage is operating at a limit where it may not be able to process additional requests, it should not be required to generate responses to such additional requests. Forcing the in-network storage to do so can impair its ability to service existing requests. 4.1.1.4.2. Insufficient Resources Gu, et al. Expires November 18, 2011 [Page 8] Internet-Draft DECADE Requirements May 2011 REQUIREMENT(S): DECADE MUST support an error condition indicating to a DECADE client that resources (e.g., storage space) were not available to service a request (e.g., storage quota exceeded when attempting to store data). RATIONALE: The currently-used resource levels within the in-network storage are not locally-discoverable, since the resources (disk, network interfaces, etc) are not directly attached. In order to allocate resources appropriately amongst remote clients, a client must be able to determine when resource limits have been reached. The client can then respond by explicitly freeing necessary resources or waiting for such resources to be freed. 4.1.1.4.3. Unavailable and Deleted Data REQUIREMENT(S): DECADE MUST support error conditions indicating that (1) data was rejected from being stored, (2) deleted, or (3) marked unavailable by a storage provider. RATIONALE: Storage providers may require the ability to (1) avoid storing, (2) delete, or (3) quarantine certain data that has been identified as illegal (or otherwise prohibited). DECADE does not indicate how such data is identified, but applications using DECADE should not break if a storage provider is obligated to enforce such policies. Appropriate error conditions should be indicated to applications. 4.1.1.4.4. Redirection REQUIREMENT(S): DECADE SHOULD support the ability for a DECADE server to redirect requests to another DECADE server. This may be in response to either an error or failure condition, or to support capabilities such as load balancing. RATIONALE: A DECADE server may opt to redirect requests to another server to support capabilities such as load balancing, or if the implementation decides that another DECADE server is in a better position to handle the request due to either its location in the network, server status, or other consideration. 4.1.2. Transfer and Latency Requirements 4.1.2.1. Low-Latency Access Gu, et al. Expires November 18, 2011 [Page 9] Internet-Draft DECADE Requirements May 2011 REQUIREMENT(S): DECADE SHOULD provide "low-latency" access for application clients. DECADE MUST allow clients to specify at least two classes of services for service: lowest possible latency and latency non-critical. RATIONALE: Some applications may have requirements on delivery time (e.g., live streaming [PPLive]). The user experience may be unsatisfactory if the use of in-network storage results in lower performance than connecting directly to remote clients over a low-speed, possibly congested uplink. Additionally, the overhead required for control-plane operations in DECADE must not cause the latency to be higher than for a low-speed, possibly congested uplink. While it is impossible to make a guarantee that a system using in-network storage will always outperform a system that does not for every transfer, the overall performance of the system should be improved compared with direct connections, even considering control overhead. 4.1.2.2. Indirect Transfer REQUIREMENT(S): DECADE MUST allow a user's in-network storage to directly fetch from other user's in-network storage. RATIONALE: As an example, a requesting remote client may get the address of the storage pertaining to the source/owner client and then tell its own in-network storage to fetch the content from the source-owner's in-network storage. This helps to avoid extra transfers across ISP network links where possible. 4.1.2.3. Data Object Size REQUIREMENT(S): DECADE MUST allow for efficient data transfer of small objects (e.g., 16KB) between a DECADE client and in-network storage with minimal additional latency required by the protocol. RATIONALE: Though Target Applications are frequently used to share large amounts of data (e.g., continuous streams or large files), the data itself is typically subdivided into smaller chunks that are transferred between clients. Additionally, the small chunks may have requirements on delivery time (e.g., in a live-streaming application). DECADE must enable data to be efficiently transferred amongst clients at this granularity. 4.1.2.4. Communication among In-network Storage Elements Gu, et al. Expires November 18, 2011 [Page 10] Internet-Draft DECADE Requirements May 2011 REQUIREMENT(S): DECADE SHOULD support the ability for two in-network storage elements in different administrative domains to store and/or retrieve data directly between each other. If such a capability is supported, this MAY be the same (or a subset or extension of) as the IAP used by clients to access data. RATIONALE: Allowing server-to-server communication can reduce latency in some common scenarios. Consider a scenario when a DECADE client is downloading data into its own storage from another client's in-network storage. One possibility is for the client to first download the data itself, and then upload it to its own storage. However, this causes unnecessary latency and network traffic. Allowing the data to be downloaded from the remote in-network storage into the client's own in-network storage can alleviate both. 4.1.3. Data Access Requirements 4.1.3.1. Reading/Writing Own Storage REQUIREMENT(S): DECADE MUST support the ability for a DECADE client to read data from and write data to its own in-network storage. RATIONALE: Two basic capabilities for any storage system are reading and writing data. A DECADE client can read data from and write data to in-network storage space that it owns. 4.1.3.2. Access by Other Users REQUIREMENT(S): DECADE MUST support the ability for a user to apply access control policies to users other than itself for its storage. The users with whom access is being shared can be under a different administrative domain than the user who owns the in- network storage. The authorized users may read from or write to the user's storage. RATIONALE: Endpoints in Target Applications may be located across multiple ISPs under multiple administrative domains. Thus, to be useful by Target Applications, DECADE allows a user to specify access control policies for users that may or may not be known to the user's storage provider. 4.1.3.3. Negotiable Data Protocol Gu, et al. Expires November 18, 2011 [Page 11] Internet-Draft DECADE Requirements May 2011 REQUIREMENT(S): DECADE MUST support the ability for a DECADE client to negotiate with its in-network storage about which protocol it can use to read data from and write data to its In-network storage. DECADE MUST specify at least one mandatory protocol to be supported by implementations; usage of a different protocol may be selected via negotiation. RATIONALE: Since typical data transport protocols (e.g., NFS and WebDAV) already provide read and write operations for network storage, it may not be necessary for DECADE to define such operations in a new protocol. However, because of the particular application requirements and deployment considerations, different applications may support different protocols. Thus, a DECADE client must be able to select an appropriate protocol also supported by the in-network storage. This requirement also follows as a result of the requirement of Separation of Control and Data Operations (Section 4.1.3.4). 4.1.3.4. Separation of Data Operations from Application Control REQUIREMENT(S): The DECADE IAP MUST only provide a minimal set of core operations to support diverse policies implemented and desired by Target Applications. RATIONALE: Target Applications support many complex behaviors and diverse policies to control and distribute data, such as (e.g., search, index, setting permissions/passing authorization tokens). Thus, to support such Target Applications, these behaviors must be logically separated from the data transfer operations (e.g., retrieve, store). Some minimal overlap (for example obtaining credentials needed to encrypt or authorize data transfer using control operations) may be required to be directly specified by DECADE. 4.1.4. Data Management Requirements 4.1.4.1. Agnostic of reliability REQUIREMENT(S): DECADE SHOULD remain agnostic of reliability/ fault-tolerance level offered by storage provider. RATIONALE: Providers of a DECADE service may wish to offer varying levels of service for different applications/users. However, a single compliant DECADE client should be able to use multiple DECADE services with differing levels of service. Gu, et al. Expires November 18, 2011 [Page 12] Internet-Draft DECADE Requirements May 2011 4.1.4.2. Time-to-live for Stored Data REQUIREMENT(S): DECADE MUST support the ability for a DECADE client to indicate a time-to-live value (or expiration time) indicating a length of time until particular data can be deleted by the in- network storage element. RATIONALE: Some data stored by a DECADE client may be usable only within a certain window of time, such as in live-streaming P2P applications. Providing a time-to-live value for stored data (e.g., at the time it is stored) can reduce management overhead by avoiding many 'delete' commands sent to in-network storage. The in-network storage may still keep the data in cache for bandwidth optimization. But this is guided by the privacy policy of the DECADE provider. 4.1.4.3. Offline Usage REQUIREMENT(S): DECADE MAY support the ability for a user to provide authorized access to its in-network storage even if the user has no DECADE applications actively running or connected to the network. RATIONALE: If an application desires, it can authorize remote clients to access its storage even after the application exits or network connectivity is lost. An example use case is mobile scenarios, where a client can lose and regain network connectivity very often. 4.1.5. Data Naming Requirements 4.1.5.1. Unique Names REQUIREMENT(S): DECADE SHOULD use a naming scheme in which the name of a data object is unique from the names of all other data objects with different content. RATIONALE: When storing a data object, a DECADE Client should be able to store it without being concerned over whether an object of the same name already exists, unless the existing object contains the exact same data. Note that it may be reasonable for DECADE to satisfy this requirement probabilistically (e.g., using a hashing mechanism). Gu, et al. Expires November 18, 2011 [Page 13] Internet-Draft DECADE Requirements May 2011 4.1.6. Resource Control 4.1.6.1. Multiple Applications REQUIREMENT(S): DECADE SHOULD support the ability for users to define resource sharing policies for multiple applications (DECADE clients) being run/managed by the user. RATIONALE: A user may own in-network storage and share the in- network storage resources amongst multiple applications. For example, the user may run a video-on-demand application and a live-streaming (or even two different live-streaming applications) application which both make use of the user's in- network storage. The applications may be running on different machines and may not directly communicate. Thus, DECADE should enable the user to determine resource sharing policies between the applications. One possibility is for a user to indicate the particular resource sharing policies between applications out-of-band (not using a standard protocol), but this requirement may manifest itself in passing values over IAP to identify individual applications. Such identifiers can be either user-generated or server-generated and do not need to be registered by IANA. 4.1.6.2. Per-Remote-Client, Per-Data Control REQUIREMENT(S): A DECADE client MUST be able to assign resource quotas to individual remote clients for reading from and writing particular data to its in-network storage within a particular range of time. The DECADE server MUST enforce these constraints. RATIONALE: Target Applications can rely on control of resources on a per-remote-client or per-data basis. For example, application policy may indicate that certain remote clients have a higher bandwidth share for receiving data [LLSB08]. Additionally, certain data (e.g., chunks) may be distributed with a higher priority. As another example, when allowing a remote client to write data to a user's in-network storage, the remote client may be restricted to write only a certain amount of data. Since the client may need to manage multiple clients accessing its data, it should be able to indicate the time over which the granted resources are usable. For example, an expiration time for the access could be indicated to the server after which no resources are granted (e.g., indicate error as access denied). Gu, et al. Expires November 18, 2011 [Page 14] Internet-Draft DECADE Requirements May 2011 4.1.6.3. Server Involvement REQUIREMENT(S): A DECADE client SHOULD be able to indicate to a DECADE server, without itself contacting the server, resource control policies for remote clients' requests. RATIONALE: One important consideration for in-network storage elements is scalability, since a single storage element may be used to support many users. Many Target Applications use small chunk sizes and frequent data exchanges. If such an application employed resource control and contacted the in-network storage element for each data exchange, this could present a scalability challenge for the server as well as additional latency for clients. An alternative is to let requesting users obtain signed resource control policies from the owning user, and then users can then present the policy to the storage directly. This can result in reduced messaging handled by the in-network storage. 4.1.7. Authorization 4.1.7.1. Per-Remote-Client, Per-Data Read Access REQUIREMENT(S): A DECADE Client MUST be able to control which individual remote clients are authorized to read particular data stored on its in-network storage. RATIONALE: A Target Application can control certain application- level policies by sending particular data (e.g., chunks) to certain remote clients. It is important that remote clients not be able to circumvent such decisions by arbitrarily reading any currently-stored data in in-network storage. 4.1.7.2. Per-User Write Access REQUIREMENT(S): A DECADE Client MUST be able to control which individual remote clients are authorized to store data into its in-network storage. RATIONALE: The space managed by a user in in-network storage can be a limited resource. At the same time, it can be useful to allow remote clients to write data directly to a user's in-network storage. Thus, a DECADE client should be able to grant only certain remote clients this privilege. Gu, et al. Expires November 18, 2011 [Page 15] Internet-Draft DECADE Requirements May 2011 Note that it is not (currently) a requirement to check that a remote client stores a particular set of data (e.g., the check that a remote client writes the expected chunk of a file). Enforcing this as a requirement would require a client to know which data is expected (e.g., the full chunk itself or a hash of the chunk) which may not be available in all applications. Checking for a particular hash could be considered as a requirement in the future that could optionally be employed by applications. 4.1.7.3. Authorization Checks REQUIREMENT(S): In-network storage MUST check the authorization of a client before it executes a supplied request. The in-network storage MAY use optimizations to avoid such authorization checks as long as the enforced permissions are the the same. RATIONALE: Authorization granted by a DECADE client are meaningless unless unauthorized requests are denied access. Thus, the in- network storage element must verify the authorization of a particular request before it is executed. 4.1.7.4. Credentials Not IP-Based REQUIREMENT(S): Access MUST be able to be granted on other credentials than the IP address RATIONALE: DECADE clients may be operating on hosts without constant network connectivity or without a permanent attachment address (e.g., mobile devices). To support access control with such hosts, DECADE servers must support access control policies that use information other than IP addresses. 4.1.7.5. Server Involvement REQUIREMENT(S): A DECADE client SHOULD be able to indicate, without contacting the server itself, access control policies for remote clients' requests. RATIONALE: See discussion in Section 4.1.6.3. 5. Discovery Requirements 5.1. Requirements Gu, et al. Expires November 18, 2011 [Page 16] Internet-Draft DECADE Requirements May 2011 5.1.1. Locating DECADE Servers REQUIREMENT(S): The DECADE Discovery mechanism MUST allow a DECADE Client to identify one or more DECADE Servers to which it is authorized to read/write data and to which it may authorize other DECADE Clients to read/write data, or fail if no such DECADE Servers are available. RATIONALE: A basic goal of DECADE is to allow DECADE Clients to read/write data for access by other DECADE Clients or itself. Executing the Discovery process should result in a DECADE Client finding a DECADE Server that it can use for these purposes. It is possible that no such DECADE Servers are available. Note that even if a DECADE Client may not successfully locate a DECADE Server that fits these critieria, it may still read/write data a DECADE Server if authorized by another DECADE Client. 5.1.2. Support for Clients Behind NATs and Firewalls REQUIREMENT(S): The Discovery mechanism MUST support DECADE Clients operating behind NATs and Firewalls without requiring additional network support (e.g., Application-level Gateways). RATIONALE: NATs and Firewalls are prevalent in network deployments, and it is common for Target Applications wishing to include a DECADE Client to be deployed behind these devices. 5.1.3. Prefer Existing Protocols REQUIREMENT(S): The DECADE Server discovery mechanism SHOULD prefer to use existing mechanisms and protocols wherever possible. RATIONALE: Existing protocols such as DNS and DHCP are widespread. Using existing protocols, or combinations of the protocols that have been specified in other contexts, is strictly preferred over developing a new discovery protocol for DECADE. 6. Storage Requirements 6.1. Requirements 6.1.1. Explicit Deletion of Stored Data Gu, et al. Expires November 18, 2011 [Page 17] Internet-Draft DECADE Requirements May 2011 REQUIREMENT(S): DECADE MUST support the ability for a DECADE client to explicitly delete data from its own in-network storage. DECADE MAY have an overwrite flag indicating that an object with the same name should be replaced. RATIONALE: A DECADE client may continually be writing data to its in-network storage. Since there may be a limit (e.g., imposed by the storage provider) to how much total storage can be used, some data may need to be removed to make room for additional data. A DECADE client should be able to explicitly remove particular data. This may be implemented using existing protocols. 6.1.2. Multiple writing REQUIREMENT(S): DECADE MUST NOT allow multiple writers for the same object. Implementations raise an error to one of the writers. RATIONALE: This avoids data corruption in a simple way while remaining efficient. 6.1.3. Multiple reading REQUIREMENT(S): DECADE MUST allow for multiple readers for an object. RATIONALE: One characteristic of Target Applications is the ability to upload an object to multiple clients. Thus, it is natural for DECADE to allow multiple readers to read the content concurrently. 6.1.4. Reading before completely written REQUIREMENT(S): DECADE MAY allow readers to read from objects before they have been completely written. RATIONALE: Some Target Applications (in particular, P2P streaming) can be sensitive to latency. A technique to reduce latency is to remove store-and-forward delays for data objects (e.g., make the object available before it is completely stored). Appropriate handling for error conditions (e.g., a disappearing writer) needs to be specified. 6.1.5. Hints concerning usage stored data Gu, et al. Expires November 18, 2011 [Page 18] Internet-Draft DECADE Requirements May 2011 REQUIREMENT(S): DECADE MAY allow writers of new objects to indicate specific hints concerning how the objects are expected to be used (e.g., access frequency or time to live). RATIONALE: Different Target Applications may have different usage patterns for objects stored at in-network storage. For example, a P2P live streaming application may indicate to a DECADE server that the objects are expected to have a shore time-to-live, but read frequently. The DECADE server may then opt to store the objects in memory instead of in disk. 6.1.6. Writing model REQUIREMENT(S): DECADE MUST provide at least a writing model (while storing an object) that appends data to data already stored. RATIONALE: Depending on the object size (e.g., chunk size) used by a Target Application, the application may need to send data to the DECADE server in multiple packets. To keep implementation simple, the DECADE must at least support the ability to write the data sequentially in the order received. Implementations MAY allow application to write data in an object out-of-order (but MUST NOT overwrite ranges of the object that have already been stored). 6.1.7. Storage Status REQUIREMENT(S): A DECADE client MUST be able to retrieve current resource usage (including list of stored data), resource quotas, and access permissions for its in-network storage. The returned information MUST include resource usage resulting from the client's own usage and usage by other clients that have been authorized to read/write objects or open connections to that client's storage. RATIONALE: The resources used by a client are not directly-attached (e.g., disk, network interface, etc). Thus, the client cannot locally determine how such resources are being used. Before storing and retrieving data, a client should be able to determine which data is available (e.g., after an application restart). Additionally, a client should be able to determine resource availability to better allocate them to remote clients. 6.2. Non-Requirements Gu, et al. Expires November 18, 2011 [Page 19] Internet-Draft DECADE Requirements May 2011 6.2.1. No ability to update REQUIREMENT(S): DECADE SHOULD NOT provide ability to update existing objects. That is, objects are immutable once they are stored. RATIONALE: Reasonable consistency models for updating existing objects would significantly complicate implementation (especially if implementation chooses to replicate data across multiple servers). If a user needs to update a resource, it can store a new resource and then distribute the new resource instead of the old one. 7. Implementation Considerations The intent of this section is to collect discussion items and implementation considerations that have been discovered as this requirements document has been produced. The content of this section will be migrated to an appropriate place as the document and the Working Group progress. 7.1. Resource Scheduling The particular resource scheduling policy may have important ramifications on the performance of applications. This document has explicitly mentioned simultaneous support for both low-latency applications and latency-tolerant applications. Denial of Service attacks may be another risk. For example, rejecting new requests due to overload conditions may introduce the ability to perform a denial of service attack depending on a particular DECADE server's scheduling implementation and resource allocation policies. 7.2. Removal of Duplicate Records There are actually two possible scenarios here. The first is the case of removing duplicates within one particular DECADE server (or logical server). While not a requirement, as it does not impact the protocol and is technically not noticeable on message across the wire, a DECADE server may implement internal mechanisms to monitor for duplicate records and use internal mechanisms to prevent duplication of internal storage. The second scenario is removing duplicates across a distributed set of DECADE servers. This is a more difficult problem, and if the group decides to support this capability, it may require protocol support. See Section 8.2 for more details. Gu, et al. Expires November 18, 2011 [Page 20] Internet-Draft DECADE Requirements May 2011 8. Discussion and Open Issues 8.1. Discussion Sometimes, several logical in-network storages could be deployed on the same physical network device. In this case, in-network storages on the same physical network device can communicate and transfer data through internal communication messages. However in-network storages deployed on different physical network devices SHOULD communicate with in-network storage Access Protocol (IAP). To provide fairness among users, the in-network storage provider should assign resource (e.g., storage, bandwidth, connections) quota for users. This can prevent a small number of clients from occupying large amounts of resources on the in-network storage, while others starve. 8.2. Open Issues Gaming of the Resource Control Mechanism: There has been some discussion of how applications may be able game the scheduling system by manipulating the resource control mechanism, for example by specifying many small peers to increase total throughput. This is a serious concern, and we need to identify specific requirements on the protocol (hopefully independent of particular scheduling/resource control schemes) to help address this. Discovery: There needs to be some mechanism for a user to discover that there is a DECADE service available for their use, and to locate that server. This is particularly important in the case of mobile applications, since the actual servers that are available at any given time may differ. However, the specifics of what mechanisms (DHCP, HTTP page, etc.) have not been discussed, or even if the protocol should specify one or leave it as an implementation detail. This needs to be defined, and specific requirements formulated if needed. Removal of Duplicate Records Across Servers: If the group wishes to allow for automated mechanisms to remove duplicates across a number of separate servers, some protocol support may need to be added. In the case of removing duplicates within a single (logical) DECADE server, this is simply an implementation concern. See Section 7 for more details. Gu, et al. Expires November 18, 2011 [Page 21] Internet-Draft DECADE Requirements May 2011 9. Security Considerations Authorization for access to in-network storage is an important part of the requirements listed in this document. Authorization for access to storage resources and the data itself is important for users to be able to manage and limit distribution of content. For example, a user may only wish to share particular content with certain peers. If the authorization technique implemented in DECADE passes any private information (e.g., user passwords) over the wire, it MUST be passed in a secure way. 10. IANA Considerations There are no IANA considerations with this document. 11. References 11.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 11.2. Informative References [I-D.ietf-decade-problem-statement] Yongchao, S., Zong, N., Yang, Y., and R. Alimi, "DECoupled Application Data Enroute (DECADE) Problem Statement", draft-ietf-decade-problem-statement-00 (work in progress), August 2010. [LLSB08] Levin, D., LaCurts, K., Spring, N., and B. Bhattacharjee, "BitTorrent is an Auction: Analyzing and Improving BitTorrent's Incentives", SIGCOMM 2008, August 2008. [PPLive] "PPLive", . Appendix A. Acknowledgments We would also like to thank Haibin Song for substantial contributions to earlier versions of this document. We would also like to thank Reinaldo Penno, Alexey Melnikov, Rich Woundy, Ning Zong, Roni Even, David McDysan, Boerje Ohlman and Dirk Kutscher for contributions (including some text used verbatim) and general feedback. Gu, et al. Expires November 18, 2011 [Page 22] Internet-Draft DECADE Requirements May 2011 Authors' Addresses Yingjie Gu Huawei No. 101 Software Avenue Nanjing, Jiangsu Province 210012 P.R.China Phone: +86-25-56624760 Email: guyingjie@huawei.com David A. Bryan Cogent Force, LLC / Huawei Email: dbryan@ethernot.org Yang Richard Yang Yale University Email: yry@cs.yale.edu Richard Alimi Google Email: ralimi@google.com Gu, et al. Expires November 18, 2011 [Page 23]