NFSv4 Working Group J. Lentini Internet-Draft C. Everhart Intended status: Standards Track NetApp Expires: January 11, 2010 D. Ellard BBN Technologies R. Tewari M. Naik IBM Almaden July 10, 2009 Administration Protocol for Federated Filesystems draft-ietf-nfsv4-federated-fs-admin-02 Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on January 11, 2010. Copyright Notice Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Lentini, et al. Expires January 11, 2010 [Page 1] Internet-Draft Admin Protocol for Federated Filesystems July 2009 Abstract This document describes the administration protocol for a federated file system that enables file access and namespace traversal across collections of independently administered fileservers. The protocol specifies a set of interfaces by which fileservers and collections of fileservers with different administrators can form a fileserver federation that provides a namespace composed of the filesystems physically hosted on and exported by the constituent fileservers. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Error Definitions . . . . . . . . . . . . . . . . . . . . . . 3 3. Administrator-Initiated Operations . . . . . . . . . . . . . . 5 3.1. Basic Definition . . . . . . . . . . . . . . . . . . . . . 5 3.2. Required Procedures . . . . . . . . . . . . . . . . . . . 7 3.2.1. FEDFS_CREATE_JUNCTION . . . . . . . . . . . . . . . . 8 3.2.2. FEDFS_DELETE_JUNCTION . . . . . . . . . . . . . . . . 9 3.2.3. FEDFS_LOOKUP_FSN . . . . . . . . . . . . . . . . . . . 10 4. Security Considerations . . . . . . . . . . . . . . . . . . . 11 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 6. Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 13 7.1. Normative References . . . . . . . . . . . . . . . . . . . 13 7.2. Informational References . . . . . . . . . . . . . . . . . 14 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . . 14 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 14 Lentini, et al. Expires January 11, 2010 [Page 2] Internet-Draft Admin Protocol for Federated Filesystems July 2009 1. Introduction A federated filesystem enables file access and namespace traversal in a uniform, secure and consistent manner across multiple independent fileservers within an enterprise (and possibly across multiple enterprises) with reasonably good performance. Traditionally, building a namespace that spans multiple fileservers has been difficult for two reasons. First, the fileservers that export pieces of the namespace are often not in the same administrative domain. Second, there is no standard mechanism for the fileservers to cooperatively present the namespace. Fileservers may provide proprietary management tools and in some cases an administrator may be able to use the proprietary tools to build a shared namespace out of the exported filesystems. Relying on vendor- proprietary tools does not work in larger enterprises or when collaborating across enterprises because it is likely that the system will contain fileservers running different software, each with their own protocols, with no common protocol to manage the namespace or exchange namespace information. The requirements for federated namespaces are described in [FEDFS-REQTS]. The filesystem federation protocol described in [FEDFS-NSDB] allows fileservers from different vendors and/or with different administrators to cooperatively build a namespace. This document describes the protocol used by administrators to configure the fileservers and construct the namespace. 2. Error Definitions The results of successful operations will consist of a status of FEDFS_OK. The results of unsuccessful operations will begin with a status, other than FEDFS_OK, that indicates the reason why the operation failed. Many of the error status names and meanings (and the prose for their descriptions) are taken from the specification for NFSv4 [RFC3530]. Note, however, that the literal values for the status codes are different. Note that the status of an unsuccessful operation will generally only indicate the first error encountered during the attempt to execute the operation. Lentini, et al. Expires January 11, 2010 [Page 3] Internet-Draft Admin Protocol for Federated Filesystems July 2009 FEDFS_OK No errors were encountered. The operation was a success. FEDFS_ERR_ACCESS Permission denied. The caller does not have the correct permission to perform the requested operation. FEDFS_ERR_BADCHAR A UTF-8 string contains a character which is not supported by the server in the context in which it being used. FEDFS_ERR_BADXDR The server encountered an XDR decoding error while processing an operation. FEDFS_ERR_EXIST The junction specified already exists. FEDFS_ERR_INVAL Invalid argument for an operation. FEDFS_ERR_IO A hard error occurred while processing the requested operation. FEDFS_ERR_NOSPC The requested operation would have caused the server's filesystem to exceed some limit (for example, if there is a fixed number of junctions per fileset or per server). FEDFS_ERR_NOTDIR The caller specified a non-directory in an operation that requires a directory. FEDFS_ERR_NOTEMPTY The caller specified a directory that is not empty as the operand of an operation that requires an empty directory. FEDFS_ERR_NOTJUNCT The caller specified a path that does not end in a junction as the operand for an operation that requires the last component of the path to be a junction. FEDFS_ERR_NOTLOCAL The caller specified a path that contains a junction in any position other than the last component. FEDFS_ERR_PERM The operation was not allowed because the caller is either not a privileged user or not the owner of an object that would be modified by the operation. FEDFS_ERR_ROFS A modifying operation was attempted on a read-only filesystem. FEDFS_ERR_SVRFAULT An unanticipated non-protocol error occurred on the server. Lentini, et al. Expires January 11, 2010 [Page 4] Internet-Draft Admin Protocol for Federated Filesystems July 2009 3. Administrator-Initiated Operations The RPC protocol used by the administration operations is ONC RPC [RFC5531]. The data structures used for the parameters and return values of these procedures are expressed in this document in XDR [RFC4506]. 3.1. Basic Definition We begin by defining basic constants and structures, in XDR notation, that will be used to specify the types of the RPCs described in the rest of this subsection. The XDR definitions below are formatted to allow the reader to easily extract them from the document. The reader can use the following shell script to extract the definitions: #!/bin/sh grep '^ *///' | sed 's?^ */// ??' | sed 's?^ *///$??' If the above script is stored in a file called "extract.sh", and this document is in a file called "spec.txt", then the reader can do: sh extract.sh < spec.txt > admin1.xdr The effect of the script is to remove leading white space from each line, plus a sentinel sequence of "///". /// enum FedFsStatus { /// FEDFS_OK = 0, /// FEDFS_ERR_ACCESS = 1, /// FEDFS_ERR_BADCHAR = 2, /// FEDFS_ERR_BADXDR = 3, /// FEDFS_ERR_EXIST = 4, Lentini, et al. Expires January 11, 2010 [Page 5] Internet-Draft Admin Protocol for Federated Filesystems July 2009 /// FEDFS_ERR_INVAL = 5, /// FEDFS_ERR_IO = 6, /// FEDFS_ERR_NOSPC = 7, /// FEDFS_ERR_NOTDIR = 8, /// FEDFS_ERR_NOTEMPTY = 9, /// FEDFS_ERR_NOTJUNCT = 10, /// FEDFS_ERR_NOTLOCAL = 11, /// FEDFS_ERR_PERM = 12, /// FEDFS_ERR_ROFS = 13, /// FEDFS_ERR_SVRFAULT = 14 /// }; /// /// typedef opaque FedFsFsnUuid<16>; /// typedef opaque FedFsNsdbName<>; /// typedef opaque FedFsPathName<>; /// /// struct FedFsFsn { /// FedFsFsnUuid fsnUuid; /// FedFsNsdbName nsdbName; /// }; /// /// struct FedFsCreateJunctionArgs { /// FedFsPathName path; /// FedFsFsn fsn; /// }; /// /// union FedFsLookupFsnRes switch (FedFsStatus status) { /// case FEDFS_OK: /// FedFsFsn fsn; /// default: /// void; /// }; /// /// program FEDFS_PROG { /// version FEDFS_VERSION { /// void FEDFS_NULL(void) = 0; /// FedFsStatus FEDFS_CREATE_JUNCTION( /// FedFsCreateJunctionArgs args) = 1; /// FedFsStatus FEDFS_DELETE_JUNCTION( /// FedFsPathName path) = 2; /// FedFsLookupFsnRes FEDFS_LOOKUP_FSN( /// FedFsPathName path) = 3; /// } = 1; /// } = 100418; The basic data types defined above MUST be formatted as follows: Lentini, et al. Expires January 11, 2010 [Page 6] Internet-Draft Admin Protocol for Federated Filesystems July 2009 FedFsFsnUuid: A universally unique identifier (UUID) as described in [RFC4122] as a version 1 UUID. The UUID should be formatted in network byte order. FedFsNsdbName: A variable length UTF-8 string that represents an NSDB's network location in either IPv4, IPv6, or DNS host name notation. FedFsPathName: A variable UTF-8 string that represents a file system path. 3.2. Required Procedures Fileservers that participate as "internal" nodes in the federated namespace MUST provide these procedures: FEDFS_NULL The null RPC, which is included, by convention, in every ONC RPC protocol. FEDFS_CREATE_JUNCTION Create a new junction from some location on the server (defined as a pathname) to an FSN. FEDFS_DELETE_JUNCTION Delete an existing junction from some location on the server (defined as a pathname). FEDFS_LOOKUP_FSN Query the server to discover the current value of the junction (if any) at a given path in the server namespace. The FEDFS_CREATE_JUNCTION, FEDFS_DELETE_JUNCTION, and FEDFS_LOOKUP_FSN operations are described in more detail in the following sections. Fileservers that participate as "leaf" nodes in the namespace (i.e., fileservers that host filesets that are the target of junctions, but that do not contain any junctions) are not required to implement any of these operations. Note that operations that modify the state of a replicated fileset MUST result in the update of all of the replicas in a consistent manner. Ideally all of the replicas SHOULD be updated before any operation returns. If one or more of the replicas are unavailable, the operation MAY succeed, but the changes MUST be applied before the unavailable replicas are brought back online. We assume that replicas are updated via some protocol that permits state changes to be reflected consistently across the set of replicas in such a manner that the replicas will converge to a consistent state within a bounded number of successful message exchanges between the servers hosting the replicas. Lentini, et al. Expires January 11, 2010 [Page 7] Internet-Draft Admin Protocol for Federated Filesystems July 2009 3.2.1. FEDFS_CREATE_JUNCTION This operation creates a junction from a server-relative path to a (potentially) remote fileset named by the given FSN. The junction directory on the server is named by a pathname in the form of a UTF-8 string that has a well-defined interpretation by the server). It is not required that this path be accessible in any other manner (e.g., to a client). This path does not appear in the federated namespace, except by coincidence; there is no requirement that the global namespace parallel the server namespace, nor is it required that this path be relative to the server pseudo-root. It does not need to be a path that is accessible via NFS (although the junction will be of limited utility if the directory specified by the path is not also accessible via NFS). If the fileset is read-only, then this operation SHOULD indicate this with a status of FEDFS_ERR_ROFS. If the path contains an invalid UTF-8 character, then status FEDFS_ERR_BADCHAR must be returned. The path is REQUIRED to exist and be completely local to the server. It MUST NOT contain a junction. If the last component of the path is a junction (i.e., this operation is attempting to create a junction where one already exists), then this operation MUST return the error FEDFS_ERR_EXISTS (even if the requested junction is identical to the current junction). If any other component of the path is a junction, then this operation MUST fail with status FEDFS_ERR_NOTLOCAL. The path may contain a symbolic link (if supported by the local server), but the traversal of the path must remain within the server-local namespace. The last component of the path MUST be an empty directory. If any component of the path does not exist, or the final component is not a directory, then the operation fails with status FEDFS_ERR_INVAL. The server MAY enforce the local permissions on the path, including the final component. If the path cannot be traversed because of insufficient permissions, or the final component is an unexecutable or unwritable directory, then the operation MAY fail with status FEDFS_ERR_ACCESS. The association between the path and the FSN MUST be durable before the operation may return successfully. If the operation return codes indicates success, then the caller may assume that the junction was successfully created and is immediately accessible. Lentini, et al. Expires January 11, 2010 [Page 8] Internet-Draft Admin Protocol for Federated Filesystems July 2009 If successful, subsequent references via NFSv4 [RFC3530] or NFSv4.1 [NFSv4.1] clients to the directory that has been replaced by the junction will result in a referral to a current location of the target fileset [FEDFS-NSDB]. Note that the effective permissions of the directory that is converted, by this operation, into a junction are the permissions of the root directory of the target fileset. The original permissions of the directory (and any other attributes it might have) are subsumed by the junction. Note that this operation does not create a fileset at the location targeted by the junction. If the target fileset does not exist, the junction will still be created. An NFS client will discover the missing fileset when it traverses the junction. 3.2.2. FEDFS_DELETE_JUNCTION This operation removes a junction specified by a server-relative path. As with FEDFS_CREATE_JUNCTION, the junction on the server is named by a pathname in the form of a UTF-8 string that has a well-defined interpretation by the server. It is not required that this path be accessible in any other manner (e.g., to a client). This path does not appear in the federated namespace, except by coincidence; there is no requirement that the global namespace reflect the server namespace, nor is it required that this path be relative to the server pseudo-root. It does not need to be a path that is accessible via NFS. If the fileset is read-only, then this operation SHOULD indicate this with a status of FEDFS_ERR_ROFS. If the path contains an invalid UTF-8 character, then status FEDFS_ERR_BADCHAR must be returned. The path used to delete a junction might not be the same path that was used to create the junction. If the namespace on the server has changed, then the junction may now appear at a different path than where it was created. If there is more than one valid path to the junction, any of them may be used. The path is REQUIRED to exist and be completely local to the server. It MUST NOT contain a junction, except as the final component, which MUST be a junction. If any other component of the path is a junction, then this operation MUST fail with status FEDFS_ERR_NOTLOCAL. If the last component of the path is not a Lentini, et al. Expires January 11, 2010 [Page 9] Internet-Draft Admin Protocol for Federated Filesystems July 2009 junction then this operation MUST return status FEDFS_ERR_INVAL. The path may contain a symbolic link (if supported by the local server), but the traversal of the path must remain within the server-local namespace. The last component of the path MUST be a junction. If any component of the path does not exist, or the final component is not a junction, then the operation fails with status FEDFS_ERR_NOTJUNCT. The server MAY enforce the local permissions on the path, including the final component. If the path cannot be traversed because of insufficient permissions, or the parent directory of the junction unexecutable or unwritable directory, then the operation MAY fail with status FEDFS_ERR_ACCESS. The removal of the association between the path and the FSN MUST be durable before the operation may return successfully. If the operation return codes indicates success, then the caller may assume that the junction was successfully destroyed. The effective permissions and other attributes of the directory that is restored by this operation SHOULD be identical to their value prior to the creation of the junction. 3.2.3. FEDFS_LOOKUP_FSN This operation queries a server to determine whether a given path ends in a junction, and if so, the FSN to which the junction refers. Ordinary NFSv4 operations do not provide any general mechanism to determine whether an object is a junction -- there is no encoding specified by the NFSv4 protocol that can represent this information. As with FEDFS_CREATE_JUNCTION, the pathname must be in the form of a UTF-8 string that has a well-defined interpretation by the server. It is not required that this path be accessible in any other manner (e.g., to a client). This path does not appear in the federated namespace, except by coincidence; there is no requirement that the global namespace reflect the server namespace, nor is it required that this path be relative to the server pseudo-root. It does not need to be a path that is accessible via NFS. If the path contains an invalid UTF-8 character, then status FEDFS_ERR_BADCHAR must be returned. The path used to lookup a junction might not be the same path that was used to create the junction. If the namespace on the server has changed, then a junction may now appear at a different path than Lentini, et al. Expires January 11, 2010 [Page 10] Internet-Draft Admin Protocol for Federated Filesystems July 2009 where it was created. If there is more than one valid path to the junction, any of them may be used. The path is REQUIRED to exist and be completely local to the server. It MUST NOT contain a junction, except as the final component. If any other component of the path is a junction, then this operation MUST fail with status FEDFS_ERR_NOTLOCAL. If the last component of the path is not a junction then this operation MUST return the status FEDFS_ERR_NOTJUNCT. The path may contain a symbolic link (if supported by the local server), but the traversal of the path must remain within the server-local namespace. The server MAY enforce the local permissions on the path, including the final component. If the path cannot be traversed because of insufficient permissions, or the parent directory of the junction unexecutable or unwritable directory, then the operation MAY fail with status FEDFS_ERR_ACCESS. 4. Security Considerations The ONC RPC protocol supports authentication, integrity and privacy via the RPCSEC_GSS framework [RFC2203]. Fileservers which support the FedFS administration protocol described above MUST support RPCSEC_GSS. 5. IANA Considerations A range of ONC RPC program numbers were assigned for use by FedFS using the procedure described in Section 7.3 "Program Number Assignment" of [RFC5531]. The FedFS range is: IETF NFSv4 Working Group - FedFS 100418 - 100421 This document describes version 1 of the ONC RPC program 100418. 6. Glossary Administrator: user with the necessary authority to initiate administrative tasks on one or more servers. Admin entity: A server or agent that administers a collection of fileservers and persistently stores the namespace information. Lentini, et al. Expires January 11, 2010 [Page 11] Internet-Draft Admin Protocol for Federated Filesystems July 2009 Client: Any client that accesses the fileserver data using a supported filesystem access protocol. Federation: A set of server collections and singleton servers that use a common set of interfaces and protocols in order to provide to their clients a federated namespace accessible through a filesystem access protocol. Fileserver: A server exporting a filesystem via a network filesystem access protocol. Fileset: The abstraction of a set of files and their containing directory tree. A fileset is the fundamental unit of data management in the federation. Note that all files within a fileset are descendants of one directory, and that filesets do not span filesystems. Filesystem: A self-contained unit of export for a fileserver, and the mechanism used to implement filesets. The fileset does not need to be rooted at the root of the filesystem, nor at the export point for the filesystem. A single filesystem MAY implement more than one fileset, if the client protocol and the fileserver permit this. Filesystem access protocol: A network filesystem access protocol such as NFSv2 [RFC1094], NFSv3 [RFC1813], NFSv4 [RFC3530], or CIFS. FSL (Fileset location): The location of the implementation of a fileset at a particular moment in time. A FSL MUST be something that can be translated into a protocol-specific description of a resource that a client can access directly, such as a fs_location (for NFSv4), or share name (for CIFS). Note that not all FSLs need to be explicitly exported as long as they are contained within an exported path on the fileserver. FSN (Fileset name): A platform-independent and globally unique name for a fileset. Two FSLs that implement replicas of the same fileset MUST have the same FSN, and if a fileset is migrated from one location to another, the FSN of that fileset MUST remain the same. Junction: A filesystem object used to link a directory name in the current fileset with an object within another fileset. The server-side "link" from a leaf node in one fileset to the root of another fileset. Lentini, et al. Expires January 11, 2010 [Page 12] Internet-Draft Admin Protocol for Federated Filesystems July 2009 Namespace: A filename/directory tree that a sufficiently-authorized client can observe. NSDB (Namespace Database Service): A service that maps FSNs to FSLs. The NSDB may also be used to store other information, such as annotations for these mappings and their components. NSDB Node: The name or location of a server that implements part of the NSDB service and is responsible for keeping track of the FSLs (and related info) that implement a given partition of the FSNs. Referral: A server response to a client access that directs the client to evaluate the current object as a reference to an object at a different location (specified by an FSL) in another fileset, and possibly hosted on another fileserver. The client re-attempts the access to the object at the new location. Replica: A replica is a redundant implementation of a fileset. Each replica shares the same FSN, but has a different FSL. Replicas may be used to increase availability or performance. Updates to replicas of the same fileset MUST appear to occur in the same order, and therefore each replica is self-consistent at any moment. We do not assume that updates to each replica occur simultaneously If a replica is offline or unreachable, the other replicas may be updated. Server Collection: A set of fileservers administered as a unit. A server collection may be administered with vendor-specific software. The namespace provided by a server collection could be part of the federated namespace. Singleton Server: A server collection containing only one server; a stand-alone fileserver. 7. References 7.1. Normative References [RFC1094] Nowicki, B., "NFS: Network File System Protocol specification", RFC 1094, March 1989. [RFC1813] Callaghan, B., Pawlowski, B., and P. Staubach, "NFS Lentini, et al. Expires January 11, 2010 [Page 13] Internet-Draft Admin Protocol for Federated Filesystems July 2009 Version 3 Protocol Specification", RFC 1813, June 1995. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2203] Eisler, M., Chiu, A., and L. Ling, "RPCSEC_GSS Protocol Specification", RFC 2203, September 1997. [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally Unique IDentifier (UUID) URN Namespace", RFC 4122, July 2005. [RFC4506] Eisler, M., "XDR: External Data Representation Standard", STD 67, RFC 4506, May 2006. [RFC5531] Thurlow, R., "RPC: Remote Procedure Call Protocol Specification Version 2", RFC 5531, May 2009. 7.2. Informational References [FEDFS-NSDB] J. Lentini, et al., "NSDB Protocol for Federated Filesystems (Work In Progress)", draft-ietf-nfsv4-federated-fs-admin , 2009. [FEDFS-REQTS] J. Lentini, et al., "Requirements for Federated File Systems (Work In Progress)", draft-ietf-nfsv4-federated-fs-reqts , 2008. [NFSv4.1] S. Shepler, et al., "NFS Version 4 Minor Version 1 (Work In Progress)", draft-ietf-nfsv4-minorversion1 , 2008. [RFC3530] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R., Beame, C., Eisler, M., and D. Noveck, "Network File System (NFS) version 4 Protocol", RFC 3530, April 2003. Appendix A. Acknowledgments We would like to thank Paul Lemahieu of EMC, Robert Thurlow of Sun Microsystems, and Mario Wurzl of EMC for helping to author this document. Lentini, et al. Expires January 11, 2010 [Page 14] Internet-Draft Admin Protocol for Federated Filesystems July 2009 Authors' Addresses James Lentini NetApp 1601 Trapelo Rd, Suite 16 Waltham, MA 02451 US Phone: +1 781-768-5359 Email: jlentini@netapp.com Craig Everhart NetApp 7301 Kit Creek Rd Research Triangle Park, NC 27709 US Phone: +1 919-476-5320 Email: everhart@netapp.com Daniel Ellard BBN Technologies 10 Moulton Street Cambridge, MA 02138 US Phone: +1 617-873-8000 Email: dellard@bbn.com Renu Tewari IBM Almaden 650 Harry Rd San Jose, CA 95120 US Email: tewarir@us.ibm.com Lentini, et al. Expires January 11, 2010 [Page 15] Internet-Draft Admin Protocol for Federated Filesystems July 2009 Manoj Naik IBM Almaden 650 Harry Rd San Jose, CA 95120 US Email: manoj@almaden.ibm.com Lentini, et al. Expires January 11, 2010 [Page 16]