NFSv4 R. Tewari Internet Draft M. Naik Intended status: Standards Track IBM Expires: April 2008 D. Ellard C. Everhart Netapp October 31, 2007 Protocol for Federated Filesystems v1.0 draft-tewari-federated-fs-protocol-00.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on April 31, 2008. Copyright Notice Copyright (C) The IETF Trust (2007). Tewari, et al. Expires April 31, 2008 [Page 1] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Abstract This document describes a file system federation protocol that enables file access and namespace traversal across collections of independently administered fileservers. The protocol specifies a set of interfaces by which fileservers and collections of fileservers with different administrators can form a fileserver federation that provides a namespace composed of the filesystems physically hosted on and exported by the constituent fileservers. Tewari, et al. Expires April 31, 2008 [Page 2] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 RFC 2119 Keywords The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 Table of Contents 1. Introduction...................................................5 1.1. Protocol Goals............................................5 2. Overview of Features and Concepts..............................6 2.1. Namespace.................................................6 2.2 Fileset...............................................7 2.2.1 Fileset Location (FSL).............................7 2.3 Namespace Repository (NSDB)..............................9 2.4 Mount Points, Junctions and Referrals....................9 2.5 Federation Root FileServers.............................10 2.6 Federation Root FileSet.................................10 2.7 Fileservers.............................................11 2.8 File-access Clients.....................................11 3 Interaction with NFSv4........................................11 4 Finding the local NSDB........................................11 5 Examples......................................................11 5.1 Create a Fileset and its FSL(s)..........................11 5.1.1 Creating a Fileset and a FSN.........................12 5.1.2 Adding a Replica of a Fileset.......................13 5.2 Junction Resolution...................................13 5.3...........................................................14 Example use case for fileset annotations......................14 6 Error Definitions.............................................14 7 Functions.....................................................16 7.1 ADMIN TO NSDB OPERATIONS ...............................16 7.1.1 FSN_CREATE...........................................16 7.1.2 FSN_DELETE...........................................17 7.1.3 FSN_MOUNT............................................19 7.1.4 FSN_UNMOUNT..........................................20 7.1.5 FSL_CREATE...........................................20 7.1.6 FSL_DELETE...........................................21 7.1.7 FSL_UPDATE ..........................................22 7.1.8 FSL_STAT ............................................22 7.1.9 FSL_FINDBYHOST, FSN_FINDBYHOST, FSL_FINDBYHOSTPATH...23 7.2 FILESERVER TO NSDB OPERATIONS ............................24 7.2.1 FSN_GET_FSL..........................................24 7.3 ADMIN TO FILESERVER OPERATIONS ...........................25 7.3.1 FSN_CREATE_JUNCTION .................................25 7.3.2 FSN_CREATE_EXPORT ...................................25 8 Security Considerations.......................................26 9 IANA Considerations........................................26 10 Conclusions................................................26 11 Glossary...................................................26 Appendix A. rnfs Schema Objects..................................30 Appendix B. Namespace Schema Class Objects.......................31 12 References.................................................32 12.1 Normative References.....................................32 Tewari, et al. Expires April 31, 2008 [Page 3] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Author's Addresses...............................................33 Intellectual Property Statement..................................34 Disclaimer of Validity...........................................34 Tewari, et al. Expires April 31, 2008 [Page 4] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 1. Introduction A federated filesystem enables file access and namespace traversal in a uniform, secure and consistent manner across multiple independent fileservers within an enterprise (and possibly across multiple enterprises) with reasonably good performance. The first requirement of a federated filesystem is the ability to traverse the data exported by different fileservers without requiring a static client configuration. The second requirement is that the location of the data should be dynamically discovered and the discovery process should be transparent to the clients. The third requirement is that it should be possible for all clients, with sufficient privilege, to view the same namespace regardless of the fileserver they connect to. Traditionally, fileserver collections are administered by a single entity. Fileservers may provide proprietary management tools and in some cases an administrator may be able to use the proprietary tools to build a shared namespace out of the exported filesystems. Relying on vendor-proprietary tools does not work in larger enterprises or when collaborating across enterprises because it is likely that the system will contain fileservers running different software, each with their own interfaces, with no common protocol to manage the namespace or exchange namespace information. There may also be independently-administered singleton servers that export some or all of their filesystem resources. A filesystem federation protocol enables the interoperation across multi- vendor fileservers managed by the same administrative entity, across singleton independent fileservers, and across independent administrative entities that may manage a collection of fileservers. The scope of the filesystem federation protocol is limited to NFSv4 capable fileservers. The support for NFSv3 fileservers is optional. 1.1. Protocol Goals The objective of this draft is to specify a set of interfaces by which fileservers and collections of fileservers with different administrators can form a fileserver federation that provides a namespace composed of the filesystems physically hosted on and exported by the fileservers of the federation. It should be possible, using a system that implements the interfaces, to share a common namespace across all the fileservers in the federation. Tewari, et al. Expires April 31, 2008 [Page 5] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 It should also be possible for different fileservers in the federation to project different namespaces and enable clients to traverse them. Such a federation may contain an arbitrary number of namespace repositories, each belonging to a different administrative entity, and each rendering a part of the namespace. Such a federation may also have an arbitrary number of administrative entities responsible for administering disjoint subsets of the fileservers. In the rest of the document the term fileserver implies a fileserver that is part of the federation. A fileserver not part of the federation is called an external fileserver. 2. Overview of Features and Concepts 2.1. Namespace The goal of a unified namespace is to make all managed data available to all clients via the same path in a common filesystem-like namespace. This should be achieved with minimal or zero client configuration. In particular, updates to the common namespace should not require configuration changes at the client. Filesets, which are the unit of data management, are a set of files and directories accessible from a single mount. Depending on the implementation, they may be anything between an individual directory of an exported filesystem to an entire exported filesystem at a fileserver. From the perspective of the clients, the common namespace is constructed by logically mounting filesets that are physically located on different fileservers. The namespace, which is defined in terms of fileset definitions, fileset identifiers, the location of each fileset in the namespace, and the physical location of the implementation(s) of each fileset, is stored in a set of namespace repositories, each managed by an administrative entity. The namespace schema defines the model used for populating, modifying, and querying the namespace repositories. It is not required by the federation that the namespace be common across all fileservers. It should be possible to have several independently rooted namespaces that should permit traversal into another namespace at defined junction points. Tewari, et al. Expires April 31, 2008 [Page 6] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 2.2 Fileset A fileset is defined to be a container of data and is the basic unit of data management. It is uniquely represented by the fileset name (FSN). An FSN is considered unique across the federation. An FSN contains information sufficient to locate the namespace repository (NSDB) that holds authoritative information about it and an identifier, called fsn_uuid, that identifies it on that NSDB. After an FSN is created, it is associated with a fileset location (FSL) on a fileserver. A fileset can be implemented by one or more FSLs. struct FSN { utf8string NSDB_fqdn; uuid_t fsn_uuid; }; The attributes of an FSN are: NFSDB_fqdn: The fully qualified domain name of an NSDB server that contains authoritative information for this FSN. fsn_uuid: a 128 bit uuid (universally unique identifier), conforming to RFC 4122, that is used to uniquely identify an FSN. 2.2.1 Fileset Location (FSL) An FSL represents the location where the fileset data resides. Each FSL maps to a host:path pair on a file server. An FSL may also have additional attributes. Each location has an associated type that determines the protocol(s) that may be used to access its data. Type information can be used to decide the list of locations that will be returned to the client. It also has associated status information. Other attributes associated with an FSL are based on the NFSv4.1 fs_locations_info attribute[RFC3530]. struct FSL { utf8string host_fqdn; utf8string pathname; FSL_ATTR attrs; }; Each FSL consists of: host_fsdn: the name of the host fileserver storing the physical data pathname: the exported pathname at that host fileserver Tewari, et al. Expires April 31, 2008 [Page 7] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 attrs: additional attributes for this FSL, as described in the FSL_ATTR structure struct FSL_ATTR { protocol_t type; int32_t currency; annotation_t annotations<>; fs_status_t status; opaque_t info<>; } The attributes associated with each FSL are: type: the protocol(s) supported by the fileserver hosting this FSL. currency: the time lag of this FSL represented as the number of time units it lags the latest version as defined by the NFSv4.1 fs_locations_info attribute. A currency value of 0 represents the latest version. Currency values are <= 0. annotations: a list of name/value pairs that can be interpreted by an individual NSDB. The semantics of the name/value pair is not defined by this protocol and is intended to be used by higher-level administration protocols. status: fls_status as defined by the NFSv4.1 status attribute. info: as defined in NFSv4.1 fs_locations_info attribute. Mutual Consistency across Fileset locations All of the FSLs that have the same FSN (thereby reference the same fileset) are equivalent from the point of view of client access; the different locations of a fileset represent the same data, though potentially at different points in time. Fileset locations are equivalent but not identical. Locations may either be read-only or read-write. Typically, multiple read-write locations are backed by a clustered filesystem while read-only locations are replicas created by a federation-initiated or external replication. Read-only locations may represent consistent point-in-time copies of a read-write location. The federation protocols, however, cannot prevent subsequent changes to a read- only location nor guarantee point-in-time consistency of a read-only location if the read-write location is changing. Tewari, et al. Expires April 31, 2008 [Page 8] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Regardless of the type, all locations exist at the same mount point in the namespace and, thus, one client may be referred to one location while another is directed to a different location. Since updates to each fileset location are not controlled by the federation protocol, it is the responsibility of administrators to guarantee the functional equivalence of the data. The federation protocol does not guarantee that the different locations are mutually consistent in terms of the currency of the data. It relies on the client file-access protocol (i.e., NFSv4) to contain sufficient information to help the clients determine the currency of the data at each location in order to ensure that the clients do not revert back in time when switching locations. This raises a concern for NFSv3 fileservers, which the federation protocol may support, that may lack such control. 2.3 Namespace Repository (NSDB) The NSDB service is a federation-wide service that provides interfaces to define, update, and query FSN information and FSN to FSL mapping information. An individual repository of namespace information is called an NSDB location. Each NSDB location is managed by a single administrative entity. A single admin entity can manage multiple NSDB locations. The difference between the NSDB service and an NSDB location is analogous to that between the DNS service and a particular DNS server. The term local NSDB is shorthand for an NSDB location that is known a priori to a server. It is typically located within the same federation member as the server, but this is not required. A local NSDB is not required. Each NSDB location stores the definition of the FSNs for which it is authoritative. It also stores the definitions of the FSLs associated with those FSNs. An NSDB location is authoritative for the filesets that it defines. An NSDB location can cache information from a peer NSDB location. The fileserver can always contact a local NSDB location (if it has been defined) or directly contact any NSDB location to resolve a junction. Each NSDB location supports an LDAP interface and can be accessed by an LDAP client. [Ed: the above description needs to be discussed] 2.4 Mount Points, Junctions and Referrals A mount point is a directory in a parent fileset where a target fileset may be attached. If a client traverses the path leading from the root of the Tewari, et al. Expires April 31, 2008 [Page 9] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 namespace to the mount point of a fileset it should be able to access the data in that fileset (assuming appropriate permissions). The directory where a fileset is mounted is represented by a junction in the underlying filesystem. In other words, a junction can be viewed as a reference from a directory in one fileset to the root of the target fileset. A junction can be implemented as a special marker on a directory that is interpreted by the fileserver as a mount point, or by some other mechanism in the underlying file system. What data is used by the underlying file system to represent the junction is not defined by this protocol. The essential property is that the server must be able to find, given the junction, the FSN for the target fileset. The FSN (as described earlier) contains both the NSDB location of the authoritative NSDB location and the fsn_uuid (a UUID for the fileset). When a client traversal reaches a junction, the client is referred to a list of FSLs associated with the FSN that was the target of the junction. The client can then redirect its connection to one of the FSLs. This act is called a referral. For NFSv4 clients, the FSL information is returned in the fs_locations or fs_locations_info attributes. The federation-fs interfaces do not limit where and how many times a fileset is mounted in the namespace. Filesets can be nested -- a fileset can be mounted under another fileset. 2.5 Federation Root FileServers A set of designated fileservers that render the common federation-wide namespace are called the federation root fileservers. The federation protocol does not mandate that federation root fileservers be defined. When a client mounts the root of the namespace from a root fileserver it can traverse the entire federation-wide namespace. It is not required for a client to mount from one of the root fileservers. If a client mounts from a non-root fileserver then it can traverse the part of the namespace that is visible starting from that fileserver. A client can mount multiple individual filesets from multiple non-root fileservers and chose to navigate the namespace in any manner. How the client discovers the root fileserver(s), if one is defined, is not in the scope of the federation protocol. Numerous external techniques such as DNS SRV records can be used for this. 2.6 Federation Root FileSet The root fileset is the optional, top-level fileset of the federation-wide namespace. The root of the namespace is the top level directory of this Tewari, et al. Expires April 31, 2008 [Page 10] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 fileset. The fileset can contain an arbitrary number of virtual directories. The leaf directories of the root fileset serve as the mount points for other filesets. It is desirable that the leaf directories not contain data. The root fileset is a simple combination of internal nodes and leaf nodes where each leaf node is a junction to a target fileset. The root fileset is replicated at all the root fileservers. The recommended replication protocols for root fileset replication are: an external protocol such as rsync or NDMP. 2.7 Fileservers Fileservers are NFSv4 servers that store the physical fileset data or fileservers that refer the client to other fileservers. 2.8 File-access Clients File access clients are standard off-the-shelf NAS clients that access file data using the NFSv4 protocol. 3 Interaction with NFSv4 The federation protocol is compatible with the requirements of NFSv4 referral mechanisms as defined in RFC 3530. 4 Finding the local NSDB The local NSDB may be used for finding the mapping from the servers local representation of a junction to an FSN. How the mapping is resolved is implementation-specific. The fed-fs protocol does not mandate how and if a local NSDB is defined or located. A fileserver could choose to have a special configuration setup for defining the local or default NSDB in a manner similar to a resolv.conf file for DNS. 5 Examples In this section we provide examples and discussion of the basic operations facilitated by the federated file system protocol: creating a fileset, adding a replica of a fileset, resolving a junction, and creating a junction. 5.1 Create a Fileset and its FSL(s) A fileset is the abstraction of a set of files and their containing directory tree. The fileset abstraction is the fundamental unit of data management in the federation. This abstraction is implemented Tewari, et al. Expires April 31, 2008 [Page 11] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 by an actual directory tree whose root location is specified by a fileset location (FSL). In this section, we describe the basic requirements for starting with a directory tree and creating a fileset that can be used in the federation protocols. Note that we do not assume that the process of creating a fileset requires any transformation of the files or the directory hierarchy. The only thing that is required by this process is assigning the fileset a fileset name (FSN) and expressing the location(s) of the implementation of the fileset as FSL(s). There are many possible variations to this procedure, depending on how the FSN that binds the FSL is created, and whether other replicas of the fileset exist, are known to the federation, and need to be bound to the same FSN. It is easiest to describe this in terms of how to create the initial implementation of the fileset, and then describe how to add replicas. 5.1.1 Creating a Fileset and a FSN 1. Choose the NSDB node that will keep track of the FSL(s) and related information for the fileset. 2. Request that the NSDB node register a new FSN for the fileset. The FSN may either be chosen by the NSDB node or by the server. The latter case is used if the fileset is being restored, perhaps as part of disaster recovery, and the server wishes to specify the FSN in order to permit existing junctions that reference that FSN to work again. At this point, the FSN exists, but its location is unspecified. 3. Send the FSN, the local volume path, the export path, and the export options for the local implementation of the fileset to the NSDB node. Annotations about the FSN or the location may also be sent. The NSDB node records this info and creates the initial FSL for the fileset. Tewari, et al. Expires April 31, 2008 [Page 12] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 5.1.2 Adding a Replica of a Fileset Adding a replica is straightforward: the NSDB node and the FSN are already known. The only remaining step is to add another FSL. Note that the federation interfaces do not include methods for creating or managing replicas: this is assumed to be a platform- dependent operation (at least at this time). The only interface required is the ability to register or remove the registration of replicas for a fileset. 5.2 Junction Resolution A fileset may contain references to other filesets. These references are represented by junctions. If a client requests access to a fileset object that is a junction, the server resolves the junction to discover the FSL(s) that implements the referenced fileset. There are many possible variations to this procedure, depending on how the junctions are represented and how the information necessary to perform resolution is represented by the server. In this example, we assume that the only thing directly expressed by the junction is the junction key; its mapping to FSN can be kept local to the server hosting the junction. Step 5 is the only step that interacts directly with the federation interfaces. The rest of the steps may use platform-specific interfaces. 1. The server determines that the object being accessed is a junction. 2. The server determines the junction key for the junction. 3. Using the junction key, the server does a local lookup to find the FSN of the target fileset. 4. Using the FSN, the server finds the NSDB node responsible for the target object. 5. The server contacts that NSDB node and asks for the set of FSLs that implement the target FSN. The NSDB node responds with a set of FSLs. Tewari, et al. Expires April 31, 2008 [Page 13] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 5.3 Example use case for fileset annotations The fileset annotations can be used to define relationships between filesets that can be used by an auxiliary replication protocol. Consider the scenario where a fileset is created and mounted at some point in the namespace. A snapshot of the read-write FSL of that fileset is taken periodically at different frequencies say a daily snapshot or a weekly snapshot. The different snapshots are mounted at different locations in the namespace. The daily snapshots are considered as a different fileset from the weekly ones but both are related to the source fileset. For this we can define an annotation labeling the filesets as source and replica. The replication protocol can use this information to copy data from one or more FSLs of the source fileset to all the FSLs of the replica fileset. The replica filesets are read-only while the source fileset is read-write. This follows the traditional AFS model of mounting the read-only volume at a path in the namespace different from that of the read-write volume. The federation protocol does not control or manage the relationship among filesets. It merely enables annotating the filesets with user-defined relationships. 6 Error Definitions ERR_OK Indicates the operation completed successfully. ERR_ACCESS Permission denied. The caller does not have the correct permission to perform the requested operation. Contrast this with ERR_PERM, which restricts itself to owner or privileged user permission failures. ERR_BADCHAR A UTF-8 string contains a character which is not supported in the context in which it being used. ERR_BADNAME A name string in a request consists of valid UTF-8 characters supported by the server but the name is not supported by the server as a valid name for current operation. ERR_BADTYPE An attempt was made to create an object of a type not supported by the server. Tewari, et al. Expires April 31, 2008 [Page 14] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 ERR_DENIED An attempt to lock a file is denied. Since this may be a temporary condition, the client is encouraged to retry the lock request until the lock is accepted. ERR_EXIST Object exists. The object specified already exists. ERR_INVALID Invalid argument or unsupported argument for an operation. ERR_IO I/O error. A hard error (for example, a disk error) occurred while processing the requested operation. ERR_NAMETOOLONG The filename in an operation was too long. ERR_NOENT No such object. The object being accessed does not exist. ERR_NOTDIR Not a directory. The caller specified a non- directory in a directory operation. ERR_NOTEMPTY An attempt was made to remove an object that was not empty. An FSN which has FSLs still defined for it. ERR_NOTSUPP Operation is not supported. ERR_PERM Not owner. The operation was not allowed because the caller is either not a privileged user (root) or not the owner of the target of the operation. ERR_WRONGSEC The security mechanism being used by the client for the operation does not match the server's security policy. The client should change the security mechanism being used and retry the operation. ERR_WRONGNSDB The NSDB location is not the one to be used for this operation Tewari, et al. Expires April 31, 2008 [Page 15] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 7 Functions The following operations can be performed to populate, update and query the NSDB. Each operation can be converted to the corresponding LDAP Add, Modify, Delete or Search operation using the defined schema when communicating with an NSDB location. There are three sets of interactions that are defined. For communicating with a fileserver a separate RPC message flow has to be defined. The operations can be classified based on the 3 types of interactions: i) Admin entity and an NSDB location. The admin entity initiates and controls the commands to manage fileset and namespace information. The admin entity, however, is stateless. All state is maintained at the NSDB locations or at the fileserver. We require that an NSDB location can act as an LDAP server and the protocol used for communicating between the admin entity and an NSDB would be LDAP. ii) Admin entity and fileserver. The fileserver may not be able to handle LDAP requests and function as an LDAP server. To handle that the fed-fs protocol needs to define a new RPC message exchange. iii) Fileserver and NSDB location. Assuming that the fileserver can act as an LDAP client then the protocol between the fileserver and an NSDB location would again be LDAP. 7.1 ADMIN TO NSDB OPERATIONS 7.1.1 FSN_CREATE Function fsn_create(struct FSN *fsn); Results Returns the new fsn_uuid. Description The admin creates a new FSN by requesting the NSDB to create a new FSN object. If the fsn_uuid is not provided then a new one is allocated by the admin. The requested NSDB location should check if the FSNs NSDB location matches its own value and return an error if not. This is to ensure that there are no dummy FSNs created at the wrong NSDB. Tewari, et al. Expires April 31, 2008 [Page 16] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Errors ERR_OK ERR_EXIST ERR_NOMEM ERR_INVALID ERR_PERM ERR_WRONGNSDB LDAP Request On fileset creation the admin server assigns the uuid of the FSN (this is the same as the junction key). For example, the command fsn_create will result in the admin server assigning a uuid (foo.bar-UUID) to the fileset and then generate an LDAP ADD request to the NSDB server using the example LDIF below. This will create a new FSNObject with the given FSNUUID in the LDAP database. DN: FSNUUID=foo.bar-UUID,ou=namespace,dc=company,dc=com objectClass: FSNObject FSNUUID: foo.bar-UUID NSDBNAME: server.com Annotation: fileset_type=source The definition of the FSNObject class is given in Appendix B. The FSNObject is uniquely defined by its distinguished name (DN) which is a combination of the FSNUUID and the LDAP database organizational units. 7.1.2 FSN_DELETE Function fsn_delete(struct FSN fsn); Results Description Deletes a Fileset with the given FSN. This assumes that all the FSLs related to that FSN have already been deleted. If FSLs exist then the function will return with an ERR_NOTEMPTY error. The FSN delete function only removes the fileset from the namespace and the NSDB server that maintained state for that FSN. The fileset data itself is not deleted. The junctions that have this FSN Tewari, et al. Expires April 31, 2008 [Page 17] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 as their target may continue to point to this non existent FSN. The dangling references are removed when a client tries to resolve the target of a junction, which was referring to the deleted FSN, and the NSDB returns an ERR_NOTFOUND. Errors ERR_OK ERR_NOTFOUND ERR_INVALID ERR_NOTEMPTY ERR_PERM LDAP Request In case an FSN is to be deleted, e.g., using the "fsn_delete foo.bar-uuid" command, the admin server sends an LDAP Search command to the NSDB server to first verify if the object exists by using the following filter: Query: Base: dc=company,dc=com Filter "(&(objectClass=FSNObject)(FSNUUID=foo.bar-uuid)" The above filter searches for all FSNObject entries in the NSDBs LDAP database which have the FSNUUID set to foo.bar-uuid. If the FSNObject is found the NSDB will return the corresponding FSNObject entry to the admin. An example entry returned is shown below. LDAP Response: Count: 1 (No of Entries) Entries: DN: FSNUUID=foo.bar-uuid,ou=namespace,dc=company,dc=com objectClass: FSNObject FSNUUID: foo.bar-uuid NSDBNAME: server.name The admin then sends an LDAP DELETE request to the NSDB server to remove the FSNObject from the NSDB server. An example delete request is shown below. LDAP Delete Request: DN: objectClass=FSNObject,FSNUUID=foo.bar- uuid,ou=namespace,dc=company,dc=com Tewari, et al. Expires April 31, 2008 [Page 18] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 7.1.3 FSN_MOUNT Function fsn_mount(struct FSN target_fsn, utf8string pathname, struct FSN parent_fsn); Results Description The fsn_mount operation logically mounts the target FSN at the given path (relative to the root) of the parent FSN. If no parent_fsn is provided then the ROOT filesets FSN is assumed if a root fileset is defined. The pathname could start from / to represent the path from the root of the common federation namespace. Otherwise it is considered a relative path starting from the root of the parent FSN. The fsn_mount operation is purely a namespace operation. The fsn_mount operation creates the parent and target FSN relationship that is used later by junction_create operation between the admin and the fileserver. The parent_fsns NSDB maintains the relationship information created on an fsn_mount. Errors ERR_OK ERR_NOTFOUND ERR_INVALID ERR_NAMETOOLONG ERR_PERM LDAP Request On fileset mount operation the admin will generate an LDAP ADD request to the NSDB server using the example LDIF below. This creates a new FSNJunctionObject that establishes the mount relationship between the parent and target FSNs. DN: PARENT_FSNUUID=foo-UUID, MOUNTPATH: relative/path/from/root/of/parent ,ou=namespace,dc=company,dc=com objectClass: FSNJunctionObject PARENT_FSNUUID: foo-UUID TARGET_FSNUUID: bar-UUID MOUNTPATH: relative/path/from/root/of/parent The definition of the FSNJunctionObject class is given in Appendix B. The FSNJunctionObject is uniquely defined by its distinguished name (DN) which is Tewari, et al. Expires April 31, 2008 [Page 19] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 a combination of the PARENT_FSNUUID, mountpath (path either relative or absolute) where the target FSN is attached and the LDAP database organizational units. 7.1.4 FSN_UNMOUNT Function fsn_unmount(struct FSN parent_fsn, struct FSN target_fsn, utf8string pathname); Results Description Detaches the target_fsn from the parent_fsn at the given path. The pathname can be relative to the root of the parent_fsn or from the root of the federation namespace. The parent_fsns NSDB handles the unmount and removes the relationship between the parent_fsn and the target_fsn. Errors ERR_OK ERR_NOTFOUND ERR_INVALID ERR_PERM LDAP Request In case a target_FSN is to be unmmounted, the associated FSNJunctionObject is deleted from the NSDB maintaining the parent fileset. An example delete request is shown below. LDAP Delete Request: DN: objectClass: FSNJunctionObject ,PARENT_FSNUUID=foo-UUID, MOUNTPATH=relative/path/from/parent ou=namespace,dc=company,dc=com 7.1.5 FSL_CREATE Function fsl_create(struct FSN fsn, struct FSL *fsl); Results Tewari, et al. Expires April 31, 2008 [Page 20] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Returns the fsl_uuid of the newly created FSL Description Creates a new Fileset location for the given FSN. fsl_uuid is the (optional) uuid for the FSL. Normally an FSL is identified by the host:path pair. A uuid is another optional way to identify an FSL if it is recovered at a different host:path after a backup/restore. If the FSL belongs to an FSN that has another FSN mounted under it then there would be a related junction_create operation. Errors ERR_OK ERR_EXIST ERR_NOTFOUND ERR_INVALID ERR_PERM LDAP Request The FSL create command will result in the admin server sending an LDAP ADD request to create a new FSLObject at the NSDB maintaining the given FSN. The example LDIF is shown below. The FSL object definition is provided in Appendix B. DN:FSL=host.domain.com:/path/at/host,FSNUUID=foo.bar-UUID, ou=namespace,dc=company,dc=com objectClass: FSLObject FSNUUID: foo.bar-UUID NSDBNAME: server.com FSL: host.domain:path/at/host Type: nfs4 Version: version Annotation: filesettype=Replicafilesetlocation 7.1.6 FSL_DELETE Function fsl_delete(struct FSL fsl); Results Tewari, et al. Expires April 31, 2008 [Page 21] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Description Deletes the given Fileset location. The admin requests the NSDB server storing the FSLObject to delete it from the database. This operation does not result in the fileset locations data being deleted at the fileserver. Errors ERR_OK ERR_NOTFOUND ERR_INVALID ERR_PERM 7.1.7 FSL_UPDATE Function fsl_update(struct FSN fsn, struct FSL fsl); Results Description Update the attributes of a given FSL. This command results in a change in the attributes of the FSLObject at the NSDB server maintaining this FSL. The attributes that cannot changes are the FSL uuid (if assigned) and the FSN uuid of the fileset this FSL belongs to. Errors ERR_OK ERR_NOTFOUND ERR_INVALID ERR_PERM 7.1.8 FSL_STAT Function fsl_stat(struct FSL fsl, struct FSL_ATTR *fsl_attr); Tewari, et al. Expires April 31, 2008 [Page 22] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Results Return the attributes of the FSL. Description Find all attributes of a given FSL from the FSLObject stored at the NSDB server. Errors ERR_OK ERR_NOTFOUND ERR_INVALID ERR_PERM 7.1.9 FSL_FINDBYHOST, FSN_FINDBYHOST, FSL_FINDBYHOSTPATH Function fsl_findbyhost(utf8string host_fqdn); fsn_findbyhost(utf8string host_fqdn); fsl_findbyhostpath(utf8string host_fqdn, utf8string path); Results Return the list of matching FSLs or FSNs for a given host/path at a fileserver. Description Find all FSL or FSNs for a given host or find the FSL given a host:path pair. Errors ERR_OK ERR_NOTFOUND ERR_INVALID ERR_NAMETOOLONG ERR_PERM Tewari, et al. Expires April 31, 2008 [Page 23] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 7.2 FILESERVER TO NSDB OPERATIONS 7.2.1 FSN_GET_FSL Function fsn_get_fsl(struct FSN fsn_foo, mask_t filter, int* count, struct FSL *fsl); Results List of FSLs Description Return the list of FSLs for the given FSN fsn_foo matching the filter. The fileserver will convert the list of FSLs to the NFSv4 fs_locations. The filter could be the type of protocol (v4, v3), or type of data access (ro, rw). Errors ERR_OK ERR_NOTFOUND ERR_INVALID ERR_PERM LDAP Request The scheme used by the NFSv4 fileservers could be different from the native schema used by the NSDB LDAP server. For example, the fileserver could use the rnfs schema to contact the NSDB server for fs_locations. The rnfs schema is defined in Appendix A. An example rnfs query and corresponding response from LDAP would look like this: Query: "(&(objectClass=rnfs)(cn=fsn_foo.fsn_uuid)" The above query filter requests an object of class rnfs with the canonical name being the fsn_uuid. The NSDB returns all the FSL objects that correspond to the given FSN. The response is a list of fslocations given by a comma separated list of path@server pairs. The Boolean Referral flag indicates if it was a pure referral. Tewari, et al. Expires April 31, 2008 [Page 24] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Response: DN: cn=fsn_uuid,dc=company,dc=com Referral: TRUE fslocations: /PATH1@SERVER-IP1, PATH2@SERVER-IP2 7.3 ADMIN TO FILESERVER OPERATIONS The fileserver is not required to act as an LDAP server thus the protocol for the admin to communicate with the fileserver is not required to LDAP. The fed-fs protocol defines an RPC message exchange between the admin and the fileserver. 7.3.1 FSN_CREATE_JUNCTION Arguments struct FSN_CREATE_JUNCTIONargs{ utf8string pathname; struct FSN target_fsn; }; Results Description The act of mounting a target_fsn at a path within a parent_fsn will trigger the admin to create a junction in each fileserver hosting the FSLs belonging to the parent FSN. The fileserver hosting the FSL will store the target_fsn.fsn_uuid (junction key) to identify the target FSN. How the fileserver maintains the junction key is not defined by the fed-fs protocol. Errors ERR_OK ERR_NOTFOUND ERR_INVALID ERR_PERM 7.3.2 FSN_CREATE_EXPORT Arguments struct FSN_CREATE_EXPORTargs{ utf8string pathname; Tewari, et al. Expires April 31, 2008 [Page 25] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 utf8string exportpath; }; Results Description Export the FSL at the given export path. Errors ERR_OK ERR_NOTFOUND ERR_INVALID ERR_PERM 8 Security Considerations To be added. 9 IANA Considerations This document has no actions for IANA. 10 Conclusions The federated filesystem protocol manages multiple independently administered fileservers to share namespace and referral information to enable clients to traverse seamlessly across them. 11 Glossary Administrator: user with the necessary authority to initiate administrative tasks on one or more servers. Admin entity: A server or agent that administers a collection of fileservers and persistently stores the namespace information. Client: Any client that accesses the fileserver data using a supported filesystem access protocol. Tewari, et al. Expires April 31, 2008 [Page 26] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Federation: A set of server collections and singleton servers that use a common set of interfaces and protocols in order to provide to their clients a federated namespace accessible through a filesystem access protocol. Fileserver: A server exporting a filesystem via a network filesystem access protocol. Fileset: The abstraction of a set of files and their containing directory tree. A fileset is the fundamental unit of data management in the federation. Note that all files within a fileset are descendants of one directory, and that filesets do not span filesystems. Filesystem: A self-contained unit of export for a fileserver, and the mechanism used to implement filesets. The fileset does not need to be rooted at the root of the filesystem, nor at the export point for the filesystem. A single filesystem MAY implement more than one fileset, if the client protocol and the fileserver permit this. Filesystem access protocol: A network filesystem access protocol such as NFSv2 ([RFC1094]), NFSv3 ([RFC1813]), NFSv4 ([RFC3530]), or CIFS. FSL (Fileset location): The location of the implementation of a fileset at a particular moment in time. A FSL MUST be something that can be translated into a protocol-specific description of a resource that a client can access directly, such as a fs_location (for NFSv4), or share name (for CIFS). Note that not all FSLs need to be explicitly exported as long as they are contained within an exported path on the fileserver. FSN (Fileset name): A platform-independent and globally unique name for a fileset. Two FSLs that implement replicas of the same fileset MUST have the same FSN, and if a fileset is migrated from one location to another, the FSN of that fileset MUST remain the same. Junction: A filesystem object used to link a directory name in the current fileset with an object within another fileset. The server-side "link" from a leaf node in one fileset to the root of Tewari, et al. Expires April 31, 2008 [Page 27] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 another fileset. Junction key: The key to lookup a junction within an NSDB node or a local table of information about junctions. Namespace: A filename/directory tree that a sufficiently-authorized client can observe. NSDB (Namespace Database Service): A service that maps FSNs to FSLs. The NSDB may also be used to store other information, such as annotations for these mappings and their components. NSDB Node: The name or location of a server that implements part of the NSDB service and is responsible for keeping track of the FSLs (and related info) that implement a given partition of the FSNs. Referral: A server response to a client access that directs the client to evaluate the current object as a reference to an object at a different location (specified by an FSL) in another fileset, and possibly hosted on another fileserver. The client re-attempts the access to the object at the new location. Replica: A replica is a redundant implementation of a fileset. Each replica shares the same FSN, but has a different FSL. Replicas may be used to increase availability or performance. Updates to replicas of the same fileset MUST appear to occur in the same order, and therefore each replica is self-consistent at any moment. We do not assume that updates to each replica occur simultaneously -- if a replica is offline or unreachable, the other replicas may be updated. Server Collection: A set of fileservers administered as a unit. A server collection may be administered with vendor-specific software. The namespace provided by a server collection could be part of the federated namespace. Singleton Server: A server collection containing only one server; a stand-alone fileserver. Tewari, et al. Expires April 31, 2008 [Page 28] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Tewari, et al. Expires April 31, 2008 [Page 29] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Appendix A. rnfs Schema Objects # Depends upon core.schema and cosine.schema # OID Base is 1.3.6.1.4.1.2312.4 # # Attribute types are under 1.3.6.1.4.1.2312.4.1 # Object classes are under 1.3.6.1.4.1.2312.4.2 # Syntaxes are under 1.3.6.1.4.1.2312.4.3 # Attribute Type Definitions attributetype ( 1.3.6.1.4.1.250.1.64 NAME ( 'fslocations') DESC 'NFS version 4 FS_Locations Information' EQUALITY caseIgnoreIA5Match SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 ) objectclass ( 1.3.6.1.4.1.250.1.65 NAME 'rnfs' DESC 'NFS version 4 FS_Locations Information' SUP top AUXILIARY MUST ( cn $ fslocations ) MAY ( description ) ) Tewari, et al. Expires April 31, 2008 [Page 30] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Appendix B. Namespace Schema Class Objects objectclass ( 1.3.6.1.4.1.4203.666.121.5 NAME 'FSNObject' DESC 'Representing a Fed-fs Fileset' STRUCTURAL SUP Base MUST ( FSNUUID $ NSDBNAME $ Type ) MAY ( Descr $ Annotation) ) objectclass ( 1.3.6.1.4.1.4203.666.121.7 NAME 'FSLObject' DESC 'Represents a physical instance of a fileset' STRUCTURAL SUP Base MUST ( FSL $ FSNUUID $ NSDBNAME $ FSLHost $ FSLPath $ Type $ State $ FsType ) MAY ( Version $ ExportOptions $ Descr $ Annotation) ) objectclass ( 1.3.6.1.4.1.4203.666.121.9 NAME 'FSNJunctionObject' DESC 'Represents a mount point' STRUCTURAL SUP Base MUST ( PARENT_FSNUUID $ TARGET_FSNUUID $ NSDBNAME $ MOUNTPATH $ ) MAY ( Descr $ Annotation) ) Tewari, et al. Expires April 31, 2008 [Page 31] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 12 References 12.1 Normative References [draft-fed-fs-req] Ellard, D., et al., Requirements for Federated File Systems, Internet Draft, June 2007. [RFC1094] Nowicki, B., "NFS: Network File System Protocol specification", RFC 1094, March 1989. [RFC1813] Callaghan, B., Pawlowski, B., and P. Staubach, "NFS Version 3 Protocol Specification", RFC 1813, June 1995. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2203] Eisler, M., Chiu, A., and L. Ling, "RPCSEC_GSS Protocol Specification", RFC 2203, September 1997. [RFC2743] Linn, J., "Generic Security Service Application Program Interface Version 2, Update 1", RFC 2743, January 2000. [RFC3530] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R., Beame, C., Eisler, M., and D. Noveck, "Network File System (NFS) version 4 Protocol", RFC 3530, April 2003. [RFC3552] Rescorla, E. and B. Korver, "Guidelines for Writing RFC Text on Security Considerations", BCP 72, RFC 3552, July 2003. [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally Unique IDentifier (UUID) URN Namespace", RFC 4122, July 2005. [RFC4346] Dierks, T. and E. Rescorla, "The Transport Layer Security (TLS) Protocol Version 1.1", RFC 4346, April 2006. [RFC4511] Sermersheim, J., "Lightweight Directory Access Protocol (LDAP): The Protocol", RFC 4511, June 2006. [RFC4513] Harrison, R., "Lightweight Directory Access Protocol (LDAP): Authentication Methods and Security Mechanisms", RFC 4513, June 2006. Tewari, et al. Expires April 31, 2008 [Page 32] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Author's Addresses Renu Tewari IBM Almaden 650 Harry Rd San Jose, CA 95120 US Email: tewarir@us.ibm.com Manoj Naik IBM Almaden 650 Harry Rd San Jose, CA 95120 US Email: manoj@almaden.ibm.com Daniel Ellard Network Appliance, Inc. 1601 Trapelo Rd, Suite 16 Waltham, MA 02451 US Phone: +1 781-768-5421 Email: ellard@netapp.com Craig Everhart Network Appliance, Inc. 7301 Kit Creek Rd Research Triangle Park, NC 27709 US Phone: +1 919-476-5320 Email: everhart@netapp.com Tewari, et al. Expires April 31, 2008 [Page 33] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The IETF Trust (2007). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Tewari, et al. Expires April 31, 2008 [Page 34] Internet-Draft draft-tewari-federated-fs-protocol-00 October 2007 Tewari, et al. Expires April 31, 2008 [Page 35]