Draft Specification NDMP Version 4 Protocol November 2000 Network Working Group Harald Skardal, INTERNET DRAFT Network Appliance, Category: Applications James Bunnell, Document: draft-skardal-ndmpv4-00.txt Spectra Logic, Tim Gardner, Chewcoba Systems, Clive Hendrie, Synaxia, Greg Linn, Network Appliance, Dave Manley, Network Appliance, Jim Ward, Workstation Solutions. Network Data Management Protocol Version 4 Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract The Network Data Management Protocol (NDMP) is an open protocol for enterprise wide network based data management. It owes heritage to NDMP version 2 and version 3. Expires June 2001 [Page 1] Draft Specification NDMP Version 4 Protocol November 2000 NDMP defines a TCP/IP based mechanism for controlling the transfer of data between primary and secondary storage subsystems on behalf of data archival or backup and recovery operations. NDMP provides architectural separation of the network attached Data Management Application (DMA), Data Servers and Tape Servers participating in archival or recovery operations. NDMP also provides low level control of tape devices and tape library robotics on behalf of DMAs. Multi- vendor interoperability and extensibility are key goals of NDMP. Copyright Copyright (C) The Internet Society (2000). All Rights Reserved. Table of Contents 1. Overview......................................................7 1.1. Motivation..................................................7 1.2. Scope.......................................................7 1.3. Audience....................................................7 1.4. Terminology.................................................7 2. Architecture..................................................9 2.1. Architectural Model.........................................9 2.2. NDMP Topologies.............................................9 2.2.1. Simple NDMP Configuration................................10 2.2.2. NDMP Two Drive Configuration.............................11 2.2.3. Tape Library Configuration...............................12 2.2.4. NDMP 3 Way Configuration.................................13 2.2.5. NDMP Tape to Tape Configuration..........................13 2.2.6. NDMP Data to Data Copy...................................15 2.3. Character and Role.........................................16 2.4. Protocol Interfaces........................................16 2.4.1. Messages from DMA to NDMP Server.........................16 2.4.2. Messages from NDMP Server to DMA.........................17 2.4.3. Optional Interfaces and Messages.........................17 2.5. Messaging Protocol.........................................19 2.6. Message Header.............................................20 2.7. Error Reporting............................................21 2.8. Message Numbers............................................24 2.9. Message Definitions........................................26 2.10. Message Sequencing and State Tables.......................27 2.10.1. General Rules...........................................27 2.10.2. Connection..............................................27 2.10.3. Authentication..........................................28 2.10.4. SCSI and Tape Devices...................................29 2.10.5. Data State Diagram......................................29 2.10.5.1. Example Race Condition................................33 2.10.6. Mover State Table.......................................34 3. NDMP Server Interfaces.......................................38 3.1. Connect Interface..........................................38 3.1.1. NDMP_CONNECT_OPEN........................................39 3.1.2. NDMP_CONNECT_CLIENT_AUTH.................................40 3.1.3. NDMP_CONNECT_CLOSE.......................................43 Expires June 2001 [Page 2] Draft Specification NDMP Version 4 Protocol November 2000 3.1.4. NDMP_CONNECT_SERVER_AUTH.................................44 3.2. Config Interface...........................................46 3.2.1. NDMP_CONFIG_GET_HOST_INFO................................47 3.2.2. NDMP_CONFIG_GET_SERVER_INFO..............................48 3.2.4. NDMP_CONFIG_GET_AUTH_ATTR................................51 3.2.5. NDMP_CONFIG_GET_BUTYPE_INFO..............................52 3.2.6. NDMP_CONFIG_GET_FS_INFO..................................56 3.2.7. NDMP_CONFIG_GET_TAPE_INFO................................58 3.2.8. NDMP_CONFIG_GET_SCSI_INFO................................60 3.3. SCSI Interface.............................................63 3.3.1. NDMP_SCSI_OPEN...........................................64 3.3.2. NDMP_SCSI_CLOSE..........................................66 3.3.3. NDMP_SCSI_GET_STATE......................................67 3.3.4. NDMP_SCSI_RESET_DEVICE...................................68 3.3.5. NDMP_SCSI_EXECUTE_CDB....................................69 3.4. Tape Interface.............................................72 3.4.1. Tape Model...............................................72 3.4.2. NDMP_TAPE_OPEN...........................................74 3.4.3. NDMP_TAPE_CLOSE..........................................76 3.4.4. NDMP_TAPE_GET_STATE......................................77 3.4.5. NDMP_TAPE_MTIO...........................................81 3.4.6. NDMP_TAPE_WRITE..........................................84 3.4.7. NDMP_TAPE_READ...........................................85 3.4.8. NDMP_TAPE_EXECUTE_CDB....................................89 3.5. Data Interface.............................................90 3.5.1. NDMP_DATA_GET_STATE......................................90 3.5.2. NDMP_DATA_LISTEN.........................................95 3.5.3. NDMP_DATA_CONNECT........................................97 3.5.4. NDMP_DATA_START_BACKUP...................................98 3.5.5. NDMP_DATA_START_RECOVER.................................100 3.5.6. NDMP_DATA_ABORT.........................................103 3.5.7. NDMP_DATA_STOP..........................................104 3.5.8. NDMP_DATA_GET_ENV.......................................105 3.6. Mover Interface...........................................106 3.6.1. Mover Interface Overview................................106 3.6.3. Mover Message Definitions...............................113 3.6.3.1. NDMP_MOVER_SET_RECORD_SIZE............................114 3.6.3.2. NDMP_MOVER_SET_WINDOW.................................116 3.6.3.3. NDMP_MOVER_CONNECT....................................119 3.6.3.4. NDMP_MOVER_LISTEN.....................................121 3.6.3.5. NDMP_MOVER_READ.......................................124 3.6.3.6. NDMP_MOVER_GET_STATE..................................127 3.6.3.7. NDMP_MOVER_CONTINUE...................................128 3.6.3.8. NDMP_MOVER_CLOSE......................................129 3.6.3.9. NDMP_MOVER_ABORT......................................130 3.6.3.10. NDMP_MOVER_STOP......................................131 4. DMA Interfaces..............................................132 4.1. Notify Interface..........................................132 4.1.1. NDMP_NOTIFY_DATA_HALTED.................................133 4.1.2. NDMP_NOTIFY_CONNECTION_STATUS...........................134 4.1.3. NDMP_NOTIFY_MOVER_HALTED................................136 4.1.4. NDMP_NOTIFY_MOVER_PAUSED................................137 4.1.5. NDMP_NOTIFY_DATA_READ...................................138 4.2. Log Interface.............................................139 4.2.1. NDMP_LOG_MESSAGE........................................140 Expires June 2001 [Page 3] Draft Specification NDMP Version 4 Protocol November 2000 4.2.2. NDMP_LOG_FILE...........................................143 4.3. File History Interface....................................145 4.3.1. NDMP_FH_ADD_FILE........................................146 4.3.2. NDMP_FH_ADD_DIR.........................................151 4.3.3. NDMP_FH_ADD_NODE........................................153 5. References..................................................153 6. Security....................................................154 7. Recognition of Prior Work...................................155 8. Authors and Contributors....................................155 8.1. Document Editor...........................................155 8.2. Authors' Addresses........................................155 8.3. Contributors..............................................156 Appendixes:....................................................157 Appendix A: MD5 Based Authentication:..........................157 Expires June 2001 [Page 4] Draft Specification NDMP Version 4 Protocol November 2000 Revision Log [This revision log will be removed from the final draft. For now it is used to keep track of what has been added.] Doc Version Update Date and Change Log --------------------------------------------------------------------- 4.0.0 This document is based on NDMP version 2 and 3. See www.ndmp.org for these documents. 4.0.1 00/11/05: Incorporated the following revisions of sections: 1: SCSI Interface - James Bunnell 2: Mover Interface: Greg Linn 3: Tape Interface: Tim Gardner and Jim Ward 4: Config Interface: Harald Skardal 5: Notify Interface, Dave Manley 6: File History Interface: Tim Gardner 7: General Issues: Clive Hendrie 4.0.2 11/12/00: Tim Gardner Revised entire document to conform to RFC 2119 guidelines. Fixed formatting of sections that was inconsistent with the rest of the document. Fixed grammar and spelling errors. Added missing content (primarily in the tape interface section). Modified content where inconsistent terminology was used. Removed or replaced embedded editorial notes with appropriate text. Changed message sections to start at the top of a page. 4.0.3 00/11/15: Corrected many small formatting nit's. Included Greg Linn's Status, Abstract (with one small addition), and Security section. Filled in company name and email for authors and contributors. Expires June 2001 [Page 5] Draft Specification NDMP Version 4 Protocol November 2000 List of Future Work: [This section will be removed in the final version of this draft.] The following tasks are planned in the following version of this draft: Inter-dispersed C-code sections: As has been done for the Mover Interface section, the C-code structs in other sections will be interleaved with the text explaining it's function. Data interface: The data interface has not yet been reviewed. The current text is from NDMP version 3. Workflow Document: The NDMP workflow document [7] will be reworked into describing a few common NDMP session creation scenarios, and added to this document as an appendix. NDMP Header file: The C-code in this document, together with other key NDMP data structures, will be collected into a "header file" which will become an appendix to this document. Key Concepts: We will add a section for "key concepts" which explains important NDMP architectural or functional elements including "mover window", "direct access recovery", "file history" and more. Formatting issues: This document is maintained and edited as a Microsoft Word document, and then printed to a file using "generic text printer". In this version of the draft there are still some outstanding formatting issues such as un-aligned TOC, incorrect printing of TAB characters, etc. These will be resolved in follow on versions of the draft. Introduction: Add introductory section which gives and overview of the structure and content of the main sections. Expires June 2001 [Page 6] Draft Specification NDMP Version 4 Protocol November 2000 1. Overview 1.1. Motivation The purpose of this protocol is to allow a network backup application to control the backup and retrieval of an NDMP compliant server without installing third party software on the server. The control and data transfer components of the backup/restore are separated. The separation allows complete interoperability at a network level. The file system vendors need only be concerned with maintaining compatibility with one, well defined protocol. The backup vendors can place their primary focus on the sophisticated central backup administration software. The NDMP protocol is targeted towards the process of backup and restore. There are extensive references to these tasks. The protocol is specifically intended to support tape drives. However, the protocol can be used for other applications and support other media in the future. 1.2. Scope This document is the specification for Network Data Management Protocol version 4. The primary focus of the version 4 protocol specification effort was to improve interoperability between NDMP products by clarifying and removing ambiguity from the NDMP version 3 specification. No new functionality has been introduced in version 4. 1.3. Audience This document is intended for use by software developers to implement Network Data Management Protocol. The reader is assumed to be familiar with network protocol specifications and with the general operation of backup software. The user is not expected to have knowledge of internal backup software behavior. 1.4. Terminology NDMP Network Data Management Protocol An open protocol for enterprise wide network based data management such as backup and restore. DMA The Data Management Application (DMA) that controls the NDMP server. In NDMP versions 1, 2 and 3 the term "NDMP client" was used instead of the DMA. NDMP host The host that executes the NDMP server application. Data is backed up from the NDMP host to either a local tape drive or to a backup device on a remote NDMP host. Expires June 2001 [Page 7] Draft Specification NDMP Version 4 Protocol November 2000 NDMP server The virtual state machine on the NDMP host that is controlled using the NDMP protocol. There is one of these for each connection to the NDMP host. This term is used independent of implementation. 1.5. Key Words The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. Expires June 2001 [Page 8] Draft Specification NDMP Version 4 Protocol November 2000 2. Architecture 2.1. Architectural Model The NDMP architecture is based on a client server model. NDMP compliant backup software, which is referred to as the Data Management Application or DMA, is considered to be the client. A DMA interacts with one or more NDMP servers, managing the transfer of data between server resident NDMP data and tape services. Each instantiation of a NDMP data or tape service is represented as a virtual state machine on the NDMP server. Data services provide an abstracted interface to the file system or primary storage of the NDMP server. A data service is the source of data during backup operations and the destination during recovery operations. Examples of data services are file servers and general compute platforms with direct or SAN attached storage. Tape services provide an abstracted interface to tape devices or other types of secondary storage attached to the NDMP server. A tape library may implement it's own NDMP server and associated tape service or may be connected through an external NDMP server. A tape service is the source of data during recover operations and the data destination during backup operations. The tape service also provides a mechanism for tape positioning and I/O on behalf of the DMA. Examples of tape services are individual tape drives, tape libraries, or servers with one or more writeable CD ROM drives. An NDMP session is an instantiation of a pair of NDMP services with data connections between the two services and control connections between the DMA and each service. The DMA creates and controls the NDMP session and is responsible for managing all session state required to fully or partially recover a file system including server topology, tape sets and numbering etc. There is exactly one control connection between the DMA and each NDMP server. The control connection is a bi-directional TCP/IP connection. If the DMA is distributed in such a way that two or more DMA processes need to communicate to one NDMP service, the DMA commands MUST be merged into a single control connection to the NDMP server. The NDMP protocol is a set of XDR encoded messages. These messages are used to control and monitor the state of each NDMP service and to collect detailed information about the NDMP session, and the data that is backed up. 2.2. NDMP Topologies This section describes typical NDMP topologies and configurations in terms of the relationship between Data Management Applications (DMAs) and NDMP server based data and tape services. Expires June 2001 [Page 9] Draft Specification NDMP Version 4 Protocol November 2000 2.2.1. Simple NDMP Configuration In the simplest configuration, a DMA will backup the data from the NDMP server to a locally attached tape subsystem. The NDMP control connection exists across the network boundary while the NDMP data connection between the data and tape services exists within the NDMP server implementation. +------------------------------+--------------------------------+ | DMA * NDMP Server | | * | | * | | +-------------+ * | | | NDMP Data | * | | | Management | <-----------------------+ | | | Application | * | | | +-------------+ * Network | | | * Boundary | | | * | | |******************************* | | | | NDMP Control | | | Connection | | V | | +-------------+ | | | NDMP | | | | Data & Tape | | | | Services | | | +-------------+ | | ^ NDMP | | | | Data V | | +-----------+ +-----------+ | | | File | | Tape | | | | System | | Subsystem | | | +-----------+ +-----------+ | | | +---------------------------------------------------------------+ Figure 1. Simple configuration Expires June 2001 [Page 10] Draft Specification NDMP Version 4 Protocol November 2000 2.2.2. NDMP Two Drive Configuration It is also possible to use NDMP to simultaneously back up to multiple backup devices physically attached to the NDMP server. In this configuration, there are two instances of the NDMP data and tape services on the NDMP server. The NDMP control connection exists across the network boundary while the NDMP data connections between the data and tape services exist within the NDMP server implementation. +------------------------------+--------------------------------+ | DMA * NDMP Server | | * | | * | | +-------------+ * | | | NDMP Data | * | | | Management | <-----------------------+ | | | Application | * | | | +-------------+ * Network | | | ^ * Boundary | | | | * | | |*************|***************** | | | | NDMP Control | NDMP Control | | | Connection | Connection | | v V | | +-------------+ +-------------+ | | | NDMP | | NDMP | | | | Data & Tape | | Data & Tape | | | | Services | | Services | | | +-------------+ +-------------+ | | ^ NDMP | ^ NDMP | | | | Data V | Data V | |+-----------+ +-----------+ +-----------+ +-----------+ | || File | | Tape | | File | | Tape | | || System | | Subsystem | | System | | Subsystem | | |+-----------+ +-----------+ +-----------+ +-----------+ | | | +---------------------------------------------------------------+ Figure 2. Two drive configuration Expires June 2001 [Page 11] Draft Specification NDMP Version 4 Protocol November 2000 2.2.3. Tape Library Configuration NDMP can be used to backup data to a tape library that is physically attached to the NDMP server. In this configuration, there is a separate instance of the NDMP server to control the robotics within the tape library. +------------------------------+--------------------------------+ | DMA * NDMP Server | | * | | * | | +-------------+ NDMP Control +------------+ | | | NDMP Data | Connection | NDMP Data | | | | Management | <------------------>| Service |---+ | | | Application | * | | | | | +-------------+ * +------------+ | | | ^ * Network ^ | | | | * Boundary | | | |************|****************** +---------+ | | | | NDMP Control | File | | | | | Connection | System | | | | v +---------+ | | | +------------+ | | | | NDMP Tape | | | | | Service | NDMP Data | | | | | Connection | | | +------------+ | | | | | | | | +---------------------------------+ | | | | | Tape Library | | | | | | +-----------+ +-----------+ | | | | +----->| Robotic | | Tape |<---------+ | | | | Control | | Subsystem | | | | | +-----------+ +-----------+ | | | +---------------------------------+ | | | +---------------------------------------------------------------+ Figure 3. Tape Library Configuration Expires June 2001 [Page 12] Draft Specification NDMP Version 4 Protocol November 2000 2.2.4. NDMP 3 Way Configuration It is possible to backup an NDMP server that supports NDMP but does not have a locally attached backup device by sending the data through a raw TCP/IP connection to another NDMP server. In this case the NDMP data service exists on one server and the NDMP tape service on a separate server. Both the NDMP control connections (to server 1 & server 2) and the NDMP data connection (between server 1 & server 2) exist across the network boundary. +-------------------+----------------------+--------------------+ | NDMP Server 1 * DMA * NDMP Server 2 | | * * | | * * | | NDMP Control * +--------------+ * NDMP Control | | Connection * | NDMP Data | * Connection | | +------------>| Management |---*---------+ | | | * | Application | * | | | | * +--------------+ * | | | | * * | | | | * Network Boundary * | | | | ************************ | | | | * | | | V * V | |+------------------+ NDMP Data +------------------+ | || NDMP Data | Connection | NDMP Tape | | || Service | -------------------->| Service | | |+------------------+ * +------------------+ | | ^ * | | | |Backup * | Backup | | |Data * | Data | | | * Network V | | +-----------+ * Boundary +-----------+ | | | File | * | Tape | | | | System | * | Subsystem | | | +-----------+ * +-----------+ | | * | +---------------------------------------------------------------+ Figure 4. Backing up NDMP host through the network to another NDMP host 2.2.5. NDMP Tape to Tape Configuration In addition to backup and recovery operations, NDMP supports tape to tape transfers. In this case one tape service performs a recovery operation while the other performs a backup operation allowing tape data to be copied from one NDMP server to another NDMP server. Tape to tape copy function could be used to duplicate backup tapes for offsite storage. Expires June 2001 [Page 13] Draft Specification NDMP Version 4 Protocol November 2000 +-------------------+----------------------+--------------------+ | NDMP Server 1 * DMA * NDMP Server 2 | | * * | | * * | | NDMP Control * +--------------+ * NDMP Control | | Connection * | NDMP Data | * Connection | | +------------>| Management |---*---------+ | | | * | Application | * | | | | * +--------------+ * | | | | * * | | | | * Network Boundary * | | | | ************************ | | | | * | | | V * V | |+------------------+ NDMP Data +------------------+ | || NDMP Tape | Connection | NDMP Tape | | || Service | -------------------->| Service | | |+------------------+ * +------------------+ | | ^ * | | | |Recovery * | Backup | | |Data * | Data | | | * Network V | | +-----------+ * Boundary +-----------+ | | | Tape | * | Tape | | | | Subsystem | * | Subsystem | | | +-----------+ * +-----------+ | | * | +---------------------------------------------------------------+ Figure 5 Tape to tape copy Expires June 2001 [Page 14] Draft Specification NDMP Version 4 Protocol November 2000 2.2.6. NDMP Data to Data Copy NDMP also supports data to data transfers. In this case one data service performs a backup operation while the other performs a recovery operation on the same data stream. This capability can be used to perform a logical duplication of a portion of a file system (data migration). +-------------------+----------------------+--------------------+ | NDMP Server 1 * DMA * NDMP Server 2 | | * * | | * * | | NDMP Control * +--------------+ * NDMP Control | | Connection * | NDMP Data | * Connection | | +---------*-->| Management |---*---------+ | | | * | Application | * | | | | * +--------------+ * | | | | * * | | | | * Network Boundary * | | | | ************************ | | | | * | | | V * V | |+------------------+ NDMP Data +------------------+ | || NDMP Data | Connection | NDMP Data | | || Service | -------------------->| Service | | |+------------------+ * +------------------+ | | ^ * | | | |Backup * | Recovery | | |Data * | Data | | | * Network V | | +-----------+ * Boundary +-----------+ | | | File | * | File | | | | System | * | System | | | +-----------+ * +-----------+ | | * | +---------------------------------------------------------------+ Figure 6 Data to data copy Expires June 2001 [Page 15] Draft Specification NDMP Version 4 Protocol November 2000 2.3. Character and Role An NDMP server provides two services: A data server and a tape server. During the backup, data server reads the data from disk, generates an NDMP data stream using a specified backup format, and sends the file history information, if requested, back to the DMA. For the retrieval, the data server reads the NDMP data stream and restores it back to the disk. The data server SHOULD NOT be aware of any backup device or medium issues, e.g. tape size, block size, end of medium and so on. The data server will work the same way when writing the backup data to an unlimited size backup tape or reading the backup data from that. The tape server either reads an NDMP data stream and writes it to tape or reads from tape and writes to the NDMP data stream, depending upon whether a backup or recover is taking place. The tape server SHOULD NOT be aware of the backup format, e.g. dump, tar and so on. All tape handling functions, such as split image issues MUST be dealt with by this service. 2.4. Protocol Interfaces NDMP messages are grouped together by functionality into several interfaces. An NDMP server implementation is NOT REQUIRED to implement all of the listed messages. See 2.4.3 for details of optional interfaces and requests. 2.4.1. Messages from DMA to NDMP Server The NDMP server MUST implement a consistent subset of the following interfaces: CONNECT interface This interface is used after a client first establishes a connection to an NDMP server. The CONNECT interface allows the NDMP server to authenticate the client and negotiate the version of protocol used. CONFIG interface This interface allows a DMA to discover the configuration of the NDMP server. The CONFIG interface can be used to discover NDMP server configuration and attributes. SCSI interface This interface is used to pass SCSI CDBs through to a SCSI device and retrieve the resulting SCSI status. The DMA uses the SCSI interface to control locally attached tape library robotics. Software on the DMA will construct SCSI CDBs and will interpret the returned status and data. The SCSI interface MAY also be used to exploit special features of SCSI backup devices. Expires June 2001 [Page 16] Draft Specification NDMP Version 4 Protocol November 2000 TAPE interface This interface supports both tape positioning and tape read/write operations. The DMA typically uses the TAPE interface to write tape volume header and trailer files. The DMA also uses the TAPE interface to position the tape during backups and recovers. DATA interface This interface is used to initiate backup and recover operations. The DMA provides all of the parameters that affect the backup or recover using the DATA interface. The DMA does not place any constraints on the format of the backup data other than it MUST be a stream of data that can be written to the tape device. MOVER interface This interface is used to control the reading/writing of backup data from/to a tape device. During a backup the MOVER reads data from the data connection, buffers the data into tape records, and writes the data to the tape device. During a recover the MOVER reads data from the tape device and writes the data to the data connection. The MOVER is responsible for handling tape exceptions and notifying the DMA. 2.4.2. Messages from NDMP Server to DMA The NDMP server's implementation MAY send the following messages to the DMA. All the messages that the DMA accepts are asynchronous. None of these messages will generate a reply message. NOTIFY interface The NDMP server uses these messages to notify the DMA that the NDMP server requires attention. FILE HISTORY interface These messages allow the NDMP server to make entries in the file history catalog for the current backup. File history information is used by the DMA track the files stored on each tape volume and to select and locate files for recovery. LOG interface These messages allow the NDMP server to make entries in the backup and recovery log. The operator uses the log to monitor the progress and completion status of the backup and recovery operations. The log MAY also be used to diagnose problems. 2.4.3. Optional Interfaces and Messages An NDMP Server MAY omit implementation of certain requests as described below. However, if the NDMP Server receives a request that it does not implement, it MUST generate a reply containing the NDMP_NOT_SUPPORTED_ERR error code. Expires June 2001 [Page 17] Draft Specification NDMP Version 4 Protocol November 2000 This section describes optional large scale features and lists the associated requests. Certain individual messages are also optional. Where this is the case it will be noted in the detailed description of the message in section 3. If a request is not explicitly indicated as optional in its description in section 3 and it is part of the feature set supported by the NDMP server, then the NDMP server MUST implement that request. As described in section 2.3 an NDMP server can provide two separate services: the DATA service and the TAPE service. The TAPE service provides access to two related device types (the tape drives themselves and robotic media changer devices) that MAY potentially be driven separately. The NDMP messages can be partitioned functionally into the following four subsets: Core Messages The core subset of NDMP requests applicable to all NDMP Servers includes: All CONNECT interface requests. General-purpose CONFIG interface requests including NDMP_CONFIG_GET_HOST_INFO, NDMP_CONFIG_GET_SERVER_INFO, NDMP_CONFIG_GET_CONNECTION_TYPE, and NDMP_CONFIG_GET_AUTH_ATTRIB. All NDMP Servers MUST be able to generate the initial NDMP_NOTIFY_CONNECTION_STATUS message and MAY generate NDMP_LOG_MESSAGE. Data Service messages. The Data service is responsible for the interfaces to the file system that is being backed up or recovered. The Data server feature consists of the following messages: All DATA interface requests. CONFIG interface NDMP_CONFIG_GET_BUTYPE_INFO and NDMP_CONFIG_GET_FS_INFO requests. The data server MSUT be able to generate a NDMP_NOTIFY_DATA_HALTED message. It MUST be able to generate NDMP_FH_ADD_FILE, and/or NDMP_FH_ADD_DIR/NDMP_FH_ADD_NODE messages if the data it returns in an NDMP_CONFIG_GET_BUTYPE_INFO reply indicates that it will. It MAY also generate NDMP_LOG_FILE notification messages. Tape Service messages. The tape service provides access to tape drives. It MAY also provide access to media changer devices. The Tape access feature consists of the following messages: Expires June 2001 [Page 18] Draft Specification NDMP Version 4 Protocol November 2000 All TAPE interface and MOVER interface requests. The CONFIG interface NDMP_CONFIG_GET_TAPE_INFO request. The server MUST be able to generate NDMP_NOTIFY_MOVER_PAUSED and NDMP_NOTIFY_MOVER_HALTED messages. Media changer access feature messages. The media changer access feature provides access to media changer devices. The media changer access feature consists of the following messages: All SCSI interface requests. The CONFIG interface NDMP_CONFIG_GET_SCSI_INFO request. 2.5. Messaging Protocol The NDMP protocol consists of NDMP request and reply messages sent over a TCP/IP connection. The protocol uses the RPC Record Marking (RM) Standard [4]. An NDMP message consists of a message header optionally followed by a message body. A message sequence number identifies each message. This message sequence number is sent as part of the header. Each message (message header plus optional message body) is XDR encoded and sent within a single RM record. (See [1] for details of XDR.) Implementation note: The XDR libraries available on UNIX/LINUX platforms include a set of xdrrec functions that provide RPC Record Marking and XDR translation functionality. All NDMP requests from the DMA to the NDMP Server have associated NDMP reply messages that MUST be returned by the server to indicate success or failure. NDMP requests from the NDMP Server to the DMA do not have associated replies. When a DMA sends a request to the NDMP Server it SHOULD wait to receive the reply before sending its next request. If the DMA sends multiple requests without waiting for the reply to a previous request, the NDMP Server may either queue the requests and deal with them sequentially or it may handle them asynchronously. Note that due to the use of reliable TCP connections no stoppages should be caused by reply messages getting lost in transmission. Expires June 2001 [Page 19] Draft Specification NDMP Version 4 Protocol November 2000 Since it is RECOMMENDED that the DMA wait for a reply before sending the next request, the NDMP Server MUST make every effort to reply to requests. In particular, if it receives an unrecognized message or has problems decoding a request with a valid message header it MUST send an NDMP reply message reporting the error. If the NDMP Server receives a message for which it cannot decode the message header it MUST discard the message. (This may happen if the RM record is too short to contain the full NDMP header.) If the NDMP server determines that the session is in an unrecoverable error state then it SHOULD disconnect the TCP session. This would be the case if the NDMP Server received a sequence of messages all of which were malformed. 2.6. Message Header A message header starts each message. The header is used to identify the message and defines how to de-serialize the arguments and dispatch the message. The following XDR block defines the message header: ndmp_header_message_type { NDMP_MESSAGE_REQUEST, NDMP_MESSAGE_REPLY }; struct ndmp_header { u_long sequence; u_long time_stamp; ndmp_header_message_type message_type; enum ndmp_message message_code; u_long reply_sequence; ndmp_error error_code; }; Message Header data definitions: sequence The sequence number is a connection local counter that starts at one and increases by one for every message sent. The client and the server both start with one and increase independently. time_stamp The time_stamp identifies the time, in seconds since 00:00:00 GMT, Jan 1, 1970, that the message was sent. message_type The message_type enum identifies the message as either a request or a reply message. message_code The message_code field identifies the message. Expires June 2001 [Page 20] Draft Specification NDMP Version 4 Protocol November 2000 reply_sequence The reply_sequence field MUST be 0 in a request message. In reply messages, the reply_sequence MUST be the sequence number from the request message to which the reply is associated. error_code The error_code field MUST be 0 in request messages. In reply messages, the error_code field identifies any problem that occurred receiving or decoding the message. If the error_code value is nonzero, no message body will follow the message header. The complete list of error codes is in the next section. When the NDMP Server has received and decoded a request that has a message field indicating a function that the NDMP Server supports, it MUST create a suitable message_reply. In this case, errors SHOULD be reported by setting the error field in the ndmp_header to NDMP_NO_ERR and setting the error field in the message_reply to the relevant error value. However, DMAs MUST be prepared to handle any error codes appearing in the error field in the ndmp_header. 2.7. Error Reporting When the NDMP Server receives a request from the DMA it MUST generate a reply that indicates success or failure. If the NDMP Server does not recognize or support a request it MUST generate an error reply and ignore the request. The error reply in this case SHOULD use the ndmp_header error_code field to report NDMP_NOT_SUPPORTED_ERR. (See section 2.6 for ndmp_header details.) All possible error codes are listed below. Some of these error codes cover a range of cases and it is strongly RECOMMENDED that the NDMP Server use the LOG Interface Log Message request (see 4.2.1) to provide further information. A log type of NDMP_LOG_ERROR SHOULD be used. The following error codes are defined: Expires June 2001 [Page 21] Draft Specification NDMP Version 4 Protocol November 2000 typedef int ndmp_error; const NDMP_NO_ERR = 0x00000000; const NDMP_NOT_SUPPORTED_ERR = 0x00000001; const NDMP_DEVICE_BUSY_ERR = 0x00000002; const NDMP_DEVICE_OPENED_ERR = 0x00000003; const NDMP_NOT_AUTHORIZED_ERR = 0x00000004; const NDMP_PERMISSION_ERR = 0x00000005; const NDMP_DEV_NOT_OPEN_ERR = 0x00000006; const NDMP_IO_ERR = 0x00000007; const NDMP_TIMEOUT_ERR = 0x00000008; const NDMP_ILLEGAL_ARGS_ERR = 0x00000009; const NDMP_NO_TAPE_LOADED_ERR = 0x0000000A; const NDMP_WRITE_PROTECT_ERR = 0x0000000B; const NDMP_EOF_ERR = 0x0000000C; const NDMP_EOM_ERR = 0x0000000D; const NDMP_FILE_NOT_FOUND_ERR = 0x0000000E; const NDMP_BAD_FILE_ERR = 0x0000000F; const NDMP_NO_DEVICE_ERR = 0x00000010; const NDMP_NO_BUS_ERR = 0x00000011; const NDMP_XDR_DECODE_ERR = 0x00000012; const NDMP_ILLEGAL_STATE_ERR = 0x00000013; const NDMP_UNDEFINED_ERR = 0x00000014; const NDMP_XDR_ENCODE_ERR = 0x00000015; const NDMP_NO_MEM_ERR = 0x00000016; const NDMP_CONNECT_ERR = 0x00000017; const NDMP_SEQUENCE_NUM_ERR = 0x00000018; const NDMP_READ_IN_PROGRESS_ERR = 0x00000019; The following list describes each error code. The errors that are returned in reply to specific requests are described in detail under the relevant interface descriptions in section 3. However, there may be cases where the NDMP Server uses other error replies and the DMA MUST be implemented to accept these. In particular there are a number of error codes describing unexpected conditions that can affect any request. These are marked in the following list as Generic Errors. NDMP_NO_ERR No error. NDMP_NOT_SUPPORTED_ERR Specified message not supported. Some NDMP implementations might only support a subset of the NDMP protocol. This error code is also used if the message code is unrecognized. Expires June 2001 [Page 22] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_DEVICE_BUSY_ERR Specified device is in use. This error is used in two circumstances. Firstly, it is used when an open/set target request fails because the device is in use by an agent other than this NDMP Server session. Secondly, it is returned by TAPE Interface commands if the tape is currently in use by the MOVER. (The MOVER is in active or listen state.) NDMP_DEVICE_OPENED_ERR A device is already open. NDMP connections are limited to having a single tape or SCSI device opened at a time. NDMP_NOT_AUTHORIZED_ERR NDMP connection not yet authenticated. Prior to issuing most requests, the NDMP connection MUST first be authenticated via the NDMP_CONNECT_AUTH_CLIENT message. This error is returned if a message requiring connection authentication is received when the connection has not yet been authenticated. NDMP_PERMISSION_ERR The user that was used to authenticate the connection does not have the access permissions to execute this message. NDMP_DEV_NOT_OPEN_ERR Device not open. An attempt was made to access a device that was not open. NDMP_IO_ERR Device I/O error. This general error SHOULD only be used if none of the more specific device failure error codes apply. A Log Message SHOULD be sent to describe the error in more detail. NDMP_TIMEOUT_ERR Command timeout error. NDMP_ILLEGAL_ARGS_ERR Message received containing one or more invalid arguments. It is RECOMMENDED that a Log Message be sent to describe the unacceptable arguments. NDMP_NO_TAPE_LOADED_ERR Tape device could not be opened because no tape was loaded. Alternatively the tape has been unloaded since the open command. (If the server cannot detect this specific condition an NDMP_IO_ERR SHOULD be reported.) NDMP_WRITE_PROTECT_ERR Tape device could not be opened in write mode because the tape is write protected. Alternatively the tape write protect state has changed since open or the open was a raw mode open. Expires June 2001 [Page 23] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_EOF_ERR The tape command failed because end-of-file was encountered. See Tape/Mover interface for details of usage. NDMP_EOM_ERR The tape command failed because the end of media mark was encountered. See Tape/Mover interface for details of usage. NDMP_FILE_NOT_FOUND_ERR During a recover operation, a specified file was not found. This error code is used in a Log Interface File Recovered message. NDMP_BAD_FILE_ERR Error due to invalid file descriptor. NDMP_NO_DEVICE_ERR Specified device does not exist. NDMP_NO_BUS_ERR Specified SCSI controller does not exist. NDMP_XDR_DECODE_ERR (Generic Error) Error decoding message. NDMP_ILLEGAL_STATE_ERR Message cannot be processed in the current state. NDMP_UNDEFINED_ERR (Generic Error) This error code SHOULD only be used if no other error code describes the condition. A Log Message SHOULD be sent to describe the condition in detail. NDMP_XDR_ENCODE_ERR (Generic Error) Error encoding reply message. NDMP_NO_MEM_ERR (Generic Error) Memory allocation error. It may be useful for diagnostic error avoidance purposes to send a Log Message giving more information about the operation that failed. NDMP_CONNECT_ERR Data Server - Tape Server data connection establishment failed. NDMP_SEQUENCE_NUM_ERR The request header received contains an invalid sequence number. 2.8. Message Numbers The following messages are defined: Expires June 2001 [Page 24] Draft Specification NDMP Version 4 Protocol November 2000 typedef int ndmp_message; /* Connect Interface */ const NDMP_CONNECT_OPEN = 0x900; const NDMP_CONNECT_CLIENT_AUTH = 0x901; const NDMP_CONNECT_CLOSE = 0x902; const NDMP_CONNECT_SERVER_AUTH = 0x903; /* Config Interface */ const NDMP_CONFIG_GET_HOST_INFO = 0x100; const NDMP_CONFIG_GET_CONNECTION_TYPE = 0x102; const NDMP_CONFIG_GET_AUTH_ATTR = 0x103; const NDMP_CONFIG_GET_BUTYPE_INFO = 0x104; const NDMP_CONFIG_GET_FS_INFO = 0x105; const NDMP_CONFIG_GET_TAPE_INFO = 0x106; const NDMP_CONFIG_GET_SCSI_INFO = 0x107; const NDMP_CONFIG_GET_SERVER_INFO = 0x108; /* SCSI Interface */ const NDMP_SCSI_OPEN = 0x200; const NDMP_SCSI_CLOSE = 0x201; const NDMP_SCSI_GET_STATE = 0x202; const NDMP_SCSI_SET_TARGET = 0x203; const NDMP_SCSI_RESET_DEVICE = 0x204; const NDMP_SCSI_RESET_BUS = 0x205; const NDMP_SCSI_EXECUTE_CDB = 0x206; /* Tape Interface */ const NDMP_TAPE_OPEN = 0x300; const NDMP_TAPE_CLOSE = 0x301; const NDMP_TAPE_GET_STATE = 0x302; const NDMP_TAPE_MTIO = 0x303; const NDMP_TAPE_WRITE = 0x304; const NDMP_TAPE_READ = 0x305; const NDMP_TAPE_EXECUTE_CDB = 0x307; /* Data Interface */ const NDMP_DATA_GET_STATE = 0x400; const NDMP_DATA_START_BACKUP = 0x401; const NDMP_DATA_START_RECOVER = 0x402; const NDMP_DATA_ABORT = 0x403; const NDMP_DATA_GET_ENV = 0x404; const NDMP_DATA_STOP = 0x407; const NDMP_DATA_LISTEN = 0x409; const NDMP_DATA_CONNECT = 0x40a; /* Notify Interface */ const NDMP_NOTIFY_DATA_HALTED = 0x501; const NDMP_NOTIFY_CONNECTION_STATUS = 0x502; const NDMP_NOTIFY_MOVER_HALTED = 0x503; const NDMP_NOTIFY_MOVER_PAUSED = 0x504; const NDMP_NOTIFY_DATA_READ = 0x505; Expires June 2001 [Page 25] Draft Specification NDMP Version 4 Protocol November 2000 /* Log Interface */ const NDMP_LOG_FILES = 0x602; const NDMP_LOG_MESSAGE = 0x603; /* File history interface */ const NDMP_FH_ADD_FILE = 0x703; const NDMP_FH_ADD_DIR = 0x704; const NDMP_FH_ADD_NODE = 0x705; /* Mover Interface */ const NDMP_MOVER_GET_STATE = 0xa00; const NDMP_MOVER_LISTEN = 0xa01; const NDMP_MOVER_CONTINUE = 0xa02; const NDMP_MOVER_ABORT = 0xa03; const NDMP_MOVER_STOP = 0xa04; const NDMP_MOVER_SET_WINDOW = 0xa05; const NDMP_MOVER_READ = 0xa06; const NDMP_MOVER_CLOSE = 0xa07; const NDMP_MOVER_SET_RECORD_SIZE = 0xa08; const NDMP_MOVER_CONNECT = 0xa09; /* Reserved for class extensibility */ /* from 0xff00 thru 0xffffffff */ const NDMP_RESERVED_BASE = 0x0000ff00; 2.9. Message Definitions Each message is described using a block of XDR specification in the following format: struct message_name_request { type request_argument1; ... type request_argumentN; }; struct message_name_reply { enum ndmp_error error; type reply_argument1; ... type reply_argumentN; }; Expires June 2001 [Page 26] Draft Specification NDMP Version 4 Protocol November 2000 Each XDR specification conforms to rpcgen format. No XDR specification is provided for the request message if the request message does not contain any arguments. No XDR specification is provided for the reply message if the reply message does not contain any arguments or if no reply message is defined. Not all request messages have an associated reply message. Following the XDR specification is a description of each request and reply argument. Each reply message contains an error code. If an error code is returned that is not equal to NDMP_NO_ERR, some of the reply arguments MAY be meaningless. A list of errors that MAY typically be returned in the reply is provided for each message. Note that this is not an exhaustive list. Generic errors, such as NDMP_NO_MEM_ERR, are not listed. 2.10. Message Sequencing and State Tables 2.10.1. General Rules This section describes the timing of NDMP messages. It describes when messages MUST be sent and when they MUST NOT be sent. The timing of messages is closely related to a number of state variables that describe the state of the NDMP Server. Some of these are simple booleans. (For instance "Is the client authorized?" or "Is the tape device open?"). However, the DATA and MOVER interfaces are more complex and are described below in state diagrams. The state machines are conceptually located in the NDMP Server and states changes are made in response to events in the NDMP Server. The DMA is informed of state changes by the receipt of NDMP messages. The DMA SHOULD register a state change when it receives a state change notification message from the NDMP Server or when it receives a reply from the NDMP Server accepting a previously issued request. If the DMA issues a request that would normally cause the NDMP Server to change state and this request is rejected then no state change is made. In normal conditions it is clear whether it is the responsibility of the DMA or the NDMP Server to send the next operational message. However, in error situations, abort messages may be sent by the party that is not currently in control. These situations can result in messages crossing in transmission causing race conditions. The usual result of this situation is that the request sent by the DMA is rejected by the NDMP Server with an NDMP_ILLEGAL_STATE_ERR error code. The DMA MUST be able to handle this situation in a reasonable fashion. (That is, it SHOULD continue to tidy up the session and it MUST not treat the NDMP Server as if it had caused the error.) An example race condition is shown in section 2.10.5.1. 2.10.2. Connection When an NDMP session first starts the DMA and Server MUST ensure that they can communicate successfully. Expires June 2001 [Page 27] Draft Specification NDMP Version 4 Protocol November 2000 The TCP/IP session is established by the DMA connecting to the port on which the NDMP Server is listening. Port 10000 has been reserved for NDMP. NDMP servers SHOULD typically listen on port 10000. However, to accommodate conflicts caused by another service already using port 10000, both DMAs and servers SHOULD be implemented so that they may be configured to use a different port. The first message sent on the connection MUST be an NDMP_NOTIFY_CONNECTION_STATUS message from the NDMP Server. This MAY either refuse the connection because of some local difficulty or it MAY suggest that the DMA use a particular version of NDMP. The DMA SHOULD then send an NDMP_CONNECT_OPEN request specifying an NDMP version that MUST be the same as or lower than that specified by the server in the NDMP_NOTIFY_CONNECTION_STATUS message. If the NDMP Server accepts the NDMP_CONNECT_OPEN request then the specified protocol version is used thenceforth by both the client and server for all successive messages. If the DMA wishes to use the same NDMP version as specified in the original NDMP_NOTIFY_CONNECTION_STATUS message then it MAY omit sending the NDMP_CONNECT_OPEN request. Sending any other request implicitly indicates acceptance of the NDMP version specified in the NDMP_NOTIFY_CONNECTION_STATUS message. If the client does not support the protocol version specified in the NDMP_NOTIFY_CONNECTION_STATUS message, the client SHOULD continue to send NDMP_CONNECT_OPEN requests with successively lower version numbers until the server accepts a message. Consider a server that supports versions 2 and 4 and a client that supports versions 2 and 3. The server SHOULD specify version 4 in the NDMP_NOTIFY_CONNECTION_STATUS message. Since the client does not support version 4, the client SHOULD send an NDMP_CONNECT_OPEN request containing a version of 3. Since the server does not support version 3, the server MUST reject the request by returning an NDMP_CONNECT_OPEN reply containing an NDMP_ILLEGAL_ARGS_ERROR error code. The client SHOULD then send an NDMP_CONNECT_OPEN request containing a version of 2. Since the server supports version 2, it SHOULD accept the request by returning an NDMP_CONNECT_OPEN reply containing an NDMP_NO_ERR error code. When the DMA has finished using the connection it SHOULD send an NDMP_CONNECT_CLOSE message prior to closing the TCP connection. The NDMP Server SHOULD not close the connection until requested to do so by the DMA. If forced to close the connection due to a local error or shutdown then it SHOULD first send an NDMP_NOTIFY_CONNECTION_STATUS request containing an NDMP_SHUTDOWN reason code. 2.10.3. Authentication The NDMP Server stores user data that MUST be protected from unauthorized access. The DMA MUST be authenticated by using the NDMP_CONNECT_CLIENT_AUTH request before it is allowed to use most of the NDMP requests. The following are the only requests that the DMA MAY use prior to authentication: Expires June 2001 [Page 28] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_CONNECT_OPEN, NDMP_CONNECT_CLOSE, NDMP_CONNECT_SERVER_AUTH, NDMP_CONFIG_GET_SERVER_INFO, NDMP_CONFIG_GET_AUTH_ATTR. Since the DMA is in control of establishing the TCP/IP connection and does not have any resources to protect there is less need to authenticate the server. There is an NDMP_CONNECT_SERVER_AUTH request that MAY be used. However, this is optional and the NDMP Server MAY choose not to implement it. 2.10.4. SCSI and Tape Devices The SCSI interface is used to access media changer devices. A single media changer device can be associated with the NDMP connection using the NDMP_SCSI_OPEN request. After finishing with the device the association is removed by issuing an NDMP_SCSI_CLOSE request. Other SCSI interface commands MAY only be issued when the SCSI device is open. The TAPE interface is similar, using NDMP_TAPE_OPEN and NDMP_TAPE_CLOSE to associate and disassociate the device with the NDMP connection. An NDMP Server is restricted to having a single device (SCSI or tape) associated with a connection at a time. The NDMP Server MUST return an NDMP_BUSY_ERROR upon receiving an NDMP_TAPE_OPEN or NDMP_SCSI_OPEN request if a device is already open. The TAPE interface is used to prepare the tape for use by the MOVER which writes/reads the actual backup data. In order that the two work together there are a number rules about when requests may be issued: The TAPE device MUST be open when the MOVER is activated using an NDMP_DATA_CONNECT/NDMP_MOVER_CONNECT connect request or the NDMP_MOVER_CONTINUE request. When the MOVER is in the Active state, no TAPE interface requests may be issued except NDMP_TAPE_GET_STATE. When the MOVER is in the Paused state, the TAPE interface may be freely used. This may even involve closing the current tape device and opening another device before reactivating the MOVER. 2.10.5. Data State Diagram In the following diagram the states are shown in boxes of *'s. Requests received from the DMA are shown through their message identifier (e.g. NDMP_DATA_CONNECT). The state transitions occur as the events take place at the NDMP Server. The DMA is informed of the transitions in the following ways: The transition to HALTED state is always indicated by an NDMP_NOTIFY_DATA_HALTED message. (This is true even if the transition was instigated by an NDMP_DATA_ABORT request.) Expires June 2001 [Page 29] Draft Specification NDMP Version 4 Protocol November 2000 The transition from LISTEN state to CONNECTED state is only indicated indirectly via the NDMP_MOVER_CONNECT reply on the MOVER interface. All other state transitions are related to direct requests on the DATA interface and are complete when the relevant reply message indicating success is received. Expires June 2001 [Page 30] Draft Specification NDMP Version 4 Protocol November 2000 +----------------->-----------------+ | | | V | ************* | ** Idle ** | ************* | | | | +--------------+ | | | | | NDMP_DATA_LISTEN | | | | | V | | ************ | | ** Listen ** NDMP_DATA_CONNECT | ************ | | | | | | | connected | | | | | | | +----->------+ | | | | | | NDMP_DATA_ABORT V V | OR ********************** | Internal Error ** Connected ** | OR ********************** | Connection Error | | | | | | | | | NDMP_DATA_START_BACKUP | | | OR | | | NDMP_DATA_START_RECOVER | V | | | | NDMP_DATA_ABORT | ^ | OR V | | Internal Error *************** | | OR ** Active ** | | Connection Error *************** | | | | | | | Successful Completion | +---->-----+ | OR | | | NDMP_DATA_ABORT | | | OR NDMP_DATA_STOP | | Internal Error | | | OR | | | Connection Error | | | | | | | +------<-----+ | | | | | V V V | *************** | ** Halted ** | *************** | | +------------<--------------+ Expires June 2001 [Page 31] Draft Specification NDMP Version 4 Protocol November 2000 Figure 7 - Data state diagram Idle State Idle is the start state of the state machine. - Transition to listen state upon receipt of an NDMP_DATA_LISTEN request. - Transition to Active state upon establishment of connection with the local or a remote NDMP server after receiving an NDMP_DATA_CONNECT request. Listen State The NDMP server remains in Listen state while waiting for a connection from either a local or remote NDMP server. - Transition to Connected state upon establishment of connection with an NDMP server. - Transition to Halted state upon receipt of an NDMP_DATA_ABORT message. - Transition to Halted state upon detection of a connection failure. - Transition to Halted state upon detection of an internal error. Connected State Once the data connection is established, the NDMP server is in the Connected state. - Transition to Active state upon receipt of an NDMP_DATA_START_BACKUP message. - Transition to Active state upon receipt of an NDMP_DATA_START_RECOVER message. - Transition to Halted state upon receipt of an NDMP_DATA_ABORT message. - Transition to Halted state upon detection of connection failure. - Transition to Halted state upon detection of an internal error. Active State The NDMP server remains in Active state while a backup or recover is active. - Transition to Halted state upon detection of a backup/recover error. (Note errors related to isolated files SHOULD be reported via the LOG interface and the backup/recover MUST continue.) Expires June 2001 [Page 32] Draft Specification NDMP Version 4 Protocol November 2000 - Transition to Halted state upon detection of a connection error. - Transition to Halted state upon receipt of an NDMP_DATA_ABORT message. - Transition to Halted state upon completion of backup/recover. Halted State The NDMP server enters Halted state after a backup/recover has either completed or been aborted. - Transition to Idle state upon receipt of an NDMP_DATA_STOP message. 2.10.5.1. Example Race Condition The NDMP Server is in Connected state waiting to receive an NDMP_DATA_START_BACKUP request. If at this point the connection fails (alternatively a local error or shutdown situation arises), then the NDMP Server will send an NDMP_NOTIFY_DATA_HALTED and move into Halted state. The DMA may send an NDMP_DATA_START_BACKUP request that crosses the halted notification. In this case the NDMP_DATA_START_BACKUP request will be rejected with an NDMP_ILLEGAL_STATE_ERR error code. DMA NDMP Server ----------- ----------- DMA tries to NDMP Server detects start backup. connection error on mover i/f NDMP_DATA_START_BACKUP NDMP_NOTIFY_DATA_HALTED --->-----\ /-----<--- \ / NDMP Server moves to \/ Halted state /\ / \ ----<----/ \---->--- DMA now thinks Since NDMP Server is NDMP Server is halted in halted state the Received NDMP_DATA_START_BACKUP is rejected ------------<------------ DMA MUST NDMP_DATA_START_BACKUP reply handle this error (NDMP_ILLEGAL_STATE_ERR) response. Expires June 2001 [Page 33] Draft Specification NDMP Version 4 Protocol November 2000 2.10.6. Mover State Table In the following diagram the states are shown in boxes of *'s. Requests received from the DMA are shown through their message identifier (e.g. NDMP_MOVER_LISTEN). The state transitions occur as the events take place at the NDMP Server. The DMA is informed of the transitions in the following ways: The transition to HALTED state is always indicated by an NDMP_NOTIFY_MOVER_HALTED message. (This is true even if the transition was instigated by an NDMP_MOVER_ABORT request.) The transition from LISTEN state to CONNECTED state is only indicated indirectly via the NDMP_DATA_CONNECT reply on the DATA interface. All other state transitions are related to direct requests on the MOVER interface and are complete when the relevant reply message indicating success is received. Expires June 2001 [Page 34] Draft Specification NDMP Version 4 Protocol November 2000 +--------------->----------------+ | | | V | ************* | ** Idle ** | ************* | | | | +------<------+ | | | | | NDMP_MOVER_LISTEN | | | | | V | | ************ | | ** Listen ** NDMP_MOVER_CONNECT | ************ | | | | | | | connected | | | +----->-----+ | +---------<--------+ | | | | | | | NDMP_MOVER_ABORT V V V | | OR ******************* | | Internal Error *** Active *** | | OR ******************* | | Connection Error | | | | | Connection Closed | ^ | | OR EOW OR NDMP_MOVER | | NDMP_MOVER_ABORT Seek OR CONTINUE | V OR EOM OR EOF | ^ | Internal Error | | | | OR | | | | Connection Error V | | | OR *************** | | | Media Error ** Paused ** | | | | *************** | | +---->-----+ | | | | | | | NDMP_MOVER_ABORT +->--+ | | | OR NDMP_MOVER_STOP | | Internal Error | | | OR | | | Connection Closed | | | OR | | | NDMP_MOVER_CLOSE | | | | | | | +---<----+ | | | | | V V V | *************** | ** Halted ** | *************** | | +------------<--------------+ NDMP_MOVER_CONTINUE Expires June 2001 [Page 35] Draft Specification NDMP Version 4 Protocol November 2000 Figure 8 - Mover state diagram Idle State Idle is the start state of the state machine. - Transition to Listen state upon receipt of an NDMP_MOVER_LISTEN message. - Transition to Active state upon establishment of connection with another NDMP server by an NDMP_MOVER_CONNECTED message. Listen State The NDMP server remains in Listen state while waiting for a connection from either a local or remote NDMP data server. - Transition to Active state upon establishment of connection with an NDMP server. - Transition to Halted state upon receipt of an NDMP_MOVER_ABORT message. - Transition to Halted state upon detection of an internal error. - Transition to Halted state upon detection of a connection error. Active State The NDMP server remains in Active state while the data connection is active. - Transition to Halted state upon detection of an internal error. - Transition to Halted state upon receipt of an NDMP_MOVER_ABORT message. - Transition to Halted state upon detection of a connection error. - Transition to Halted state upon detection of connection close. - Transition to Halted state upon detection of media error. - Transition to Paused state upon detection of End of Media (EOM). - Transition to Paused state upon detection of End of File (EOF). - Transition to Paused state upon reaching end of data window. Expires June 2001 [Page 36] Draft Specification NDMP Version 4 Protocol November 2000 Halted State The NDMP server enters Halted state after a backup/recover has either completed or been aborted. - Transition to Idle state upon receipt of an NDMP_MOVER_STOP message. Paused State The NDMP server remains in Paused state while waiting for a tape to be changed or a new mover window to be set. - Transition to Active state upon receipt of an NDMP_MOVER_CONTINUE message. - Transition to Halted state upon receipt of an NDMP_MOVER_ABORT message. - Transition to Halted state upon receipt of an NDMP_MOVER_CLOSE message. - Transition to Halted state upon detection of an internal error. - Transition to Halted state upon detection of connection close. Expires June 2001 [Page 37] Draft Specification NDMP Version 4 Protocol November 2000 3. NDMP Server Interfaces This section defines the protocol interfaces implemented by the NDMP server. 3.1. Connect Interface This interface is used to authenticate the client and negotiate the version of protocol that will be used. The DMA first connects to a well-known port (10,000). The NDMP server accepts the connection and sends an NDMP_NOTIFY_CONNECTION_STATUS message. The DMA then sends an NDMP_CONNECT_OPEN message. The DMA is authenticated by the NDMP server using an NDMP_CONNECT_CLIENT_AUTH message. Optionally, the DMA MAY use an NDMP_CONNECT_SERVER_AUTH message to authenticate the NDMP server as well. Expires June 2001 [Page 38] Draft Specification NDMP Version 4 Protocol November 2000 3.1.1. NDMP_CONNECT_OPEN This message is used to negotiate the protocol version to be used between the DMA and server. This message is OPTIONAL if the DMA agrees to the protocol version specified in the NDMP_NOTIFY_CONNECTION_STATUS message. If sent, it MUST be the first message type sent by the DMA. If the suggested protocol version is not supported on the NDMP server, an NDMP_ILLEGAL_ARGS_ERR MUST be returned. The DMA SHOULD continue to try this same request with a different protocol version until the negotiation succeeds or there are no more protocol versions to try. Once the protocol version has been successfully negotiated, it remains until the end of the NDMP session. It is illegal to send this message once any other type of message has been sent. Message XDR definition /* NDMP_CONNECT_OPEN */ struct ndmp_connect_open_request { u_short protocol_version; }; struct ndmp_connect_open_reply { ndmp_error error; }; Request Arguments protocol_version Protocol version suggested by the DMA. The valid protocol_version is 1, 2, 3 or 4. Reply Errors NDMP_NO_ERR Protocol version suggested by the client is supported by the server. NDMP_ILLEGAL_ARGS_ERR Protocol version suggested by the client is not supported by the server. The client SHOULD retry the request with a lower protocol version number. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. NDMP_ILLEGAL_STATE_ERR The protocol has already been negotiated. Expires June 2001 [Page 39] Draft Specification NDMP Version 4 Protocol November 2000 3.1.2. NDMP_CONNECT_CLIENT_AUTH This message is used to authenticate the DMA to the NDMP server. Only some of the request messages within the CONFIG and CON- NECT interfaces may be processed on a connection that has not yet been authenticated. A reply message containing an NDMP_NOT_AUTHORIZED_ERR error will be returned in response to any other request messages received when the connection has not yet been authenticated. NDMP servers MUST support at least one of the following authentication methods. NONE No authentication required. TEXT Connection is authenticated using an auth id and clear text password. MD5 Connection is authenticated using an MD5 algorithm. The MD5 method uses the MD5 message digest algorithm described in RFC1321 to authenticate the client and/or the server using a shared secret (password). The message used to compute the MD5 digest is a concatenation of the password, null padding, the 64 byte fixed length challenge and a repeat of the password. The length of the null padding is chosen to result in a 128 byte fixed length message. The length of the padding can be computed as 64 - 2*(length of the password). The client digest is computed using the server challenge from the NDMP_CONFIG_GET_AUTH_ATTR reply. Expires June 2001 [Page 40] Draft Specification NDMP Version 4 Protocol November 2000 Message XDR definition /* NDMP_CONNECT_CLIENT_AUTH */ enum ndmp_auth_type { NDMP_AUTH_NONE, NDMP_AUTH_TEXT, NDMP_AUTH_MD5 }; struct ndmp_auth_text { string auth_id<>; string auth_password<>; }; struct ndmp_auth_md5 { string auth_id<>; opaque auth_digest[16]; }; union ndmp_auth_data switch (enum ndmp_auth_type auth_type) { case NDMP_AUTH_NONE: void; case NDMP_AUTH_TEXT: struct ndmp_auth_text auth_text; case NDMP_AUTH_MD5: struct ndmp_auth_md5 auth_md5; }; struct ndmp_connect_client_auth_request { ndmp_auth_data auth_data; }; struct ndmp_connect_client_auth_reply { ndmp_error error; }; Request Arguments auth_data Authentication data. NDMP servers MUST support at least one of the following authentication methods: NONE No authentication required. Expires June 2001 [Page 41] Draft Specification NDMP Version 4 Protocol November 2000 TEXT Connection is authenticated using an auth id and nonencrypted password. MD5 Connection is authenticated using auth id and MD5 digest of the server challenge and the client password. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR Connection successfully authenticated. NDMP_NOT_AUTHORIZED_ERR Incorrect authentication data. NDMP_NOT_SUPPORTED_ERR The request sequence is not supported for this implementation. NDMP_ILLEGAL_ARGS_ERR Specified authentication method not supported. Expires June 2001 [Page 42] Draft Specification NDMP Version 4 Protocol November 2000 3.1.3. NDMP_CONNECT_CLOSE This message is used when the client wants to close the NDMP connection. The DMA SHOULD send this message before shutting down the TCP/IP connection. Message XDR definition /* NDMP_CONNECT_CLOSE */ /* no request arguments */ /* no reply message */ Expires June 2001 [Page 43] Draft Specification NDMP Version 4 Protocol November 2000 3.1.4. NDMP_CONNECT_SERVER_AUTH This message is used to authenticate the NDMP server to the DMA. This request message is OPTIONAL and MUST only be processed after the server has authenticated the DMA. The same client authentication methods are supported for the server authentication. Please refer to section 3.1.2 for usage of the authentication methods. If the NDMP_AUTH_MD5 authentication method is applied, the server digest is computed using the client challenge from the request. Message XDR definition /* NDMP_CONNECT_SERVER_AUTH */ union ndmp_auth_attr switch (enum ndmp_auth_type auth_type) { case NDMP_AUTH_NONE: void; case NDMP_AUTH_TEXT: void; case NDMP_AUTH_MD5: opaque challenge[64]; }; struct ndmp_connect_server_auth_request { ndmp_auth_attr client_attr; }; struct ndmp_connect_server_auth_reply { ndmp_error error; ndmp_auth_data server_result; }; Request Arguments client_attr The following attribute is defined: challenge For NDMP_AUTH_MD5 the DMA will include a per session challenge. Reply Arguments error Error code. Expires June 2001 [Page 44] Draft Specification NDMP Version 4 Protocol November 2000 server_result Authentication result. NDMP servers may return information to the DMA to authenticate the server to the client. NONE No authentication returned. TEXT Connection is authenticated using an auth id and clear text password. The auth id and password authenticate the auth id on DMA host. MD5 Connection is authenticated using user name and MD5 digest of the challenge and the user password. Reply Errors NDMP_NO_ERR Connection successfully authenticated. NDMP_NOT_SUPPORTED_ERR Request not supported. NDMP_ILLEGAL_ARGS_ERR Specified authentication method not supported. Expires June 2001 [Page 45] Draft Specification NDMP Version 4 Protocol November 2000 3.2. Config Interface This interface allows the DMA to discover the configuration of the NDMP server. Expires June 2001 [Page 46] Draft Specification NDMP Version 4 Protocol November 2000 3.2.1. NDMP_CONFIG_GET_HOST_INFO This request is used to get information about the host on which the NDMP server is running. Message XDR definition /* NDMP_CONFIG_GET_HOST_INFO */ /* no request arguments */ struct ndmp_config_get_host_info_reply { ndmp_error error; string hostname<>; string os_type<>; string os_vers<>; string hostid<>; }; Request Arguments This request does not have a message body. Reply Arguments error Error code. hostname Host name of the NDMP server os_type Name of NDMP server operating system (for example, Solaris). os_vers Version of NDMP server operating system (for example, 2.5). hostid NDMP server host identifier. Value returned by the gethostid(3) API, for example. It SHOULD be a globally unique and persistent value for the host computer such as the CPU type:serial_number, the on board MAC address, etc. This value MAY be used by the DMA for licensing purposes. Reply Errors NDMP_NO_ERR Request successfully processed. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. Expires June 2001 [Page 47] Draft Specification NDMP Version 4 Protocol November 2000 3.2.2. NDMP_CONFIG_GET_SERVER_INFO This request is used to get information about the NDMP server implementation. Message XDR definition /* NDMP_CONFIG_GET_SERVER_INFO */ /* no request arguments */ struct ndmp_config_get_server_info_reply { ndmp_error error; string vendor_name<>; string product_name<>; string revision_number<>; ndmp_auth_type auth_type<>; }; Request Arguments This request does not have a message body. Reply Arguments error Error code. vendor_name The name of the vendor that implements the NDMP server. product_name The product name of the NDMP server provided by the vendor. revision_number The revision number of the NDMP server. auth_types Connection authentication types supported by the NDMP server. Reply Errors NDMP_NO_ERR Request successfully processed. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. 3.2.3. NDMP_CONFIG_GET_CONNECTION_TYPE This request returns a list of the data connection types supported by the NDMP server. Expires June 2001 [Page 48] Draft Specification NDMP Version 4 Protocol November 2000 Message XDR definition /* NDMP_CONFIG_GET_CONNECTION_TYPE */ /* no request arguments */ enum ndmp_addr_type { NDMP_ADDR_LOCAL, NDMP_ADDR_TCP, NDMP_ADDR_IPC }; struct ndmp_config_get_connection_type_reply { ndmp_error error; ndmp_addr_type addr_types<>; }; Request Arguments This request does not have a message body. Reply Arguments error Error code. addr_types Array of supported connection types. NDMP_ADDR_LOCAL One tape server should listen for a connection from the data server that is collocated with the tape. This means that the data server and the tape server are controlled via the same DMA connection. The communication mechanism is implementation dependent. NDMP_ADDR_TCP One NDMP server should listen for a connection from a remote NDMP server using a TCP/IP port. NDMP_ADDR_IPC Two NDMP servers are on the same host, but controlled by separate DMA connections. * Notice that the value NDMP_ADDR_FC has been removed. Clients MUST still tolerate the value, but treat it as RESERVED. Reply Errors NDMP_NO_ERR Returned the supported connection type successfully. Expires June 2001 [Page 49] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. Expires June 2001 [Page 50] Draft Specification NDMP Version 4 Protocol November 2000 3.2.4. NDMP_CONFIG_GET_AUTH_ATTR This message is used to query the attributes of the supported authentication methods. If the connection will be authenticated using the MD5 method, the client MUST use this message to get the server challenge before sending the NDMP_CONNECT_CLIENT_AUTH message. Message XDR definition /* NDMP_CONFIG_GET_AUTH_ATTR */ struct ndmp_config_get_auth_attr_request { ndmp_auth_type auth_type; }; struct ndmp_config_get_auth_attr_reply { ndmp_error error; ndmp_auth_attr server_attr; }; Request Arguments auth_type The specific authentication method to be used to authenticate the DMA to the NDMP server. Reply Arguments error Error code. server_attr Returned attributes required for a specific authentication scheme. The following attribute is defined: challenge For NDMP_AUTH_MD5, the NDMP server will return a per session challenge. See appendix A for an outline of the client authentication process. Reply Errors NDMP_NO_ERR Returned the specific authentication type attributes successfully. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. NDMP_ILLEGAL_ARGS_ERR Specified authentication method not supported. Expires June 2001 [Page 51] Draft Specification NDMP Version 4 Protocol November 2000 3.2.5. NDMP_CONFIG_GET_BUTYPE_INFO This message is used to query the backup types supported by the NDMP server and the capability of each supported backup type. Message XDR definition /* NDMP_CONFIG_GET_BUTYPE_INFO */ /* No request arguments */ /* backup type attributes */ const NDMP_BUTYPE_BACKUP_FILELIST = 0x0002; const NDMP_BUTYPE_RECOVER_FILELIST = 0x0004; const NDMP_BUTYPE_BACKUP_DIRECT = 0x0008; const NDMP_BUTYPE_RECOVER_DIRECT = 0x0010; const NDMP_BUTYPE_BACKUP_INCREMENTAL = 0x0020; const NDMP_BUTYPE_RECOVER_INCREMENTAL = 0x0040; const NDMP_BUTYPE_BACKUP_UTF8 = 0x0080; const NDMP_BUTYPE_RECOVER_UTF8 = 0x0100; const NDMP_BUTYPE_BACKUP_FH_FILE = 0x0200; const NDMP_BUTYPE_BACKUP_FH_DIR = 0x0400; struct ndmp_butype_info { string butype_name<>; ndmp_pval default_env<>; u_long attrs; }; struct ndmp_config_get_butype_attr_reply { ndmp_error error; ndmp_butype_info butype_info<>; }; Request Arguments This request does not have a message body. Reply Arguments error Error code. butype_info Information about the backup types supported by the NDMP server. Backup types are NDMP server implementation dependent. The following information is provided: butype_name Name of the backup application (such as dump, tar, cpio). Expires June 2001 [Page 52] Draft Specification NDMP Version 4 Protocol November 2000 default_env The default value of the environment variables specific to the backup type. The following are examples of the environment variables that can be defined by the NDMP server and the corresponding default values. Variable Meaning Value Default Value --------------------------------------------------------------- DIRECT Utilize direct access y/n n retrieval. PREFIX Prefix path for path name (No default value) the request. TYPE Backup type. dump, tar, (No default value) cpio,... USER User id to run user name (No default value) Backup. HIST Specifies type of file f/d/n n history to generate. f = file format history d = node/dir format history n = no history The following are examples of the environment variables that can be defined by dump type and the corresponding default values. Expires June 2001 [Page 53] Draft Specification NDMP Version 4 Protocol November 2000 Variable Meaning Value The Default Value ----------------------------------------------------------------- FILESYSTEM device or file file system or (No default system name to device name, value) be backed up e.g. /dev/rsd0a LEVEL dump level 0 - 9 0 EXTRACT "y" specifies y/n y the -x option for the extraction, otherwise the -r option is used for the extraction. UPDATE update the y/n y dumpdates file The following are examples of environment variables that can be defined by tar type and the corresponding default values. Variable Meaning Value The Default Value ----------------------------------------------------------------- FILES list of files e.g. ./* ./*.c (No default to be backed ./*.h value) up The following are examples of environment variables that can be defined by cpio type and the corresponding default values. Variable Meaning Value The Default Value ----------------------------------------------------------------- CMD command to e.g. find . (No default generate the -name -print value) file list for cpio. attrs Backup attributes bit mask. The following attribute bits are defined: NDMP_BUTYPE_BACKUP_FILELIST The backup type supports archiving of selective files as specified by a file list. Expires June 2001 [Page 54] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_BUTYPE_RECOVER_FILELIST The backup type supports restoration of individual files. NDMP_BUTYPE_BACKUP_DIRECT The backup type generates valid fh_info data usable for direct access restore. NDMP_BUTYPE_RECOVER_DIRECT The backup type supports direct access restore (positioning to an offset within a backup image and restoring the specified file). NDMP_BUTYPE_BACKUP_INCREMENTAL The backup type supports incremental backup. NDMP_BUTYPE_RECOVER_INCREMENTAL The backup type supports incremental-only restoration (a full restore MUST be done before an incremental restore). NDMP_BUTYPE_BACKUP_UTF8 The backup type supports UTF8 format in the file history. NDMP_BUTYPE_RECOVER_UTF8 The backup type supports UTF8 format in the restored file list. NDMP_BUTYPE_BACKUP_FH_FILE The backup type supports the generation of file format file history. NDMP_BUTYPE_BACKUP_FH_DIR The backup type supports the generation of node/dir format file history. Reply Errors NDMP_NO_ERR The backup type information successfully returned. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. Expires June 2001 [Page 55] Draft Specification NDMP Version 4 Protocol November 2000 3.2.6. NDMP_CONFIG_GET_FS_INFO This message is used to query information about the file systems on the NDMP server host. Message XDR definition /* NDMP_CONFIG_GET_FS_INFO */ /* No request arguments */ /* unsupported bits */ const NDMP_FS_INFO_TOTAL_SIZE_UNS = 0x00000001; const NDMP_FS_INFO_USED_SIZE_UNS = 0x00000002; const NDMP_FS_INFO_AVAIL_SIZE_UNS = 0x00000004; const NDMP_FS_INFO_TOTAL_INODES_UNS = 0x00000008; const NDMP_FS_INFO_USED_INODES_UNS = 0x00000010; struct ndmp_fs_info { u_long unsupported; string fs_type<>; string fs_logical_device<>; string fs_physical_device<>; ndmp_u_quad total_size; ndmp_u_quad used_size; ndmp_u_quad avail_size; ndmp_u_quad total_inodes; ndmp_u_quad used_inodes; ndmp_pval fs_env<>; string fs_status<>; }; struct ndmp_config_get_fs_info_reply { ndmp_error error; ndmp_fs_info fs_info<>; }; Request Arguments This request does not have a message body. Reply Arguments error Error code. fs_info Information about the file system. The following attributes are defined: unsupported The unsupported bitmask is used to identify unsupported arguments in the message. Expires June 2001 [Page 56] Draft Specification NDMP Version 4 Protocol November 2000 fs_type The type of the file system. fs_logical_device The mount point or share name of the file system. fs_physical_device The physical device name of the file system, e.g. /dev/rsd0c. total_size The total size of the file system in bytes. NDMP_FS_INFO_TOTAL_SIZE_INVALID is set if this argument is not supported. used_size The used size of the file system in bytes. NDMP_FS_INFO_USED_SIZE_INVALID is set if this argument is not supported. avail_size The available size of the file system in bytes. NDMP_FS_INFO_AVAIL_SIZE_INVALID is set if this argument is not supported. total_inodes The total number of inodes within the file system. NDMP_FS_INFO_TOTAL_INODES_INVALID is set if this argument is not supported. used_inodes The number of the inodes being used within the file system. NDMP_FS_INFO_USED_INODES_INVALID is set if this argument is not supported. fs_env The environment variables defined for the file system. fs_status The current status of the file system. Reply Errors NDMP_NO_ERR File system information successfully returned. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. Expires June 2001 [Page 57] Draft Specification NDMP Version 4 Protocol November 2000 3.2.7. NDMP_CONFIG_GET_TAPE_INFO This message is used to query information about the tape devices connected to the NDMP server host. Message XDR definition /* NDMP_CONFIG_GET_TAPE_INFO */ /* tape attributes */ const NDMP_TAPE_ATTR_REWIND = 0x00000001; const NDMP_TAPE_ATTR_UNLOAD = 0x00000002; const NDMP_TAPE_ATTR_RAW = 0x00000004; /* No request arguments */ struct ndmp_device_capability { string device<>; u_long attr; ndmp_pval capability<>; }; struct ndmp_device_info { string model<>; ndmp_device_capability caplist<>; }; struct ndmp_config_get_tape_info_reply { ndmp_error error; ndmp_device_info tape_info<>; }; Request Arguments This request does not have a message body. Reply Arguments error Error code. tape_info Information about the tape device. The following attributes are defined: model The tape device model name. For example: EXB-8500. Expires June 2001 [Page 58] Draft Specification NDMP Version 4 Protocol November 2000 caplist The tape device capability list. One physical tape device can have more than one device name, each with different capabilities. Each entry in the list contains the following: device The device name of the tape device. For example: /dev/rmt/0mn. attr The bit mask of tape attributes. NDMP_TAPE_ATTR_REWIND The tape will be rewound when the device is closed. NDMP_TAPE_ATTR_UNLOAD The tape will be unloaded when the device is closed. NDMP_TAPE_ATTR_RAW The device supports raw mode open. capability The capability environment variables for the tape drive device. The following are examples of the environment variables that can be defined for a tape device. Variable Name Meaning Value The Default Value ---------------------------------------------------------------- COMPRESSION Compression ratio integer 1 (no compression) Reply Errors NDMP_NO_ERR Tape information successfully returned. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. Expires June 2001 [Page 59] Draft Specification NDMP Version 4 Protocol November 2000 3.2.8. NDMP_CONFIG_GET_SCSI_INFO This message is used to query information about the SCSI media changer devices connected to the NDMP server host. The RECOMMENDED naming convention for these devices which will allow visual identification of the location in the scsi system of the device is such: LibraryX_Y_Z for media changer devices TapeX_Y_X for tape devices where X=the ASCII representation of the controller value with leading zeros eliminated, Y=the ASCII representation of the SCSI id value with leading zeros eliminated, Z=the ASCII representation of the LUN value with leading zeros eliminated. Expires June 2001 [Page 60] Draft Specification NDMP Version 4 Protocol November 2000 Message XDR definition /* NDMP_CONFIG_GET_SCSI_INFO */ /* No request arguments */ struct ndmp_pval { string ; string ; }; struct ndmp_device_capability { string device <>; u_long attr; ndmp_pval capability <>; }; struct ndmp_device_info { string model <>; ndmp_device_capability caplist <>; }; struct ndmp_config_get_scsi_info_reply { ndmp_error error; ndmp_device_info scsi_info <>; }; Request Arguments This request does not have a message body. Reply Arguments error Error code. Given only if operation fails. scsi_info Information about the SCSI media changer device. The following attributes are defined: model The model name of the SCSI media changer device. For example: Spectra 10000F. caplist The capability list for the SCSI jukebox device. Each entry in the list contains the following: Expires June 2001 [Page 61] Draft Specification NDMP Version 4 Protocol November 2000 device The device name of the SCSI media changer device. For example: /dev/scsi0. NOTE: This name is the one required for use with the SCSI_OPEN command. attr The bit mask of SCSI attributes. SHARED_ROBOT 0x00000001 Indicates additional time required for movement commands. capability The capability environment variables for the SCSI media changer device. NOTE: Capabilities strings are optional and MAY include: SELECT_TIMEOUT Time in milliseconds. Reply Errors NDMP_NO_ERR SCSI information successfully returned. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 62] Draft Specification NDMP Version 4 Protocol November 2000 3.3. SCSI Interface The SCSI interface provides low-level control of SCSI devices. This interface is primarily intended to provide control of media changer devices, and to a lesser extent, tape devices. However, this interface is not limited to media changer or tape drive devices. The types of SCSI devices allowed to be controlled via this interface are implementation dependent. Implementation Guideline: an NDMP server implementer SHOULD carefully consider the security implications of providing access via this interface to SCSI other device types such as random access devices. Low-level SCSI control of tape devices is also provided by the tape interface via the NDMP_TAPE_EXECUTE_CDB request. However, implementation of this request is OPTIONAL as it is impractical to implement on some NDMP server platforms. The SCSI interface provides DMAs an alternative interface for accessing tape devices on NDMP server implementations that do not support NDMP_TAPE_EXECUTE_CDB. Expires June 2001 [Page 63] Draft Specification NDMP Version 4 Protocol November 2000 3.3.1. NDMP_SCSI_OPEN Opens the specified SCSI device. This operation is REQUIRED before any other SCSI requests may be executed. Although any SCSI based device can be opened with this command, a matching device name string MUST be supplied. The NDMP server software SHALL NOT do any I/O to the requested device during the open sequence. It is the responsibility of the NDMP server to offer best effort exclusive access to the device. Message XDR definition /* NDMP_SCSI_OPEN */ struct ndmp_scsi_open_request { string device<>; }; struct ndmp_scsi_open_reply { ndmp_error error; }; Request Arguments device Name of SCSI interface device to open. This argument MUST reference and open access to a single device. Ideally, the name string provided here SHOULD match a device in either the NDMP_CONFIG_GET_SCSI_INFO or NDMP_CONFIG_GET_TAPE_INFO device list, provided they are SCSI devices in nature. NOTE: This does not prohibit the opening of a device other than in the above list, provided the string can be interpreted by the server and access granted. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR SCSI interface device successfully opened. NDMP_DEVICE_OPENED_ERR The connection already has a tape device or SCSI device open. Expires June 2001 [Page 64] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_DEVICE_BUSY_ERR Another NDMP connection currently has the specified device open. NDMP_NOT_AUTHORIZED_ERR Connection not authorized or device is not a tape or media changer. NDMP_NO_DEVICE_ERR Invalid device specified. Expires June 2001 [Page 65] Draft Specification NDMP Version 4 Protocol November 2000 3.3.2. NDMP_SCSI_CLOSE This request closes the currently open SCSI device. No further requests SHALL be made until another open request is successfully executed. Message XDR definition /* NDMP_SCSI_CLOSE */ /* no request arguments */ struct ndmp_scsi_close_reply { ndmp_error error; }; Request Arguments This request does not have a message body. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR Device successfully closed. NDMP_DEV_NOT_OPEN_ERR No device currently open by the connection. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 66] Draft Specification NDMP Version 4 Protocol November 2000 3.3.3. NDMP_SCSI_GET_STATE This request returns the current state of the SCSI interface. The target information provides information about which SCSI device is controlled by this interface. Implementation of this request is OPTIONAL. Message XDR definition /* NDMP_SCSI_GET_STATE */ /* no request arguments */ struct ndmp_scsi_get_state_reply { ndmp_error error; short target_controller; short target_id; short target_lun; }; Request Arguments This request does not have a message body. Reply Arguments error Error code. target_controller Identifier of the SCSI controller to which the currently targeted SCSI device is attached. target_id SCSI target identifier. Specifies the SCSI bus address of the targeted device. target_lun Logic unit number of the targeted device. Reply Errors NDMP_NO_ERR Target device information successfully returned. NDMP_DEV_NOT_OPEN_ERR No SCSI device currently open by the connection. NDMP_NOT_SUPPORTED_ERR The request is not supported by the implementation. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 67] Draft Specification NDMP Version 4 Protocol November 2000 3.3.4. NDMP_SCSI_RESET_DEVICE This request sends a SCSI device reset message to the currently opened SCSI device. This is an OPTIONAL request. If not implemented, the server MUST return an NDMP_NOT_SUPPORTED_ERR error code. Implementation of this request MUST guarantee that only the specific device is reset, without affecting any other devices on the SCSI bus. Message XDR definition /* NDMP_SCSI_RESET_DEVICE */ /* no request arguments */ struct ndmp_scsi_reset_device_reply { ndmp_error error; }; Request Arguments This request does not have a message body. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR SCSI device successfully reset. NDMP_DEV_NOT_OPEN_ERR No SCSI device currently open by the connection. NDMP_NOT_SUPPORTED_ERR The request is not supported by the implementation. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 68] Draft Specification NDMP Version 4 Protocol November 2000 3.3.5. NDMP_SCSI_EXECUTE_CDB This request sends a SCSI Control Data Block to a SCSI device. If a check condition is generated, then the extended sense data is also retrieved. Data can be transferred to or from the SCSI device as part of the command. The server SHALL NOT modify the CDB. It is the responsibility of the DMA to construct a valid CDB for the target device. The server selects the SCSI target, specified in the NDMP_SCSI_OPEN command. The CDB is sent to the SCSI device in command mode. If DATA_OUT is set in the flags, then the dataout is sent to the SCSI device in data mode. If timeout is zero, the server MUST wait indefinitely for the command to complete. If timeout is not zero, the server MUST wait for the command to complete and return an NDMP_TIMEOUT_ERR if the command does not complete within timeout miliseconds. If the target reselects and the status is CHECK CONDITION, then the server MUST execute a REQUEST SENSE cdb. If the DATA_IN flag is set, the server reads data from the target in data mode. The SCSI status, the DATA_IN, and the extended sense data MUST be returned. There are no limitations to commands given to a SCSI device in this interface. The OPEN command SHOULD limit the type of device that can be targeted by this command. Expires June 2001 [Page 69] Draft Specification NDMP Version 4 Protocol November 2000 Message XDR definition /* NDMP_SCSI_EXECUTE_CDB */ const NDMP_SCSI_DATA_IN = 0x00000001; const NDMP_SCSI_DATA_OUT = 0x00000002; struct ndmp_execute_cdb_request { u_long flags; u_long timeout; u_long datain_len; opaque cdb<>; opaque dataout<>; }; struct ndmp_execute_cdb_reply { ndmp_error error; u_char status; u_long dataout_len; opaque datain<>; opaque ext_sense<>; }; Request Arguments flags Specifies the data transfer (if any) direction. DATA_IN and DATA_OUT reference the server. DATA_IN refers to data from the SCSI device into the server. DATA_OUT refers to data from the server to the SCSI device. timeout Number of milliseconds to wait for command completion to occur. If timeout is zero, then the server MUST wait indefinitely for the command to complete. This timeout is command and implementation dependent. It is the responsibility of the DMA to set the timeout appropriately. datain_len If the data transfer direction is DATA_IN, the expected number of data bytes to read. If the return contains more than this value, the data is truncated to the requested length. In such a case, the NDMP_NO_ERR return MUST be returned. cdb The SCSI command data block. dataout If the data transfer direction is DATA_OUT, the data to be transferred to the SCSI device. Expires June 2001 [Page 70] Draft Specification NDMP Version 4 Protocol November 2000 Reply Arguments error Error code. status SCSI status byte. dataout If the data transfer direction is DATA_OUT, the number of data bytes transferred to the device. datain If the data transfer direction is DATA_IN, the data transferred from the SCSI device. This number MUST not exceed the datain_len value. ext_sense Extended SCSI sense data. Reply Errors NDMP_NO_ERR Message successfully processed. Does not necessarily indicate that the SCSI command was successfully executed. The returned SCSI status byte MUST be used to determine if the command was successful. NDMP_DEV_NOT_OPEN_ERR No SCSI device currently open by the connection. NDMP_IO_ERR This message is restricted to errors encountered with a local driver in processing the request. NDMP_ILLEGAL_ARGS Invalid argument in request message. NDMP_TIMEOUT_ERR The SCSI command timed out. The SCSI device is in an unknown state as the result of a timeout error NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 71] Draft Specification NDMP Version 4 Protocol November 2000 3.4. Tape Interface The TAPE interface provides complete and exclusive control of a tape drive. If the tape drive is a SCSI tape drive, then the TAPE interface also provides low-level CDB access to the tape drive. 3.4.1. Tape Model Tape device names Tape device names are server implementation dependent and not a topic of this specification. At the implementer's option, an implementation MAY use more than one device name to identify the same physical tape device. Also, the implementer MAY choose to imbue behavioral semantics onto the tape device based upon the device name. (Such are also outside the purview of this specification, but may include rewind or unload on close, density, fixed block size, compression, or the like.) Exclusive device access All devices accessed by implementations of this protocol are subject to the following restrictions: 1. An NDMP connection may have open at most one device at any time. 2. An NDMP server SHALL, to the extent possible, exclude other entities from accessing any device that is open by an DMA. These requirements pertain to both SCSI and TAPE interfaces. Implicit file mark generation The TAPE interface MUST automatically write a file mark to the tape under the following circumstances: 1. The tape drive is opened in either NDMP_TAPE_WRITE_MODE or NDMP_TAPE_RAW_MODE; and, 2. Data have been written to the tape using either NDMP_TAPE_WRITE or a mover (*1) that are not followed by a file mark (*2); and, 3. The operation being performed is one of NDMP_TAPE_CLOSE or any NDMP_TAPE_MTIO except NDMP_MTIO_EOF. If any one or more of these conditions are not satisfied, no file mark is generated. (*1) Automatic file mark creation does not occur if all write operations were performed via NDMP_TAPE_EXECUTE_CDB. (*2) File marks generated by NDMP_TAPE_EXECUTE_CDB aren't recognized by this rule. Expires June 2001 [Page 72] Draft Specification NDMP Version 4 Protocol November 2000 NDMP connection closure In the event the NDMP connection between the client and server is closed while a tape device is open, the server performs an implicit NDMP_TAPE_CLOSE. Using NDMP_TAPE_EXECUTE_CDB with other TAPE operations DMAs are permitted to use NDMP_TAPE_EXECUTE_CDB to control tape drive state or tape position. Any such use of NDMP_TAPE_EXECUTE_CDB may, at the implementer's option, cause NDMP_TAPE_GET_STATE to return undefined data with respect to absolute tape position. Absent intervening NDMP_TAPE_EXECUTE_CDB requests, NDMP_TAPE_GET_STATE requests that are intermingled with tape operations performed by the MOVER or by the client via the TAPE interface MUST return valid data relative to one another. Using NDMP_TAPE_EXECUTE_CDB and NDMP_TAPE_MTIO operations in the same session MAY result in undefined behavior. NDMP_TAPE_READ and NDMP_TAPE_WRITE ops for tape drives in fixed block size mode The number of tape blocks affected by a NDMP_TAPE_READ or NDMP_TAPE_WRITE operation performed on a tape drive in fixed block size mode is a whole integer number computed as follows: = int ( ( + - 1) / ) CDBs and data expressed in TAPE_EXECUTE_CDB requests NDMP servers MUST NOT modify any CDB contents or related data communicated from or to the client in TAPE_EXECUTE_CDB requests and replies. NDMP server SHOULD NOT interpret any CDB contents or related data communicated from or to the client in TAPE_EXECUTE_CDB requests and replies. Expires June 2001 [Page 73] Draft Specification NDMP Version 4 Protocol November 2000 3.4.2. NDMP_TAPE_OPEN This request opens the tape device in the specified mode. This operation is required before any other tape requests can be executed. If the drive does not have a tape loaded, an error MUST be returned unless the mode is NDMP_TAPE_RAW_MODE. If the loaded media is write protected, then the device can be opened only in NDMP_TAPE_READ_MODE or NDMP_TAPE_RAW_MODE mode. Message XDR definition /* NDMP_TAPE_OPEN */ enum ndmp_tape_open_mode { NDMP_TAPE_READ_MODE, NDMP_TAPE_RDWR_MODE, NDMP_TAPE_RAW_MODE }; Request Arguments mode One of the following modes MUST be specified when opening the tape device: NDMP_TAPE_READ_MODE Open the tape device for read only. NDMP_TAPE_RDWR_MODE Open the tape device for read/write. NDMP_TAPE_RAW_MODE Open the tape device for read/write. Permit the open to succeed if no tape is present. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR Tape device successfully opened. NDMP_DEVICE_OPENED_ERR The NDMP server already has a SCSI device or tape device open. NDMP_NO_DEVICE_ERR The specified device does not exist. Expires June 2001 [Page 74] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_DEVICE_BUSY_ERR The device is already open by another NDMP server or system process. NDMP_IO_ERR Device I/O error. MAY also be returned if no drive is present. NDMP_WRITE_PROTECT_ERR Device cannot be opened in write mode because the tape is write protected. NDMP_NO_TAPE_LOADED_ERR No tape loaded in the tape device. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. NDMP_PERMISSION_ERR The user who authenticated the connection does not have permission to open the tape device. NDMP_NO_TAPE_LOADED_ERR There is no tape loaded and ready for operation in the tape drive. This error MUST only be reported if the mode is NDMP_TAPE_READ_MODE or NDMP_TAPE_WRITE_MODE. Expires June 2001 [Page 75] Draft Specification NDMP Version 4 Protocol November 2000 3.4.3. NDMP_TAPE_CLOSE This request closes the tape drive. For this request to succeed, any MOVER using this tape device MUST be not either in an active or a listen state. If the tape device is opened in NDMP_TAPE_READ_MODE, the CLOSE operation MUST not change the tape position. If the tape device is opened in NDMP_TAPE_WRITE_MODE or NDMP_TAPE_RAW_MODE, file mark generation occurs as specified earlier. Message XDR definition /* NDMP_TAPE_CLOSE */ /* no request arguments */ struct ndmp_tape_close_reply { ndmp_error error; }; Request Arguments This message does not have a message body. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR Tape device successfully closed. NDMP_DEV_NOT_OPEN_ERR No tape device currently open by the connection. NDMP_IO_ERR Device I/O error. NDMP_DEVICE_BUSY_ERR A MOVER associated with this tape drive is either in an active or listen state. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 76] Draft Specification NDMP Version 4 Protocol November 2000 3.4.4. NDMP_TAPE_GET_STATE This request returns the state of the tape drive interface. If any TAPE interface operations are supported by a server, support of this message is required. Message XDR definition /* NDMP_TAPE_GET_STATE */ /* no request arguments */ /* generic unsigned 64-bit value */ struct ndmp_u_quad { u_long high; u_long low; }; /* flags */ const NDMP_TAPE_STATE_NOREWIND = 0x0008; /* non-rewind device */ const NDMP_TAPE_STATE_WR_PROT = 0x0010; /* write-protected */ const NDMP_TAPE_STATE_ERROR = 0x0020; /* media error */ const NDMP_TAPE_STATE_UNLOAD = 0x0040; /* tape unloaded upon close */ /* unsupported bits */ const NDMP_TAPE_STATE_FILE_NUM_UNS = 0x00000001; const NDMP_TAPE_STATE_SOFT_ERRORS_UNS = 0x00000002; const NDMP_TAPE_STATE_BLOCK_SIZE_UNS = 0x00000004; const NDMP_TAPE_STATE_BLOCKNO_UNS = 0x00000008; const NDMP_TAPE_STATE_TOTAL_SPACE_UNS = 0x00000010; const NDMP_TAPE_STATE_SPACE_REMAIN_UNS = 0x00000020; struct ndmp_tape_get_state_reply { u_long unsupported; ndmp_error error; u_long flags; u_long file_num; u_long soft_errors; u_long block_size; u_long blockno; ndmp_u quad total_space; ndmp_u quad space_remain; }; Request Arguments This message does not have a message body. Expires June 2001 [Page 77] Draft Specification NDMP Version 4 Protocol November 2000 Reply Arguments unsupported This bit field identifies the arguments in the reply message whose values are not set by the server. error Error code. flags A bit field of the following tape device modes: NDMP_TAPE_STATE_NOREWIND Upon device close, the tape will not be rewound. NDMP_TAPE_STATE_WR_PROT The tape currently loaded is write protected. NDMP_TAPE_STATE_ERROR A media error was detected during the previous tape operation; this bit is cleared at the start of each tape operation. NDMP_TAPE_STATE_UNLOAD The tape currently loaded will be unloaded when the device is closed. file_num Current file number: the number of file marks between BOT and the current tape position. This value shall be set to all ones if it is unknown to or unsupported by the server, or if there is no tape currently loaded in the tape drive. Further, if the server does not support this value, it SHALL set the NDMP_TAPE_STATE_FILE_NUM_UNS bit in the "unsupported" bit field to one. soft_errors Total number of soft media errors detected since the device was opened. This value SHALL be set to all ones if it is unknown to or unsupported by the server. Further, if the server does not support this value, it SHALL set the NDMP_TAPE_STATE_SOFT_ERRORS_UNS bit in the "unsupported" bit field to one. Expires June 2001 [Page 78] Draft Specification NDMP Version 4 Protocol November 2000 block_size If the tape drive is in fixed block size mode, this is the fixed tape block size in bytes. If the tape is in variable block size mode, this value is zero. If the server supports this value, it SHALL always report it to the client. If the server does not support this value, it SHALL set it to all ones and set the NDMP_TAPE_STATE_BLOCK_SIZE_UNS bit in the "unsupported" bit field to one. blockno The current block number: the number of tape blocks between the current tape position and either the preceding file mark or BOT, which ever occurs nearest. This value SHALL be set to all ones if it is unknown to or unsupported by the server. Further, if the server does not support this value, it SHALL set the NDMP_TAPE_STATE_BLOCKNO_UNS bit in the "unsupported" bit field to one. total_space The total tape capacity in bytes. If the tape drive is capable of data compression, this is the total capacity with compression disabled (regardless of its actual state). This value SHALL be set to all ones if it is unknown to or unsupported by the server. Further, if the server does not support this value, it SHALL set the NDMP_TAPE_STATE_TOTAL_SPACE_UNS bit in the "unsupported" bit field to one. space_remain The total remaining tape capacity in bytes. If the tape drive is capable of data compression, this is the total remaining tape capacity with compression disabled (regardless of its actual state). This value SHALL be set to all ones if it is unknown to or unsupported by the server. Further, if the server does not support this value, it SHALL set the NDMP_TAPE_STATE_SPACE_REMAIN_UNS bit in the "unsupported" bit field to one. Reply errors: NDMP_NO_ERR Tape state successfully returned. NDMP_DEV_NOT_OPEN_ERR No tape device is presently open by the connection. NDMP_NOT_SUPPORTED_ERR This request is not supported by this server. Expires June 2001 [Page 79] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_NOT_AUTHORIZED_ERR The connection is not authorized. NDMP_IO_ERR A device I/O error occurred while processing this request. Expires June 2001 [Page 80] Draft Specification NDMP Version 4 Protocol November 2000 3.4.5. NDMP_TAPE_MTIO This request provides access to common magnetic tape I/O operations. Message XDR definition /* NDMP_TAPE_MTIO */ enum ndmp_tape_mtio_op { NDMP_MTIO_FSF, NDMP_MTIO_BSF, NDMP_MTIO_FSR, NDMP_MTIO_BSR, NDMP_MTIO_REW, NDMP_MTIO_EOF, NDMP_MTIO_OFF }; struct ndmp_tape_mtio_request { ndmp_tape_mtio_op tape_op; u_long count; }; struct ndmp_tape_mtio_reply { ndmp_error error; u_long resid_count; }; Request Arguments tape_op One of the following tape operations: NDMP_MTIO_FSF Forward space over file marks. The tape is positioned on the EOT side of the last file mark skipped. Should this operation encounter blank tape, NDMP_NO_ERR and a non-zero are reported and the tape is positioned on the EOT side of the last recorded tape block or file mark. NDMP_MTIO_BSF Backward space over file marks. The tape is positioned on the BOT side of the last file mark skipped, such that the next read or NDMP_MTIO_FSR encounters EOF. Should this operation encounter beginning of tape, NDMP_NO_ERR and a non-zero are reported and the tape is positioned at BOT. Expires June 2001 [Page 81] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_MTIO_FSR Forward space over tape blocks. The tape is positioned on the EOT side of the last block skipped. When a file mark is encountered as a result of an NDMP_MTIO_FSR operation, the server reports a non-zero and NDMP_NO_ERR. In such case, the server leaves the tape positioned at the BOT side of the file mark. Subsequent NDMP_MTIO_FSR operations return a equal to and leave the tape position unchanged. Blank tape encounters are handled identically as for NDMP_MTIO_FSF. NDMP_MTIO_BSR Backward space over tape blocks. The tape is positioned on the BOT side of the last tape block skipped. When a file mark is encountered as a result of an NDMP_MTIO_BSR operation, the server returns a non-zero and NDMP_NO_ERR. In such case, the server leaves the tape positioned at the EOT side of the file mark. Subsequent NDMP_MTIO_BSR operations return a equal to and leave the tape position unchanged. Beginning of tape encounters are handled identically as for NDMP_MTIO_BSF. NDMP_MTIO_REW Rewind the tape to BOT. The value of is ignored. Implementers may choose whether NDMP_MTIO_REW is an "immediate" operation or not. Regardless, any subsequent TAPE operation MUST NOT fail because an immediate operation is incomplete. This requirement does not apply to NDMP_TAPE_EXECUTE_CDB. NDMP_MTIO_EOF Write end of file marks. If this operation encounters logical end of medium, it succeeds and reports NDMP_NO_ERR and a zero . Implementers may choose whether NDMP_MTIO_EOF is an "immediate" operation or not. Regardless, any subsequent TAPE operation MUST NOT fail because an immediate operation is incomplete. This requirement does not apply to NDMP_TAPE_EXECUTE_CDB. Expires June 2001 [Page 82] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_MTIO_OFF Eject the tape from the device. The value of is ignored. Implementers may choose whether NDMP_MTIO_OFF is an "immediate" operation or not. Regardless, any subsequent TAPE operation MUST NOT fail because an immediate operation is incomplete. This requirement does not apply to NDMP_TAPE_EXECUTE_CDB. Also, implementers MAY choose whether to position the tape to BOT, EOT or some intermediate point before ejecting it. count Number of operations to perform. The field is ignored for NDMP_MTIO_REW and NDMP_MTIO_OFF operations. For all other operations, a zero count causes the tape position to be left unchanged, NDMP_NO_ERR and a of zero are reported. Reply Arguments error Error code. resid_count Residual operation count. Represents the number of operations that could not be done due to encountering beginning of tape, end of tape, end of written media, or a tape error. Reply Errors NDMP_NO_ERR Tape operation successfully completed. NDMP_DEV_NOT_OPEN_ERR No tape device currently open by the connection. NDMP_IO_ERR Device I/O error. NDMP_ILLEGAL_ARGS_ERR Invalid tape operation specified. NDMP_NOT_SUPPORTED_ERR The request is not supported by this implementation. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. NDMP_WRITE_PROTECT_ERR Tape is write protected. Expires June 2001 [Page 83] Draft Specification NDMP Version 4 Protocol November 2000 3.4.6. NDMP_TAPE_WRITE This request writes data to the tape device. The number of tape blocks written depends on the mode of the tape drive: * In variable block size mode, the NDMP server writes bytes of data to one tape block. * In fixed block size mode, the NDMP server writes the data to the number of tape blocks computed as specified earlier. It is the client's responsibility to ensure that is a multiple that fixed block size. A client uses NDMP_TAPE_GET_STATE to sense whether a device is in fixed or variable block size mode. Setting fixed or variable block size mode -- for devices for which it is configurable -- is outside the scope of this specification. The NDMP server does not buffer or pad the data. This request is typically used by the DMA to write tape header and trailer file data. When a write operation succeeds but encounters logical end of medium, it reports NDMP_NO_ERR and bytes written. ("Logical end of medium is generated when crossing the SCSI "early warning indicator.") The next write operation fails, reporting zero bytes written and NDMP_EOM_ERR. Subsequent write operations succeed with NDMP_NO_ERR reported. If physical EOM is reached, the write operation fails with NDMP_IO_ERR. Message XDR definition /* NDMP_TAPE_WRITE */ struct ndmp_tape_write_request { opaque data_out<>; }; struct ndmp_tape_write_reply { ndmp_error error; u_long count; }; Request Arguments data_out The data to be written to the tape device. Expires June 2001 [Page 84] Draft Specification NDMP Version 4 Protocol November 2000 Reply Arguments error Error code. count Number of data bytes written. Reply Errors NDMP_NO_ERR All data successfully written to the tape device. NDMP_DEV_NOT_OPEN_ERR No tape device currently open by the connection. NDMP_IO_ERR Device I/O error. This error MAY be returned if no tape is loaded (see above). Also returned if the physical end of medium was encountered. NDMP_EOM_ERR Logical end of medium (SCSI early warning indicator) was encountered; no data was written. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. NDMP_WRITE_PROTECT_ERR Tape is write protected. NDMP_PERMISSION_ERR The device is open in NDMP_TAPE_READ_MODE. NDMP_NO_TAPE_LOADED_ERR If the server is able to distinguish this condition from a general NDMP_IO_ERR, it reports this error when no tape is loaded. NDMP_DEVICE_BUSY_ERR A MOVER associated with this tape drive is either in an active or listen state. 3.4.7. NDMP_TAPE_READ This request reads data from the tape drive. The number of tape blocks read depends on the mode of the tape drive: Expires June 2001 [Page 85] Draft Specification NDMP Version 4 Protocol November 2000 * In variable block size mode, the NDMP server reads one tape block and returns, at most, bytes of data. If the tape block contains more than bytes, that data is discarded. * In fixed block size mode, the NDMP server reads data from the number of tape blocks computed as described earlier. It is the client's responsibility to ensure that is a multiple of the fixed block size. If the last tape block read contains excess data, that data is discarded. When a file mark is encountered in lieu of the first data block to read, the server returns zero data bytes and an NDMP_EOF_ERR error. In such case, the server leaves the tape positioned at the BOT side of the file mark. Subsequent reads encounter the same file mark and act identically. When blank tape (end of recorded data) is encountered in lieu of the first data block to read, the server returns zero data bytes and an NDMP_EOM_ERR error. The tape remains positioned on the EOT side of the last recorded tape block or file mark. Should an NDMP_TAPE_READ operation encounter a file mark or blank tape on the second or subsequent tape block read from a drive in fixed block size mode, all data from blocks read are returned, the value of is set to the actual number of data bytes returned and NDMP_NO_ERR is reported. The server leaves the tape positioned on the EOT side of the last block read, such that the next NDMP_TAPE_READ will report the condition that caused the early termination of this operation. If a tape drive is open and a client requests a read with a equal to zero, no action occurs, NDMP_NO_ERR is generated, and a of zero is returned. Servers behave this way regardless of any other state of the tape drive. Upon successful completion of NDMP_TAPE_READ, the tape is positioned on the EOT side of the last tape block read. Expires June 2001 [Page 86] Draft Specification NDMP Version 4 Protocol November 2000 Message XDR definition /* NDMP_TAPE_READ */ struct ndmp_tape_read_request { u_long count; }; struct ndmp_tape_read_reply { ndmp_error error; opaque data_in<>; }; Request Arguments count Number of bytes to read. Reply Arguments error Error code. data_in The data read from the tape drive. Reply Errors NDMP_NO_ERR A read from the tape was successful. NDMP_DEV_NOT_OPEN_ERR No tape device currently open by the connection. NDMP_IO_ERR Device I/O error during read. The tape position following this error is undetermined. Also, this error may be returned if no tape is loaded. NDMP_NO_TAPE_LOADED_ERR If the server is able to distinguish this condition from a general NDMP_IO_ERR, it reports this error when no tape is loaded. NDMP_EOF_ERR End of file was encountered while reading. A data_in_len of zero is returned. NDMP_EOM_ERR A blank tape was encountered while reading. A data_in_len of zero is returned. Expires June 2001 [Page 87] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_DEVICE_BUSY_ERR A MOVER associated with this tape drive is either in an active or listen state. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 88] Draft Specification NDMP Version 4 Protocol November 2000 3.4.8. NDMP_TAPE_EXECUTE_CDB This message behaves in exactly the same way as the SCSI_EXECUTE_CDB request except that it sends the CDB to the tape device. This request SHOULD not be used to change the state of the tape device (such as tape positioning). Message XDR definition /* NDMP_TAPE_EXECUTE_CDB */ typedef ndmp_scsi_execute_cdb_request ndmp_tape_execute_cdb_request; typedef ndmp_scsi_execute_cdb_reply ndmp_tape_execute_cdb_reply; Expires June 2001 [Page 89] Draft Specification NDMP Version 4 Protocol November 2000 3.5. Data Interface This interface controls backup and recover operations. 3.5.1. NDMP_DATA_GET_STATE This request returns data state information that can be used to monitor the progress of the current data operation. Expires June 2001 [Page 90] Draft Specification NDMP Version 4 Protocol November 2000 Message XDR definition /* NDMP_DATA_GET_STATE */ /* no request arguments */ enum ndmp_data_operation { NDMP_DATA_OP_NOACTION, NDMP_DATA_OP_BACKUP, NDMP_DATA_OP_RESTORE }; enum ndmp_data_state { NDMP_DATA_STATE_IDLE, NDMP_DATA_STATE_ACTIVE, NDMP_DATA_STATE_HALTED, NDMP_DATA_STATE_LISTEN, NDMP_DATA_STATE_CONNECTED }; enum ndmp_data_halt_reason { NDMP_DATA_HALT_NA, NDMP_DATA_HALT_SUCCESSFUL, NDMP_DATA_HALT_ABORTED, NDMP_DATA_HALT_INTERNAL_ERROR, NDMP_DATA_HALT_CONNECT_ERROR, }; /* ndmp_addr */ struct ndmp_tcp_addr { u_long ip_addr; u_short port; }; struct ndmp_ipc_addr { opaque comm_data<>; }; union ndmp_addr switch (ndmp_addr_type addr_type) { case NDMP_ADDR_LOCAL: void; case NDMP_ADDR_TCP: ndmp_tcp_addr tcp_addr; case NDMP_ADDR_IPC: ndmp_ipc_addr ipc_addr; }; /* unsupported bitmask bits */ Expires June 2001 [Page 91] Draft Specification NDMP Version 4 Protocol November 2000 const NDMP_DATA_STATE_EST_BYTES_REMAIN_UNS = 0x00000001; const NDMP_DATA_STATE_EST_TIME_REMAIN_UNS = 0x00000002; struct ndmp_data_get_state_reply { u_long unsupported; ndmp_error error; ndmp_data_operation operation; ndmp_data_state state; ndmp_data_halt_reason halt_reason; ndmp_u_quad bytes_processed; ndmp_u_quad est_bytes_remain; u_long est_time_remain; ndmp_addr data_connection_addr; ndmp_u_quad read_offset; ndmp_u_quad read_length; }; Request Arguments This message does not have a message body. Reply Arguments unsupported This bitmask is used to identify any unsupported arguments in the reply message. error Error code. operation Data operation currently in progress. NDMP_DATA_OP_NOACTION No data operation currently in progress. NDMP_DATA_OP_BACKUP Backup operation currently in progress. NDMP_DATA_OP_RESTORE Restore operation currently in progress. state Current state of the NDMP server. NDMP_DATA_STATE_IDLE No active data operation. NDMP_DATA_STATE_ACTIVE Data operation in progress. Expires June 2001 [Page 92] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_DATA_STATE_HALTED Data operation completed. NDMP_DATA_STATE_LISTEN Wait for the establishment of the connection. NDMP_DATA_STATE_CONNECTED Data connection is established. halt_reason Reason the data operation is halted. NDMP_DATA_HALT_NA Data operation not in progress or not in the halt state. NDMP_DATA_HALT_SUCCESSFUL Data operation completed successfully. NDMP_DATA_HALT_ABORTED Data operation aborted by the DMA. NDMP_DATA_HALT_INTERNAL_ERROR Data operation halted due to unrecoverable error incurred by the NDMP server data backup/recover software. NDMP_DATA_HALT_CONNECT_ERROR Error establishing connection to tape server. bytes_processed Total number of bytes processed by the data operation. est_bytes_remain Estimated number of bytes processed remaining to be processed by the data operation. NDMP_DATA_STATE_EST_BYTES_REMAIN_UNS is set if this feature is not supported. est_time_remain Estimated number of seconds until the data operation to completes. NDMP_DATA_STATE_EST_TIME_REMAIN_UNS is set if this feature is not supported. data_connection_addr When the data server listens a data connection: In the LISTEN state, data_connection_addr is the address to listen the data connection. While the state transition to CONNECTED or ACTIVE State, data_connection_addr is the address of client, which connects to this data server. In the IDLE state, the field is ignored. Expires June 2001 [Page 93] Draft Specification NDMP Version 4 Protocol November 2000 When the data server connects to the other NDMP server: In the ACTIVE state, data_connection_addr is the address of server, which listens the data connection. In the IDLE state, the field is ignored. read_offset Offset value specified in last NDMP_NOTIFY_DATA_READ request. read_len Length value specified in last NDMP_NOTIFY_DATA_READ request. Reply Errors NDMP_NO_ERR Data state successfully returned. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 94] Draft Specification NDMP Version 4 Protocol November 2000 3.5.2. NDMP_DATA_LISTEN The data server SHOULD begin listening for a data connection from the local or remote NDMP server. The NDMP_DATA_LISTEN message returns an address that is used to make the data connection. Message XDR definition /* NDMP_DATA_LISTEN */ struct ndmp_tcp_addr { u_long ip_addr; u_short port; }; struct ndmp_ipc_addr { opaque comm_data<>; }; union ndmp_addr switch (ndmp_addr_type addr_type) { case NDMP_ADDR_LOCAL: void; case NDMP_ADDR_TCP: ndmp_tcp_addr tcp_addr; case NDMP_ADDR_IPC: ndmp_ipc_addr ipc_addr; }; struct ndmp_data_listen_request { ndmp_addr_type addr_type; }; struct ndmp_data_listen_reply { ndmp_error error; ndmp_addr connect_addr; }; Request Arguments addr_type The specific type of data connection. Reply Arguments error Error code. Expires June 2001 [Page 95] Draft Specification NDMP Version 4 Protocol November 2000 connect_addr Address on which the data server is listening for a data connection. Reply Errors NDMP_NO_ERR Listen successful. NDMP_ILLEGAL_STATE_ERR The data server is not currently in idle state. NDMP_ILLEGAL_ARGS_ERR Invalid mode or address type specified. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 96] Draft Specification NDMP Version 4 Protocol November 2000 3.5.3. NDMP_DATA_CONNECT The data server SHOULD connect to the address specified by the local or remote NDMP server. If a failure occurs connecting to the address, an NDMP_CONNECT_ERR is returned. Message XDR definition /* NDMP_DATA_CONNECT */ struct ndmp_data_connect_request { ndmp_addr addr; }; struct ndmp_data_connect_reply { ndmp_error error; }; Request Arguments addr The address where a server is listening for a data connection. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR Connection successful. NDMP_ILLEGAL_STATE_ERR The data server is not currently in the idle state. NDMP_ILLEGAL_ARGS_ERR Invalid mode or address type specified. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. NDMP_CONNECT_ERR Failed to establish the data connection. Expires June 2001 [Page 97] Draft Specification NDMP Version 4 Protocol November 2000 3.5.4. NDMP_DATA_START_BACKUP This request begins a backup. The bu_type is the name of the backup type. The type of backup is implementation dependent. The env is a list of parameters that might affect the behavior of the backup. The env returned by the DATA_GET_ENV MUST be saved and made available to the NDMP Server when a recover is performed. Message XDR definition /* NDMP_DATA_START_BACKUP */ struct ndmp_data_start_backup_request { string bu_type<>; ndmp_pval env<>; }; struct ndmp_data_start_backup_reply { ndmp_error error; }; Request Arguments bu_type The name of the backup type. Backup types are NDMP server implementation dependent. env List of parameter names and values for configuring the backup type. Backup type parameters are NDMP server implementation dependent. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR Backup operation successfully started. NDMP_ILLEGAL_STATE_ERR A data operation is already in progress. Only one data operation per connection is allowed at a time. NDMP_NOT_SUPPORTED_ERR The request is not supported by this implementation. NDMP_ILLEGAL_ARGS_ERR Invalid backup type, invalid backup type parameter, or invalid backup type parameter value specified. Expires June 2001 [Page 98] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 99] Draft Specification NDMP Version 4 Protocol November 2000 3.5.5. NDMP_DATA_START_RECOVER This request recovers the files specified in nlist from the backup. The env is the list of parameters and values saved at the end of the backup. The bu_type is the name of the backup type. Message XDR definition /* NDMP_DATA_START_RECOVER */ struct ndmp_name { string original_path<>; string destination_dir<>; string new_name<>; string other_name<>; ndmp_u_quad node; ndmp_u_quad fh_info; }; struct ndmp_data_start_recover_request { ndmp_pval env<>; ndmp_name nlist<>; string bu_type<>; }; struct ndmp_data_start_recover_reply { ndmp_error error; }; Request Arguments env The backup environment that was returned from a data get environment request made prior to notifying the NDMP server that the backup was complete through a data stop message. nlist List of files to be recovered and the location to which each file is to be recovered. Definition of list entry: original_path Path of a file/directory to be recovered. The original_path is the original backup path name and is relative to the backup root directory. If the original_path is not specified, the entire backup image will be restored. destination_dir Destination directory to be used when recovering the file. If destination_dir is not specified, the files will be restored to the original directory. Expires June 2001 [Page 100] Draft Specification NDMP Version 4 Protocol November 2000 new_name This argument is used for the direct access retrieval only. If new_name is specified, the restored file is renamed to new_name in the destination_dir directory. other_name This argument is used for the direct access retrieval only and supported by some file systems, that support multiple file names for a single file. For example, NT File Systems support both NT file names and dos file names. node This argument is used for the direct access retrieval only. node is used to verify the retrieval if the backup method supports inode type of backup, e.g. dump. The argument is set to 0 if not supported. fh_info File history tape positioning data recorded when the file was backed up. This data may be used by the restore method to position tape for direct access data retrieval. The positioning data is NDMP server dependent. Typically it will be the byte or record offset from the beginning of the tape of the file to be recovered. This field is ignored by data method implementations that do not support this feature. bu_type Name of the recover method. Recover methods are NDMP server implementation dependent. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR Recover operation successfully started. NDMP_ILLEGAL_STATE_ERR A data operation is already in progress. Only one data operation per connection is allowed to execute at a time. NDMP_ILLEGAL_ARGS_ERR Invalid recover method, invalid recover method parameter, invalid recover method parameter value, or invalid name list entry specified. NDMP_NOT_SUPPORTED_ERR The request is not supported by this implementation. Expires June 2001 [Page 101] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 102] Draft Specification NDMP Version 4 Protocol November 2000 3.5.6. NDMP_DATA_ABORT This request sends a message to abort the current backup or restore operation. The NDMP Server SHOULD terminate the operation as soon as practical. Message XDR definition /* NDMP_DATA_ABORT */ /* no request arguments */ struct ndmp_data_abort_reply { ndmp_error error; }; Request Arguments This message does not have a request body. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR Data operation successfully terminated. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 103] Draft Specification NDMP Version 4 Protocol November 2000 3.5.7. NDMP_DATA_STOP This request sends a message to inform the NDMP server that the current backup is complete. The NDMP server MUST change to the idle state and prepare to process another request. The environment variable set defined from the previous backup MUST be cleared. Message XDR definition /* NDMP_DATA_STOP */ /* no request arguments */ struct ndmp_data_stop_reply { ndmp_error error; }; Request Arguments This message does not have a message body. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR Message successfully processed. NDMP_ILLEGAL_STATE_ERR The request cannot be run in the current state. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 104] Draft Specification NDMP Version 4 Protocol November 2000 3.5.8. NDMP_DATA_GET_ENV This request gets the backup environment variable set. It returns the environment set specified in the NDMP_DATA_START_BACKUP request along with any additional parameters added or modified by the backup method. The returned environment set MUST be saved by the DMA and passed back to the NDMP Server in the NDMP_DATA_START_RECOVER request whenever data from the backup is to be recovered. Message XDR definition /* NDMP_DATA_GET_ENV */ /* no request arguments */ struct ndmp_data_get_env_reply { ndmp_error error; ndmp_pval env<>; }; Request Arguments This message does not have a message body. Reply Arguments error Error code. env The backup method environment parameters and values. Reply Errors NDMP_NO_ERR Environment successfully returned. NDMP_NOT_SUPPORTED_ERR The request is not supported for this implementation. NDMP_ILLEGAL_STATE_ERR Illegal state. A data operation is not currently in progress. NDMP_NOT_AUTHORIZED_ERR Connection not authorized. Expires June 2001 [Page 105] Draft Specification NDMP Version 4 Protocol November 2000 3.6. Mover Interface The mover interface manages the transfer of backup stream data between a data or tape server and the local tape subsystem. The mover interface also provides control over the size and location of the mover window relative to the start of a backup stream. Mover windows represent logical boundaries of tape control and provide a mechanism for differentiating backup stream data from meta data on NDMP generated tapes. Data that resides outside of the mover window is controlled by the DMA and represents meta data such as header and trailer information. Data written within the mover window is controlled by the data server and represents the backup stream data. The DMA is responsible for establishing a mover window that differentiates meta data from backup stream data. Mover windows can also represent physical boundaries of the backup stream layout on tape. The window represents the portion or segment of the backup stream data that can be accessed by the tape subsystem without requesting DMA controlled tape change or tape positioning intervention. 3.6.1. Mover Interface Overview The mover interface consists of ten unique request/reply message pairs. These message pairs can be loosely categorized as providing connection management, data transfer management, and status reporting. NDMP_MOVER_LISTEN, NDMP_NOVER_CONNECT, NDMP_MOVER_ABORT, NDMP_MOVER_STOP and NDMP_MOVER_CLOSE provide control over the mover data connection. NDMP_MOVER_SET_RECORD_SIZE, NDMP_MOVER_SET_WINDOW, NDMP_MOVER_READ and NDMP_MOVER_CONTINUE control the transfer of data between servers and the local tape subsystem. NDMP_MOVER_GET_STATE provides a method of querying the mover for status information. During a backup operation the mover reads the backup stream from the data connection, buffers the data into tape records, and writes data to the tape subsystem. During a recover operation the mover reads data from the tape subsystem and writes the backup stream to the data connection. The mover is also responsible for handling tape exceptions and notifying the DMA when tape related intervention is required. During a backup operation, window length can be used to partition the data stream into multiple stream segments by limiting the amount of data written to each segment. This provides the DMA an opportunity to interleave meta data between the backup stream segments. During a recover operation, the DMA is responsible for positioning the tape over any non-backup stream meta data (headers, etc). The DMA MUST also establish a mover window representing the size and location of the current backup stream segment within the entire backup stream image. Expires June 2001 [Page 106] Draft Specification NDMP Version 4 Protocol November 2000 During a recover operation, the DMA may receive several successive data read requests from the DATA server. The DMA MUST forward these requests to the mover as mover read requests without changing the offset or length arguments. If a mover read offset falls within the currently defined mover window, the mover is expected to position to that offset relative to the start of the window with some simple math. If a mover read specifies an offset that falls outside the current mover window, the mover MUST pause and notify the DMA that a new tape position is required. The DMA MUST examine the requested offset and with the aid of information in its catalog, determine how to get to the correct offset (skipping over meta data on the same tape or switching to the tape containing the desired offset). The DMA MUST then set a new mover window, position the tape to the start of the window, and instruct the mover to continue the read operation. Note: upon receipt of the NDMP_MOVER_CONTINUE request, the mover is responsible for positioning to the required offset within the new window. The mover window avoids the need for the DMA to control tape positioning for each data read it receives. In the case of a local recover operation where the data server and the mover are on the same system, the DMA does not receive data read messages. Therefore the mover MUST be capable of performing tape positioning within a window. In addition to backup and recover operations where backup stream data transfer occurs between data servers and mover/tape servers, copy operations are also supported. A data connection between two movers provides the basis for NDMP tape duplication. This occurs when one mover performs a backup operation and the other a recovery operation. 3.6.2. Mover Interface Variables & Constants There are a number of variables and constants that are key to the operation of the mover interface. These variables are exposed by the NDMP protocol definition and MUST be maintained in a consistent manner by all NDMP implementations. The definitions contained in this section will be referenced in subsequent mover interface sections. mover_state This integer value identifies the current state of the NDMP Tape Server's mover state machine. Valid mover states are defined by the ndmp_mover_state enumeration and consist of IDLE, LISTEN, ACTIVE, PAUSED, and HALTED. Refer to section 2 for a complete definition of the mover state machine. Expires June 2001 [Page 107] Draft Specification NDMP Version 4 Protocol November 2000 pause_reason This integer value identifies the event that caused the mover state machine to enter the PAUSED state. Pause reason events are defined by the ndmp_mover_pause_reason enumeration. The pause reason MUST be set to NDMP_MOVER_PAUSE_NA whenever the mover state machine is initialized and MUST be set to NDMP_MOVER_PAUSE_EOM (end of media), NDMP_MOVER_PAUSE_EOF (end of file), NDMP_MOVER_PAUSE_SEEK (seek required), or NDMP_MOVER_PAUSE_EOW (end of window) as appropriate upon transition to the PAUSED state. The pause reason MUST be set to NDMP_MOVER_PAUSE_NA upon mover transition out of the PAUSED state. The pause reason is valid only when the mover is in the PAUSED state. NDMP_MOVER_PAUSE_NA The mover is not in the paused state. NDMP_MOVER_PAUSE_EOM The last mover operation (read or write) encountered an End of Media condition. DMA intervention is required. NDMP_MOVER_PAUSE_EOF The last mover operation (read or write) encountered an End of File condition. DMA intervention is required. NDMP_MOVER_PAUSE_SEEK The last mover operation (read) exceeded the bounds of the current mover window. DMA intervention is required. NDMP_MOVER_PAUSED_EOW The last mover operation (write) exceeded the bounds of the current mover window. DMA intervention is required. Following a transition to the PAUSED state, the mover MUST issue a NDMP_NOTIFY_MOVER_PAUSED message to identify the pause reason and request corrective action by the DMA. Transition to the PAUSED state can result from expected or unexpected conditions. After the appropriate corrective action is taken, paused mover operations are resumed when the DMA issues a NDMP_MOVER_CONTINUE request causing the mover to transition back to the ACTIVE state. Expires June 2001 [Page 108] Draft Specification NDMP Version 4 Protocol November 2000 halt_reason This integer value identifies the event that caused the mover state machine to enter the HALTED state. Valid halt reasons are defined by the ndmp_mover_halt reason enumeration. The halt reason MUST be set to NA when the mover state machine is initialized and MUST be set to NDMP_MOVER_HALT_CONNECTION_CLOSED, NDMP_MOVER_HALT_ABORTED, NDMP_MOVER_HALT_INTERNAL_ERROR, or NDMP_MOVER_HALT_CONNECT_ERROR as appropriate upon transition to the HALTED state. The halt reason MUST be set to NDMP_MOVER_HALT_NA upon mover transition out of the HALTED state. The halt reason is valid only when the mover is in the HALTED state. NDMP_MOVER_HALT_NA The mover is not in the halted state. NDMP_MOVER_HALT_CONNECT_CLOSED The mover's connection to the data server was closed. NDMP_MOVER_HALT_ABORTED The mover received an ndmp_mover_abort_request from the DMA. NDMP_MOVER_HALT_INTERNAL_ERROR The mover detected an unrecoverable error condition. NDMP_MOVER_HALT_CONNECT_ERROR The mover detected a connection failure while in the LISTEN state. NDMP_MOVER_HALT_MEDIA_ERROR The mover encountered a non-recoverable error while reading from or writing to tape. Following a transition to the HALTED state, the mover MUST issue an NDMP_NOTIFY_MOVER_HALTED message to identify the halt reason and allow the DMA to cleanup. Transition to the HALTED state can result from expected or unexpected conditions. In progress mover operations MUST NOT continue after a transition to the HALTED state. record_size This unsigned long value represents the current mover record size in bytes. The tape server MUST set the record size to zero when the mover state machine is initialized for the first time. Since zero is not a valid operational value for mover record size, the mover record size MUST be explicitly set by the DMA before the mover transitions out of the IDLE state. Record size MAY also be updated by the DMA when the mover is in the PAUSED state. Record size is persistent between mover connections and state transitions and remains in effect until reestablished by the DMA. Expires June 2001 [Page 109] Draft Specification NDMP Version 4 Protocol November 2000 The DMA establishes a mover record size by sending an NDMP_MOVER_SET_RECORD_SIZE_request. The mover record size MUST be set to a multiple of the tape block size when the tape subsystem is operating in fixed block mode. When in variable block mode, as indicated by a tape block_size value of zero, the mover record size defines the actual block size used by the tape subsystem. record_number This unsigned long value represents the last tape record processed by the mover. Record number MUST be set to zero whenever the mover transitions to the IDLE state. Record number is updated upon receipt of a NDMP_MOVER_SET_WINDOW_request, as a result of mover tape positioning operations, and whenever the mover transfers backup data to or from the tape subsystem. A DMA MAY change the record number by sending a NDMP_MOVER_SET_WINDOW request. However, this can only be done when the mover is in the IDLE state. Upon receipt of a set window request, the mover record number MUST be set to the window offset divided by the mover record size. Record number MUST be incremented each time the mover reads or writes a mover record from the tape subsystem regardless of whether any data was transferred to the data connection. As the mover initiates forward or backward tape positioning operations it MUST update the record_number appropriately to reflect the new position. Record number reflects only data records processed by the mover. It does not include file marks or meta data processed via the tape interface. bytes_moved This double unsigned long value represents the cumulative number of data stream bytes written to the data connection or the number of data stream bytes read from the data connection and written to the tape subsystem, depending on the mode of mover operation. Bytes moved MUST be set to zero whenever the mover transitions to the IDLE state. When the mover is in write mode (transferring data to the data connection), bytes moved MUST be incremented with the actual data byte count each time the mover writes data to the data connection. Bytes moved does not represent the number of data bytes transferred from the tape subsystem. For example the mover record size can be greater than the mover read request length, resulting in data read from tape that is not transferred to the data connection. Expires June 2001 [Page 110] Draft Specification NDMP Version 4 Protocol November 2000 When the mover is in read mode (transferring data to the tape subsystem) bytes moved MUST be incremented with the actual count of data stream bytes following each successful transfer of data from the data connection to the tape subsystem. The bytes moved value does not include any trailing pad data used to align the data stream segment to a full tape record. When the mover is in read mode and in either the PAUSED or HALTED state, the DMA MAY reference bytes_moved to determine the data stream segment size actually written to the tape subsystem. seek_position This double unsigned long value represents the data stream offset of the first byte the DMA requested the mover to transfer to the data connection during a mover read operation. Seek position MUST be set to zero whenever the mover transitions to the IDLE state. Upon receipt of a NDMP_MOVER_READ request, seek position MUST be set to the specified read offset value of the request. Seek position is not updated as a result of read operations from the tape subsystem. Its purpose is to allow the DMA to query the mover to determine the start of the last tape read operation. bytes_left_to_read This double unsigned long value represents the number of data bytes remaining to be transferred to the data connection to satisfy the current NDMP_MOVER_READ request. Bytes left to read MUST be set to zero whenever the mover transitions to the IDLE state. Upon receipt of a mover read request, the bytes_left_to_read value MUST be set to the specified read length value. During a mover read operation, the bytes left to read value MUST be decremented by the number of bytes successfully written to the data connection. A bytes left to read value of zero indicates that the last mover read operation completed and that the mover is waiting for the next read request. Data is not always transferred from the tape subsystem to the data connection in mover record size units. Since the data connection is a flow-controlled stream, it is possible that the transfer of a single mover record will require multiple writes to the data connection. The bytes left to read value MUST accurately represent the actual amount of data remaining to be transferred to the data connection. The data represented by bytes_left_to_read can reside either on tape or buffered within the mover. Expires June 2001 [Page 111] Draft Specification NDMP Version 4 Protocol November 2000 window_offset This double unsigned value represents the absolute offset of the first byte of the mover window within the overall data stream. The window offset and window length (described below) together define the portion of the overall data stream that is accessible to the mover without intervening DMA tape manipulation. Window offset is only applicable to recover operations and has no meaning for backup operations. Window offset MUST be set to zero whenever the mover transitions to the IDLE state Upon receipt of an NDMP_MOVER_SET_WINDOW request, while in either the mover IDLE or PAUSED state, the mover window offset MUST be set to the data stream offset value specified in the set window request. Prior to issuing a set window request, the DMA is expected to position the tape so that the next byte read will be from the specified data stream offset. Window offset is not updated as result of mover data transfer or tape positioning operations. The only events that cause window offset updates are set window requests and transitions to the IDLE state. window_length This double unsigned long value represents the length of the current mover window in bytes. The window length and window offset (described above) together define the portion of the overall data stream that is accessible to the mover without intervening DMA tape manipulation. Window length is applicable to both backup and recover operations. For backup operations, window length MAY be used to partition the backup stream into multiple stream segments by limiting the amount of data written to each segment. This provides the DMA an opportunity interleave meta data between the data stream segments. Window length MUST be set to zero whenever the mover transitions to the IDLE state; indicating an invalid window definition. The DMA MUST establish a valid window size and endpoint by issuing a NDMP_MOVER_SET_WINDOW request. Upon receipt of a set window request, while in either the IDLE or PAUSED state, the window length MUST be set to the length value specified in the request. Window length MUST be set to a multiple of the mover record_size except when specifying a mover window prior to a recover operation that will include the last mover record of the backup stream. In this case the window length MUST NOT be greater than the end of the backup stream and MUST NOT include any pad bytes written to tape. Window length is not updated as result of mover data transfer or tape positioning operations. The only events that cause window length updates are set window requests and transitions to the IDLE state. Expires June 2001 [Page 112] Draft Specification NDMP Version 4 Protocol November 2000 data_connection_address This structure represents the connection endpoint information for the passive peer (listening data or tape server) of a data connection. The type of data connection determines the format of the address as follows. A connection between systems can be established via TCP/IP (NDMP_ADDR_TCP). A connection within a system can be either null (NDMP_ADDR_LOCAL) or inter-process (NDMP_ADDR_IPC). 3.6.3. Mover Message Definitions The following section defines each of the ten mover interface request/reply message pairs. Message pair definitions are presented in typical usage order: set record size, set window, connect, listen, read, get state, continue, close, abort, and stop. NDMP server support of the mover interface is OPTIONAL. It is possible for a server to implement the data interface without the mover interface. However, if the mover interface is implemented, all ten mover request messages MUST be supported. If the mover interface is not implemented, any mover request message MUST result in a NDMP_NOT_SUPPORTED error reply. Expires June 2001 [Page 113] Draft Specification NDMP Version 4 Protocol November 2000 3.6.3.1. NDMP_MOVER_SET_RECORD_SIZE This request is used by the DMA to establish the record size used for mover initiated tape read and write operations. The mover record size MUST be set to a multiple of the tape block size when the tape subsystem is operating in fixed block mode. When in variable block mode, as indicated by a tape block_size value of zero, the mover record size defines the actual block size used by the tape subsystem. The mover record size MUST be set to zero when the mover state machine is initialized for the first time. The mover record size MUST be explicitly set to a valid operational value by the DMA before the mover transitions out of the IDLE state. NDMP_MOVER_CONNECT and NDMP_MOVER_LISTEN requests MUST fail if the DMA has not previously establish a valid mover record size. Record size MAY also be updated by the DMA when the mover is in the PAUSED state. Record size is persistent between mover connections and state transitions and remains in effect until reestablished by the DMA. A DMA defined mover record size is not reset by subsequent mover transitions to the IDLE state. During backup operations, the mover buffers the backup stream read from the data connection until a full mover record is received, then writes the mover record to the tape subsystem. During recover operations, the mover requests full mover records from the tape subsystem, then writes some or all of the mover record to the data connection as required to satisfy the current mover read request. Depending on the mover record size, one or more tape blocks may be required to complete the read. Message XDR definition /* NDMP_MOVER_SET_RECORD_SIZE */ struct ndmp_mover_set_record_size_request { u_long len; }; struct ndmp_mover_set_record_size_reply { ndmp_error error; }; Request Arguments len The length of the mover record specified in bytes. Reply Arguments error Error code. Expires June 2001 [Page 114] Draft Specification NDMP Version 4 Protocol November 2000 Reply Errors NDMP_NO_ERR The mover set record size request was successfully processed. The specified mover record size is now in effect. NDMP_ILLEGAL_ARGS_ERR The mover set record size request specified an invalid record size. Maximum mover record size is implementation dependent but MUST be set to a multiple of the tape block size when the tape subsystem is operating in fixed block mode. NDMP_ILLEGAL_STATE_ERR The mover set record size request was received while the mover state machine was in a state that prevented processing this request. Set record requests are only valid in the IDLE or PAUSED states. Expires June 2001 [Page 115] Draft Specification NDMP Version 4 Protocol November 2000 3.6.3.2. NDMP_MOVER_SET_WINDOW This request establishes a mover window in terms of offset and length. A mover window represents the portion of the overall backup stream that is accessible to the mover without intervening DMA tape manipulation. The location and size of the mover window is specified by the set window request offset and length arguments where the offset is an absolute byte offset from the start of the data stream and the length is the byte length of the window. The window offset plus the window length MUST not result in an overflow condition. There is no default mover window. Whenever the mover transitions to the IDLE state, the mover window is marked invalid by setting both the offset and length to zero. The DMA MUST issue a set window request to establish a valid mover window before causing the mover to transition out of the IDLE state via a mover connect or mover listen operation. The DMA MUST also issue a set window request before causing the mover to transition out of the PAUSED state via a mover continue request. Prior to issuing a set window request, the DMA MUST position the tape so that the next byte read from tape will be from the data stream offset specified as the start of the window. The set window mover request may only be issued when the mover is in the IDLE or PAUSED states. A mover window length of all ones (binary) defines a maximum length window allowing restore operations to extend to tape file limits. This will result in an NDMP_NOTIFY_MOVER_PAUSED notification with a NDMP_MOVER_PAUSED_EOF reason rather than an NDMP_MOVER_PAUSE_SEEK reason if an EOW was detected. A window length of all ones (binary) MUST only be specified with a zero offset since the offset plus the length MUST not result in an overflow condition. If a maximum length window is required following a mover transition to the PAUSED state, a window length of all ones minus the current window offset MUST be specified. Other than allowing restore operations to extend to backup tape file limits, a maximum length window requires no special restore processing. During backup operations, window length MAY be used to partition the data stream into multiple stream segments by limiting the amount of data written to each segment. Detection of an EOW condition causes the mover to transition to the PAUSED state and issue a notify mover paused message to the DMA. This provides the DMA an opportunity to interleave meta data between the data stream segments. Window offset is not applicable to backup operations. Expires June 2001 [Page 116] Draft Specification NDMP Version 4 Protocol November 2000 During recover operations, the DMA MAY define a mover window to optimize selective file recovery by performing a Direct Access Recovery (DAR). The DMA SHOULD command the media changer (via the SCSI interface) to load the required tape, then position the tape (via the tape interface) to the data record corresponding to the desired mover window location. Note: the start of a mover window need not be aligned with the start of a tape file but MUST be aligned with a mover record. Once the tape is properly positioned, a mover window MUST be established to specify the data stream range the mover is allowed to access via one or more subsequent mover read requests. Message XDR definition /* NDMP_MOVER_SET_WINDOW */ struct ndmp_mover_set_window_request { ndmp_u_quad offset; ndmp_u_quad length; }; struct ndmp_mover_set_window_reply { ndmp_error error; }; Request Arguments offset The start of the mover window specified as an absolute byte offset from the start of the backup stream. This field is ignored if the mover is writing the backup stream to the tape subsystem. length The size of the mover window specified in bytes. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR The mover set window request was successfully processed. The specified mover window is now in effect. NDMP_ILLEGAL_ARGS_ERR The mover set window request specified an invalid window. A mover window length of zero is invalid. Additionally, the window offset plus the window length MUST not result in an overflow condition. Expires June 2001 [Page 117] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_ILLEGAL_STATE_ERR The mover set window request was received while the mover state machine was in a state that prevented processing this request. Set window requests are only valid in the IDLE or PAUSED states. NDMP_NOT_AUTHORIZED_ERR The NDMP server requires DMA authentication, but has not received a valid NDMP_CONNECT_CLIENT_AUTH request from the DMA. Expires June 2001 [Page 118] Draft Specification NDMP Version 4 Protocol November 2000 3.6.3.3. NDMP_MOVER_CONNECT This request is used by the DMA to instruct the mover to initiate an "active open" and create a data connection to a data or tape server. A connect request is only valid when the mover is in the IDLE state. A successful connect request causes the mover to transition to the ACTIVE state. A mover data connection is used to transfer backup stream data between the tape subsystem associated with the mover that initiated the connection and the data or tape subsystem specified in the connect request. The data connection can be established locally within a given system or between remote networked systems. The direction of the data transfer is specified by the connect mode argument as either reading from or writing to the data connection. The type of connection is specified by the addr_type argument. A connection within a system can be either null (ADDR_LOCAL) or inter process (ADDR_IPC), while a connection between systems can be established via TCP/IP (ADDR_TCP). Note: It is permissible to establish a connection between two movers for tape to tape transfers. Message XDR definition /* NDMP_MOVER_CONNECT */ struct ndmp_mover_connect_request { ndmp_mode mode; ndmp_addr addr; }; struct ndmp_mover_connect_reply { ndmp_error error; }; Request Arguments mode Specifies the direction of the data transfer as follows: NDMP_MOVER_MODE_READ The mover reads the backup stream from the data connection and writes the data to tape. This mode is used for backup operations. NDMP_MOVER_MODE_WRITE The mover reads the backup stream from tape and writes the data to the data connection. This mode is used for recover operations. Expires June 2001 [Page 119] Draft Specification NDMP Version 4 Protocol November 2000 addr Specifies the endpoint address the mover will use when establishing a data connection. The ndmp_addr structure conveys both the address type (NDMP_ADDRIPC, NDMP_ADDR_LOCAL, or NDMP_ADDR_TCP) as well as the address information appropriate for the specified type. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR The mover connect request was successfully processed. The mover has transitioned to the ACTIVE state and successfully connected to the specified server address. NDMP_ILLEGAL_STATE_ERR The mover connect request was received while the mover state machine was in a state that prevented processing this request. Connect requests are only valid in the IDLE state. NDMP_ILLEGAL_ARGS_ERR The mover connect request specified an invalid mode or address type. NDMP_NOT_AUTHORIZED_ERR The NDMP server requires DMA authentication, but has not received a valid NDMP_CONNECT_CLIENT_AUTH request from the DMA. NDMP_CONNECT_ERR Failed to establish the data connection. Expires June 2001 [Page 120] Draft Specification NDMP Version 4 Protocol November 2000 3.6.3.4. NDMP_MOVER_LISTEN This request is used by the DMA to instruct the mover to prepare for a "passive open" by creating a connection end point and listening for a subsequent data connection from a data or tape server. This request is also used by the DMA to obtain the connection end point address information from the mover. A listen request is only valid when the mover is in the IDLE state. A successful listen request causes the mover to transition to the LISTEN state. The mover will remain in the LISTEN state until a data connection is established resulting in a transition to the ACTIVE state, or until the mover enters the HALTED state following the detection of an internal error or receipt of an NDMP_MOVER_ABORT request. A mover data connection is used to transfer backup stream data between the server initiating the connection and the tape subsystem associated with the mover. The data connection can be established locally within a given system or between remote networked systems. The direction of the data transfer is specified by the listen mode argument as either reading from or writing to the data connection. The type of connection is specified by the addr_type argument. A connection within a system can be either null (NDMP_ADDR_LOCAL) or inter process (NDMP_ADDR_IPC), while a connection between systems can be established via TCP/IP (NDMP_ADDR_TCP). Note: It is permissible to establish a connection between two movers for tape to tape transfers. Message XDR definition /* NDMP_MOVER_LISTEN */ struct ndmp_mover_listen_request { ndmp_mode mode; ndmp_addr_type addr_type; }; struct ndmp_mover_listen_reply { ndmp_error error; ndmp_addr connect_addr; }; Request Arguments mode Specifies the direction of the data transfer as follows: Expires June 2001 [Page 121] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_MOVER_MODE_READ The mover reads the backup stream from the data connection and writes the data to tape. This mode is used for backup operations. NDMP_MOVER_MODE_WRITE The mover reads the backup stream from tape and writes the data to the data connection. This mode is used for recover operations. addr_type NDMP_ADDR_IPC The mover listens for a connection from the data server that is collocated with the mover. The mover and the data server are controlled by separate DMA connections. The communication mechanism is implementation dependent. NDMP_ADDR_LOCAL The mover listens for a data connection from a collocated data server. The mover and the data server are controlled by a single DMA connection. The communication mechanism is implementation dependent. NDMP_ADDR_TCP The mover listens for a TCP connection from a remote data or tape server on a specific IP address and TCP port. Reply Arguments connect_addr Specifies the endpoint address that the mover is listening at for a connection. The ndmp_addr structure conveys both the address type (NDMP_ADDR_IPC, NDMP_ADDR_LOCAL, or NDMP_ADDR_TCP) as well as the address information appropriate for the specified type. If the address type is NDMP_ADDR_TCP, then the reply connect address contains the IP address and TCP port number that the mover is listening on. error Error code. Reply Errors NDMP_NO_ERR The mover listen request was successfully processed. The mover has transitioned to the LISTEN state and the connect address information contained in this reply message is valid. NDMP_ILLEGAL_STATE_ERR The mover listen request was received while the mover state machine was in a state that prevented processing this request. Listen requests are only valid in the IDLE state. Expires June 2001 [Page 122] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_ILLEGAL_ARGS_ERR The mover listen request specified an invalid mode or address type. NDMP_NOT_AUTHORIZED_ERR The NDMP server requires DMA authentication, but has not received a valid NDMP_CONNECT_CLIENT_AUTH request from the DMA. NDMP_DEV_NOT_OPEN_ERR No tape device currently open. Expires June 2001 [Page 123] Draft Specification NDMP Version 4 Protocol November 2000 3.6.3.5. NDMP_MOVER_READ This request is used by the DMA to instruct the mover to begin transferring the specified backup stream segment from the tape subsystem to the data connection. The mover MUST be in the ACTIVE state to accept and process a mover read request. Multiple outstanding read requests are not allowed. The location and size of the stream segment to be transferred is specified by the read request offset and length arguments where the offset is an absolute byte offset from the start of the backup stream and the length is the length in bytes of the stream segment. The mover read offset plus read length MUST not result in an overflow condition. A mover read length of all ones is valid in conjunction with a mover read offset of zero. This form of the request allows the read to proceed until interrupted by detection of an EOF or EOW condition resulting in a transition to the PAUSED state. The mover is responsible for positioning to the specified read offset within the current mover window. If the read offset is not accessible within the current mover window, the mover notifies the DMA that a tape change or seek is required by issuing a NDMP_NOTIFY_MOVER_PAUSED message then enters the PAUSED state. The mover reads the data stream from the tape subsystem and transfers the stream to the data connection. The mover read operation continues until the specified stream segment length has been completely transferred. If an EOF or EOW condition is encountered, the mover notifies the DMA that a tape change or seek is required via the NDMP_NOTIFY_MOVER_PAUSED message and then enters the PAUSED state. While processing a mover read request, the tape server MUST continue to accept additional messages from the DMA. Expires June 2001 [Page 124] Draft Specification NDMP Version 4 Protocol November 2000 Message XDR definition /* NDMP_MOVER_READ */ struct ndmp_mover_read_request { ndmp_u_quad offset; ndmp_u_quad length; }; struct ndmp_mover_read_reply { ndmp_error error; }; Request Arguments offset The location of the first byte of data to be sent to the data connection specified as an absolute byte offset from the start of the backup stream. length Number of data bytes to be read and sent to the data connection. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR The mover read request was successfully processed. The specified mover read is in progress. NDMP_ILLEGAL_ARGS_ERR The mover read request specified an invalid data segment. A mover read length of zero is invalid. Additionally, the read offset plus the read length MUST NOT result in an overflow condition. NDMP_ILLEGAL_STATE_ERR The mover read request was received while the mover state machine was in a state that prevented processing this request. Read requests are only valid in the ACTIVE state. NDMP_READ_IN_PROGRESS_ERR (new error) The mover read request was received while a previous mover read operation was in progress. Only one read request may be processed at any time. Expires June 2001 [Page 125] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_NOT_AUTHORIZED_ERR The NDMP server requires DMA authentication, but has not received a valid NDMP_CONNECT_CLIENT_AUTH request from the DMA. Expires June 2001 [Page 126] Draft Specification NDMP Version 4 Protocol November 2000 3.6.3.6. NDMP_MOVER_GET_STATE This request is used by the DMA to obtain information about the mover's operational state as represented by the standard mover variable set. Refer to section 3.6.3 for a complete definition of the standard mover variables and associated enumerations. Message XDR definition /* NDMP_MOVER_GET_STATE */ /* no request arguments */ struct ndmp_mover_get_state_reply { ndmp_error error; ndmp_mover_state state; ndmp_mover_pause_reason pause_reason; ndmp_mover_halt_reason halt_reason; u_long record_size; u_long record_num; ndmp_u_quad data_written; ndmp_u_quad seek_position; ndmp_u_quad bytes_left_to_read; ndmp_u_quad window_offset; ndmp_u_quad window_length; ndmp_addr data_connection_addr; }; Request Arguments The mover get state request does not have a message body or message arguments. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR The mover get state request was successfully processed. The mover get state reply message body accurately represents the mover's current operational state. NDMP_NOT_AUTHORIZED_ERR The NDMP server requires DMA authentication, but has not received a valid NDMPCONNECT_CLIENT_AUTH request from the DMA. Expires June 2001 [Page 127] Draft Specification NDMP Version 4 Protocol November 2000 3.6.3.7. NDMP_MOVER_CONTINUE This request is used by the DMA to instruct the mover to transition from the PAUSED state to the ACTIVE state and to resume the transfer of data stream between the data connection and the tape subsystem. This request is typically issued after the DMA has completed a tape change or tape positioning operation in response to a NDMP_NOTIFY_MOVER_PAUSED message. A mover continue request can only be issued when the mover is in the PAUSED state. The DMA MUST issue a new NDMP_MOVER_SET_WINDOW request to establish the new absolute offset within the data stream prior to issuing the mover continue request. Message XDR definition /* NDMP_MOVER_CONTINUE */ /* no request arguments */ struct ndmp_mover_continue_reply { ndmp_error error; }; Request Arguments The mover continue request does not have a message body or message arguments. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR The mover continue request was successfully processed. The mover has transitioned to the ACTIVE state and resumed the transfer of backup stream data. NDMP_ILLEGAL_STATE_ERR The mover continue request was received while the mover was in a state that prevented processing this request. Continue requests are only valid when the mover is in the PAUSED state. NDMP_NOT_AUTHORIZED_ERR The NDMP server requires DMA authentication, but has not received a valid NDMP_CONNECT_CLIENT_AUTH request from the DMA. Expires June 2001 [Page 128] Draft Specification NDMP Version 4 Protocol November 2000 3.6.3.8. NDMP_MOVER_CLOSE This request is used by the DMA to instruct the mover to gracefully close the current data connection and transition to the HALTED state. This request is typically issued after the mover has successfully transferred all backup stream data to the tape subsystem, or after all specified recovery data has been received by a remote data or tape server. A mover close request can only be issued when the mover is in the PAUSED state. Message XDR definition /* NDMP_MOVER_CLOSE */ /* no request arguments */ struct ndmp_mover_close_reply { ndmp_error error; }; Request Arguments The mover close request does not have a message body or message arguments. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR The mover close request was successfully processed. The mover data connection has been closed and the mover has transitioned to the HALTED state. NDMP_ILLEGAL_STATE_ERR The mover close request was received while the mover state machine was in a state that prevented processing this request. Close requests are only valid in the PAUSED state. NDMP_NOT_AUTHORIZED_ERR The NDMP server requires DMA authentication, but has not received a valid NDMPCONNECT_CLIENT_AUTH request from the DMA. Expires June 2001 [Page 129] Draft Specification NDMP Version 4 Protocol November 2000 3.6.3.9. NDMP_MOVER_ABORT This request is used by the DMA to instruct the mover to terminate any in progress mover operation, close the data connection if present, and transition the mover to the to the HALTED state. An abort request can be issued from any mover state except IDLE. Message XDR definition /* NDMP_MOVER_ABORT */ /* no request arguments */ struct ndmp_mover_abort_reply { ndmp_error error; }; Request Arguments The mover abort request does not have a message body or message arguments. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR The mover abort request was successfully processed. All mover operations have been terminated, the data connection closed, and the mover has transitioned to the HALTED state. NDMP_ILLEGAL_STATE_ERR The mover close request was received while the mover state machine was in a state that prevented processing this request. Abort requests are not valid in the IDLE state. NDMP_NOT_AUTHORIZED_ERR The NDMP server requires DMA authentication, but has not received a valid NDMP_CONNECT_CLIENT_AUTH request from the DMA. Expires June 2001 [Page 130] Draft Specification NDMP Version 4 Protocol November 2000 3.6.3.10. NDMP_MOVER_STOP This request is used by the DMA to instruct the mover to release all resources, reset all mover state variables, and transition the mover to the IDLE state. Message XDR definition /* NDMP_MOVER_STOP */ /* no request arguments */ struct ndmp_mover_stop_reply { ndmp_error error; }; Request Arguments The mover close request does not have a message body or message arguments. Reply Arguments error Error code. Reply Errors NDMP_NO_ERR The mover stop request was successfully processed. All mover resources have been released, mover state variables reset, and the mover has transitioned to the IDLE state. NDMP_ILLEGAL_STATE_ERR The mover stop request was received while the mover state machine was in a state that prevented processing this request. Stop requests are only valid in the HALTED state. NDMP_NOT_AUTHORIZED_ERR The NDMP server requires DMA authentication, but has not received a valid NDMP_CONNECT_CLIENT_AUTH request from the DMA. Expires June 2001 [Page 131] Draft Specification NDMP Version 4 Protocol November 2000 4. DMA Interfaces This section defines the protocol interfaces implemented by the DMA. Note that none of the messages in this section (LOG, NOTIFY, or FILE_HISTORY) have responses associated with them. As a result, the initial calls are suffixed with ææpostÆÆ instead of æærequest.ÆÆ 4.1. Notify Interface This interface is used by the NDMP server to indicate to the DMA that a new state has been entered and/or some direct action is required. Upon receiving the message the DMA MUST move to the next phase in the backup/recovery procedure, carry out the requested action, or take appropriate error reporting/recovery action. Implementation guideline: It is the recommended client behavior that all of this information SHOULD be placed in a file for purposes of logging. If there is textual information to be communicated to the user, an NDMP_LOG_MESSAGE MAY be sent following the NOTIFY message. Expires June 2001 [Page 132] Draft Specification NDMP Version 4 Protocol November 2000 4.1.1. NDMP_NOTIFY_DATA_HALTED This message is used to notify the DMA that the NDMP data server has halted. Post Message struct ndmp_notify_data_halted_post { ndmp_data_halt_reason reason; }; Post Message Arguments reason Reason the data operation halted. NDMP_DATA_HALT_SUCCESSFUL Data operation completed successfully. NDMP_DATA_HALT_ABORTED Data operation aborted by the DMA. NDMP_DATA_HALT_INTERNAL_ERROR Data operation halted due to unrecoverable error incurred by the NDMP server or the data backup/recover method. Expires June 2001 [Page 133] Draft Specification NDMP Version 4 Protocol November 2000 4.1.2. NDMP_NOTIFY_CONNECTION_STATUS This message is sent in response to a connection establishment attempt. This message is always the first message sent on a new connection. It is also used prior to NDMP server shutdown to inform the client that the server is shutting down. For reasons of backward compatibility, it is guaranteed that the parameters of this message will not change in any future release. The parameters MUST not change since this message is sent prior to protocol version negotiation. Post Message enum ndmp_connection_status_reason { NDMP_CONNECTED, NDMP_SHUTDOWN, NDMP_REFUSED }; struct ndmp_notify_connection_status_post { ndmp_connection_status_reason reason; u_short protocol_version; string text_reason<>; }; Post Message Arguments reason Reason code describing the current connection state. NDMP_CONNECTED NDMP connection successfully established. This code will only be returned in a message sent immediately after successful connection establishment. NDMP_SHUTDOWN This reason will only be used after an NDMP connection has been established and a NOTIFY has previously been sent with an NDMP_CONNECTED reason. It is sent when shutting down the NDMP host to gracefully close down the NDMP connection. NDMP_REFUSED NDMP connection refused by the NDMP server. This code will only be returned in a message sent immediately after a connection establishment attempt to notify the DMA that the NDMP server is not able to accept the connection at the current time. This will typically be used if the NDMP server implementation limits the total number of concurrent NDMP connections, when NDMP services on the NDMP host are disabled, or when the NDMP host is in the process of shutting down. Expires June 2001 [Page 134] Draft Specification NDMP Version 4 Protocol November 2000 protocol_version Highest version of protocol supported by the NDMP server implementation. This argument is only valid when the reason argument is NDMP_CONNECTED. If the DMA does not support the protocol version specified by protocol_version then the DMA MUST negotiate an acceptable version using the NDMP_CONNECT_OPEN message. text_reason If this message has a reason of NDMP_REFUSED, this NDMP server implementation dependent message SHOULD be a string indicating why the connection was refused. In all other cases, the text_reason MUST be a zero length string. Expires June 2001 [Page 135] Draft Specification NDMP Version 4 Protocol November 2000 4.1.3. NDMP_NOTIFY_MOVER_HALTED This message is used to notify the DMA that the NDMP tape server has entered the halted state. Post Message struct ndmp_notify_mover_halted_post { ndmp_mover_halt_reason reason; }; Post Message Arguments reason Reason the tape server halted. NDMP_MOVER_HALT_CONNECTION_CLOSED Close of the data detected. NDMP_MOVER_HALT_ABORTED Operation aborted by the DMA. NDMP_MOVER_HALT_INTERNAL_ERROR Operation halted due to unrecoverable error incurred by the tape server. NDMP_MOVER_HALT_CONNECT_ERROR Error establishing data connection. Expires June 2001 [Page 136] Draft Specification NDMP Version 4 Protocol November 2000 4.1.4. NDMP_NOTIFY_MOVER_PAUSED This message is used to notify the DMA that the NDMP tape server has paused. Post Message struct ndmp_notify_mover_paused_post { ndmp_mover_pause_reason reason; ndmp_u_quad seek_position; }; Post Message Arguments reason Reason the tape server paused. NDMP_MOVER_PAUSE_NA Operation not in progress or not in the pause state. NDMP_MOVER_PAUSE_EOM Operation encountered end of media. DMA attention required. NDMP_MOVER_PAUSE_EOF Operation encountered end of file. DMA attention required. NDMP_MOVER_PAUSE_SEEK Data operation requested data stream offset that is outside of the current mover window. DMA attention required. NDMP_MOVER_PAUSE_MEDIA_ERROR Error while reading/writing tape. NDMP_MOVER_PAUSE_EOW Operation encountered end of mover window. DMA attention required. seek_position If reason is NDMP_MOVER_PAUSE_SEEK, indicates the desired data stream seek position. The DMA MUST load the tape containing the requested seek_position, position the tape appropriately, set a new mover window, and then continue the tape server. Expires June 2001 [Page 137] Draft Specification NDMP Version 4 Protocol November 2000 4.1.5. NDMP_NOTIFY_DATA_READ This message is used to notify the DMA that the NDMP server wants to read data from a remote tape server. The NDMP server MUST send at least one NDMP_NOTIFY_DATA_READ message to the DMA if the tape server is remote. In response to this message, the NMDP client MUST send an NDMP_MOVER_READ message to the remote tape server. Post Message struct ndmp_notify_data_read_post { ndmp_u_quad offset; ndmp_u_quad length; }; Post Message Arguments offset Data stream offset of first byte that should be sent to the data connection. length Number of data bytes the tape server should read from tape and send to the data connection. Expires June 2001 [Page 138] Draft Specification NDMP Version 4 Protocol November 2000 4.2. Log Interface This interface is used by the NDMP server to send informational and diagnostic data to the DMA. This data is used by the client to monitor the progress of the currently running data operation and to diagnose problems. Expires June 2001 [Page 139] Draft Specification NDMP Version 4 Protocol November 2000 4.2.1. NDMP_LOG_MESSAGE This request sends an informational message to the DMA. It MAY be used to send log and diagnostic messages generated by the backup or recover method or give updates on any incremental update. It MAY also be used to send expanded textual diagnostics about any error condition or NOTIFY message. Post Message enum ndmp_log_type { NDMP_LOG_NORMAL = 0, NDMP_LOG_DEBUG = 1, NDMP_LOG_ERROR = 2, NDMP_LOG_WARNING = 3 }; struct ndmp_log_message_post { ndmp_log_type log_type; u_long message_id; string entry<>; boolean associated_message_valid; u_long associated_message_sequence; }; Post Message Arguments log_type One of the following: NDMP_LOG_NORMAL: The message doesnÆt require immediate attention. This kind of message SHOULD be used to report the status of the backup/retrieval process. Some examples of NDMP_LOG_NORMAL log messages follow as an implementation guideline: Msg: Date of this level 0 dump: Fri Aug 11 20:24:13 2000. Msg: creating "/vol/vol0/../snapshot_for_backup.0" snapshot. Msg: Using subtree dump NDMP_LOG_DEBUG: This kind of message SHOULD be used for diagnostic purposes. This feature is primarily intended to be used during software development and when troubleshooting. Some examples of NDMP_LOG_DEBUG log messages follow as an implementation guideline: Expires June 2001 [Page 140] Draft Specification NDMP Version 4 Protocol November 2000 Msg: "Unrecognized environment variable foo ignored." Msg: "Executing Line 77 of File Ndmpmonkey.java." Msg: "Trace entered NdmpGetDonut function in NdmpHsimpson.c." NDMP_LOG_ERROR: This message reports an error condition on the NDMP server. Users SHOULD pay immediate attention to this message. Some examples of NDMP_LOG_ERROR log messages follow as an implementation guideline: Msg: Tape write failed. Msg: Error dumping file. Msg: Cannot dump inode 2848 NDMP_LOG_WARNING: This message reports a warning condition on the NDMP server. Users SHOULD pay attention to this message. Some examples of NDMP_LOG_WARNING log messages follow as an implementation guideline: Msg: Tape rst0a needs to be cleaned. Msg: Raid disk is down. message_id The message_id is NDMP server dependent. NDMP servers MAY use this field to assign a unique identifier to each message that associates the message with information contained in a reference document. entry Text message. associated_message_valid associated_message_valid indicates whether this LOG message is associated with a previous NDMP message. If TRUE, the log message is associated with the NDMP message identified by the server sequence number contained in the associated_message_sequence field. If FALSE, no message association exists and the associated_message_sequence field MUST be disregarded. Expires June 2001 [Page 141] Draft Specification NDMP Version 4 Protocol November 2000 associated_message_sequence associated_message_sequence identifies the sequence number of a previous NDMP message sent by the NDMP server associated with this LOG message. Assignment of a non-zero value to this field is optional and only valid if the associated_message_valid field is TRUE. The association is intended to allow servers to provide additional information for any message based event. When set, the associated_message_sequence field MUST always refer to a server sequence number. Furthermore, of all associated messages, this message needs to be sent last, which is only logical since the association cannot refer to a message that does not yet exist. Expires June 2001 [Page 142] Draft Specification NDMP Version 4 Protocol November 2000 4.2.2. NDMP_LOG_FILE This request sends a file recovered message to the DMA. It is used during recovery to notify the DMA that a file/directory specified in the recovery list sent in the NDMP_DATA_START_RECOVER request has or has not been recovered. This message SHOULD NOT be sent for every recovered or failed file, just files having a name that matches a name in the recovery list. Post Message enum ndmp_recovery_status { NDMP_FILE_RECOVERY_SUCCESSFUL = 0, NDMP_FILE_RECOVERY_FAILED_PERMISSION = 1, NDMP_FILE_RECOVERY_FAILED_NOT_FOUND = 2, NDMP_FILE_RECOVERY_FAILED_NO_DIRECTORY = 3, NDMP_FILE_RECOVERY_FAILED_OUT_OF_MEMORY = 4, NDMP_FILE_RECOVERY_FAILED_IO_ERROR = 5, NDMP_FILE_RECOVERY_FAILED_UNDEFINED_ERROR = 6 }; struct ndmp_log_file_post { string name<>; ndmp_recovery_status recovery_status; }; Post Message Arguments name File name. recovery_status One of the following: NDMP_FILE_RECOVERY_SUCCESSFUL File successfully recovered. NDMP_FILE_RECOVERY_FAILED_PERMISSION File recovery failed due to a permission problem. NDMP_FILE_RECOVERY_FAILED_NOT_FOUND File not found during restore. NDMP_FILE_RECOVERY_FAILED_NO_DIRECTORY Directory not found. NDMP_RECOVERY_FAILED_OUT_OF_MEMORY Memory allocation failed during restore. Expires June 2001 [Page 143] Draft Specification NDMP Version 4 Protocol November 2000 NDMP_FILE_RECOVERY_FAILED_IO_ERROR IO error encountered during restore. NDMP_FILE_RECOVERY_FAILED_UNDEFINED_ERROR Error encountered during restore other than one of those listed above. Expires June 2001 [Page 144] Draft Specification NDMP Version 4 Protocol November 2000 4.3. File History Interface The NDMP server uses this interface to send file history entries to the DMA. The file history entries provide a file by file record of every file backed up by the backup method. The file history data is defined using a UNIX file system or an NT file system compatible format. The backup method can generate UNIX, NT, or both UNIX and NT file system compatible format file history for each file. There are two sets of messages for sending file history data. The first set consists of the NDMP_FH_ADD_FILE message. This set is for use by filename based backup methods (such as tar and cpio) for which the full pathname and file attributes are available at the time each file is backed up. The second set consists of the NDMP_FH_ADD_DIR and NDMP_FH_ADD_NODE messages. This set is for use by inode based backup methods (such as UNIX dump) for which the full pathname is not necessarily available at the time the fileÆs data is backed up. It is NOT REQUIRED that the backup method support the sending of file history data. However, the NDMP server MUST accurately report the type of file history supported (if any) for each backup type in the NDMP_CONFIG_GET_BUTYPE_INFO. Expires June 2001 [Page 145] Draft Specification NDMP Version 4 Protocol November 2000 4.3.1. NDMP_FH_ADD_FILE This request adds a list of file paths with the corresponding attribute entries to the file history. The file path can be either the absolute pathname or the pathname relative to the backup root directory. An absolute pathname is denoted by the presence of a leading path separation character ('/' for UNIX; '\' for NT and DOS). During an incremental backup or a selective file backup a backup method is NOT REQUIRED to send add file entries for intermediate directories leading to files being backed up. A DMA SHOULD only expect to receive add file entries for those files actually backed up to the backup data stream. Example: for an incremental backup of /a where only the file /a/b/c/d was modified, the backup method need only send one add file entry for the /a/b/c/d file. Entries for /a, /a/b, and /a/b/c need not be sent. Expires June 2001 [Page 146] Draft Specification NDMP Version 4 Protocol November 2000 Post Message /* NDMP_FH_ADD_FILE */ enum ndmp_fs_type { NDMP_FS_UNIX, NDMP_FS_NT, NDMP_FS_OTHER }; typedef string ndmp_path<>; struct ndmp_nt_path { ndmp_path nt_path; ndmp_path dos_path; }; union ndmp_file_name switch (ndmp_fs_type fs_type) { case NDMP_FS_UNIX: ndmp_path unix_name; case NDMP_FS_NT: ndmp_nt_path nt_name; default: ndmp_path other_name; }; /* file type */ enum ndmp_file_type { NDMP_FILE_DIR, NDMP_FILE_FIFO, NDMP_FILE_CSPEC, NDMP_FILE_BSPEC, NDMP_FILE_REG, NDMP_FILE_SLINK, NDMP_FILE_SOCK, NDMP_FILE_REGISTRY, NDMP_FILE_OTHER }; /* file stat */ /* unsupported bitmask */ const NDMP_FILE_STAT_ATIME_UNS = 0x00000001; const NDMP_FILE_STAT_CTIME_UNS = 0x00000002; const NDMP_FILE_STAT_GROUP_UNS = 0x00000004; struct ndmp_file_stat { u_long unsupported; ndmp_fs_type fs_type; Expires June 2001 [Page 147] Draft Specification NDMP Version 4 Protocol November 2000 ndmp_file_type ftype; u_long mtime; u_long atime; u_long ctime; u_long owner; u_long group; u_long fattr; ndmp_u_quad size; u_long links; }; struct ndmp_file { ndmp_file_name name<>; ndmp_file_stat stat<>; ndmp_u_quad node; ndmp_u_quad fh_info; }; struct ndmp_fh_add_file_post { ndmp_file files<>; }; /* no reply */ Post Message Arguments files Array of file history entries. Each entry contains: name Array of the file names for a single file. Each file can have one or more file names. Multiple names are typically used by multi-protocol file servers to provide both the UNIX and NT file name for a file being backed up. The name union contains the following: unix_name UNIX path name of backed up file. MAY be either the absolute path name or the path name relative to the backup root directory. The first character of an absolute path name MUST be a '/'. The first character of a relative path name MUST NOT be a '/'. The path separation character is '/'. Expires June 2001 [Page 148] Draft Specification NDMP Version 4 Protocol November 2000 nt_name NT path name of backed up file. MAY be either the absolute path name or the path name relative to the backup root directory. The first character of an absolute path name MUST be a '\'. The first character of a relative path name MUST NOT be a '\'. The path separation character is '\'. other_name Path name of backed up file. MAY be either the absolute path name or the path name relative to the backup root directory. The first character of an absolute path name MUST be a path separation character. The first character of a relative path name MUST NOT be a path separation character. The path separation character is file system dependent. This field SHOULD be used when the file system is of a type other than UNIX or NT. stat Array of the file attributes for a single file. The following attributes are defined in the ndmp_file_stat structure: unsupported Identifies unsupported message arguments. ftype File type. mtime Time the file was last modified (in seconds since 00:00:00 GMT, Jan 1, 1970). atime Time the file was last accessed (in seconds since 00:00:00 GMT, Jan 1, 1970). The NDMP_FILE_STAT_ATIME_UNS bit MUST be set in the unsupported bitmask field if this feature is not supported. ctime Time the file status was last modified (in seconds since 00:00:00 GMT, Jan 1, 1970). Indicates the last time that either the file data or the file attributes were modified. The NDMP_FILE_STAT_CTIME_UNS bit MUST be set in the unsupported bitmask field if this feature is not supported. owner File owner identifier. uid SHOULD be used for UNIX file system type. This field is undefined for NT file system type. Expires June 2001 [Page 149] Draft Specification NDMP Version 4 Protocol November 2000 group File group identifier. gid SHOULD be used for UNIX file system type. This field is undefined for NT file system type. fattr System file attribute. UNIX file mode SHOULD be used for UNIX file system type. On UNIX the file mode and type are typically encoded together. fattr MUST be set to just the mode bits, excluding the type bits. NT fattr SHOULD be used for NT file system type. size File size in bytes. links File link count. node inode number. This field is only used for UNIX file system only. A value of 0 MUST be used if this field is not supported. fh_info File history positioning data representing the data stream position of the file. This data MAY be used by the restore method to perform direct access file retrieval. The positioning data is NDMP server dependent. Typically it is the byte offset within the data stream of the start of the file data. This field MUST be set to 0 if the server implementation does not support direct access file retrieval. Expires June 2001 [Page 150] Draft Specification NDMP Version 4 Protocol November 2000 4.3.2. NDMP_FH_ADD_DIR This message is used to report name and inode information for backed up files. The following rules apply to the generation of add directory entries: 1. Add directory entries MUST be ordered to represent a top down, breadth first scan of the directory tree being backed up. 2. For each directory, a "." add directory entry and a ".." add directory entry MUST immediately precede all add directory entries for files contained in the directory. A "." add directory entry MUST contain "." as one of the names in the name array. The node and parent values MUST be equivalent. A ".." add directory entry MUST contain ".." as one of the names in the name array. The parent value MUST match the node value from the "." entry and the node value MUST match the node value of the directory's parent directory. The only exception to this rule is for the backup root directory in which case the node value MUST be equivalent to the parent value. A DMA MUST NOT make any assumptions with regard to the value of the node for the backup root directory. 3. The first add directory entry MUST be the "." entry for the backup root directory. The second ADD_DIR entry MUST be the ".." entry for the backup root directory. 4. All add directory entries for files/directories contained in a directory MUST immediately follow the ".." add directory entry for the directory. 5. For any given node, the add directory entry for the node MUST be sent prior to the add node entry for the node. In the event that multiple hard links exist for a node, one add directory entry MUST be sent for each link but only one add node entry SHOULD be sent for the node. The first add directory entry for a node with multiple hard links MUST be sent prior to the add node entry. However, the server is NOT REQUIRED to send the remaining add directory entries before sending the add node entry. 6. For an incremental backup, add directory entries MAY be sent for which no associated add node entry is sent. The server MUST send add directory entries for the entire contents of every directory leading to each backed up file. Add node entries MUST only be sent for each backed up file and the directories leading to each backed up file. 7. The node number in each add directory entry MUST be unique amongst all entries sent during the course of a backup. The only exception to this rule is for entries for files having multiple hard links. All entries for links to the same file MUST have the same node number. Expires June 2001 [Page 151] Draft Specification NDMP Version 4 Protocol November 2000 Post Message /* NDMP_FH_ADD_DIR */ struct ndmp_dir { ndmp_file_name name<>; ndmp_u_quad node; ndmp_u_quad parent; }; struct ndmp_fh_add_dir_post { ndmp_dir dirs<>; }; Post Message Arguments dirs Array of directory entries. Each entry contains: name Array of file names for a single node. The name is not the full pathname; just the base name relative to the node's parent directory. node Node identifier that matches the node in a corresponding add node message. NDMP server implementation dependent but will typically be the inode number of the file. parent Node identifier of the node's parent directory. NDMP server implementation dependent but will typically be the inode number of the file's parent directory. Expires June 2001 [Page 152] Draft Specification NDMP Version 4 Protocol November 2000 4.3.3. NDMP_FH_ADD_NODE This request adds a list of file attribute entries to the file history. Each entry MUST match a corresponding node number from a previously sent add directory entry. Post Message /* NDMP_FH_ADD_NODE */ struct ndmp_node { ndmp_file_stat stats<>; ndmp_u_quad node; ndmp_u_quad fh_info; }; struct ndmp_fh_add_node_post { ndmp_node nodes<>; }; Post Message Arguments nodes Array of file history node entries. Each entry contains: stats Array of file attribute data for a single file. node Node identifier that matches a node in a corresponding add directory entry. NDMP server implementation dependent but typically is the inode number of the file. fh_info File history positioning data representing the data stream position of the file. This data MAY be used by the restore method to perform direct access file retrieval. The positioning data is NDMP server dependent. Typically it is the byte offset within the data stream of the start of the file data. This field MUST be set to all ones (binary) if the server implementation does not support direct access file retrieval. 5. References [1] RFC 1832, "XDR: External Data Representation Standard", R. Srinivasan, Sun Microsystems, August 1995 [2] RFC 1321, "The MD5 Message-Digest Algorithm", R. Rivest, MIT Laboratory for Computer Science and RSA Data Security, Inc., April 1992 Expires June 2001 [Page 153] Draft Specification NDMP Version 4 Protocol November 2000 [3] RFC 2044, "UTF-8 a transformation format of Unicode and ISO 10646", F. Yergeau, Alis Technologies, October 1996 [4] RFC 1831, "Remote Procedure Call Protocol Version 2", Srinivasan, R., Sun Microsystems, Inc., August 1995. [5] "NDMP Version 2", Dave Hitz, Network Appliance, Inc., and Roger Stager, Legato Software, Inc., September 1997. [6] "NDMP Version 3", Dave Hitz, Network Appliance, Inc., and Roger Stager, Legato Software, Inc., April, 1998. [7] "NDMP workflow document", www.ndmp.org 6. Security NDMP through firewalls is problematic if the data and tape services reside in the interior of separate firewalls such that an NDMP data connection must originate from the exterior of one firewall. If only a single firewall exists, the NDMP server inside the firewall SHOULD originate the connection as firewalls generally allow any outbound connection. NDMP server implementations SHOULD resolve the two firewall problem by providing configurable control over the port number range that will be used for NDMP data connection listens. This control SHOULD be used by system administrators to constrain NDMP servers to a limited range of TCP ports that correspond to ports the firewall will allow inbound connections on. NDMP is incompatible with Network Address Translation (NAT) firewalls because IP address and TCP port information is conveyed as payload data between NDMP peers (connect_addr in NDMP_MOVER_LISTEN & NDMP_DATA_LISTEN replies). The NDMP client normally is authenticated by the NDMP server using a secure MD5 digest. However, the NDMP server optionally can authenticate using a clear text password or even permit access without authentication. Once authenticated, privileges are not specified by the NDMP protocol, but it is expected that NDMP server implementations will permit data to be transferred to and from tape using the protocol. The NDMP SCSI interface provides low-level access to SCSI media changer devices. The NDMP server SHOULD prevent access to other SCSI devices (such as disk drives) to prevent the NDMP client from bypassing file system security. File history information is transferred to the NDMP client through a TCP/IP connection. Expires June 2001 [Page 154] Draft Specification NDMP Version 4 Protocol November 2000 7. Recognition of Prior Work This work is based on NDMP version 1, 2 [5] and 3 [6], as developed by Dave Hitz, Network Appliance Inc., and Roger Stager, Legato Software Inc. These documents can be retrieved from www.ndmp.org. Version 4 of the NDMP specification is largely a cleanup effort from version 2 and version 3 of the NDMP specification. Very few aspects have changed between the two versions, and the architecture of NDMP from versions 2 and 3 of the protocol is preserved. 8. Authors and Contributors 8.1. Document Editor Harald Skardal Network Appliance, Inc. 495 East Java Drive Sunnyvale, CA 94089, USA Tel: 603.882.3881 Email: hskardal@netapp.com 8.2. Authors' Addresses [Companies, email addresses, phone numbers etc will be added.] James Bunnell Spectra Logic Inc. Email: jamesb@spectralogic.com Tim Gardner Chewcoba Inc. Email: tim@chewcoba.com Clive Hendrie Synaxia Networks Email: clive.hendrie@synaxia.com Greg Linn Network Appliance Inc. Email: Greg.Linn@netapp.com Dave Manley Network Appliance Email: David.Manley@netapp.com Jim Ward Workstation Solutions Inc. Email: jimw@worksta.com Expires June 2001 [Page 155] Draft Specification NDMP Version 4 Protocol November 2000 8.3. Contributors In addition to the authors, the following people have helped the effort by participating in the reviews: Steve Kappel Veritas Software Email: steve.kappel@veritas.com Gordon Waidhofer Traakan Inc. Email: gww@traakan.com Expires: June 2001 Expires June 2001 [Page 156] Draft Specification NDMP Version 4 Protocol November 2000 Appendixes: Appendix A: MD5 Based Authentication: This appendix describes the typical process of authenticating the DMA to an NDMP server host. The requirement for authentication is a shared secret between the DMA and the NDMP server host where the data or tape servers are running. The typical implementation is to use a user account on the host such as æændmpÆÆ, with a password that is the shared secret. This user account (æændmp_userÆÆ) has the right privileges to run the necessary backup programs such as tar, cpio, dump etc., or to use the tape devices. The DMA uses the command NDMP_CONFIG_GET_AUTH_ATTR to get the 64 byte challenge from the remote host. The client combines the challenge, the ændmp_userÆ password and null byte padding as follows: The input string to the MD5 algorithm is 128 bytes, or 1024 bits. Notice that this string is the ææmessageÆÆ part in the MD5 input string, as defined by RFC 1321. The MD5 routine will add its own padding and the length of the input message before the digest is computed. The challenge part is 64 bytes. This is the string returned from the NDMP server host using the NDMP_CONFIG_GET_AUTH_ATTR request. The password is up to 32 bytes. Notice that it is the same password that is repeated. If the password is longer than 32 bytes, it is truncated to 32 bytes. The padding is a sequence of null bytes (0x00) that pads the remainder of the MD5 input string such that it becomes 128 bytes. This resulting string is the input to the MD5 digest algorithm, as specified in RFC 1321. The MD5 algorithm takes this input and produces the digest, a 128 bit (=16 byte) quantity. The digest goes into the NDMP_CONNECT_CLIENT_AUTH command as the ææauth_digestÆÆ value, together with the ndmp_user id. The host receives the ndmp_user ID and the MD5 digest. The host computes its own digest based on the stored password for the ndmp_user, the challenge it sent to the client, and padding, according to the NDMP convention for generating an MD5 message. The resulting digest is compared to the digest received from the client. If the computed and the received digests are equal, the client is authenticated. Expires June 2001 [Page 157]