LDAP Data Interchange Format (LDIF) Gordon Good INTERNET-DRAFT Netscape Communications Status: Standards-Track 22 June 1999 The LDAP Data Interchange Format (LDIF) - Technical Specification Filename: draft-good-ldap-ldif-04.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet Draft expires December 22, 1999. Abstract This document describes a file format suitable for describing directory information or modifications made to directory information. The file format, known as LDIF, for LDAP Data Interchange Format, is typically used to import and export directory information between LDAP-based directory servers, or to describe a set of changes which are to be applied to a directory. Background and Intended Usage There are a number of situations where a common interchange format is desirable. For example, one might wish to export a copy of the contents of a directory server to a file, move that file to a different machine, and import the contents into a second directory server. Additionally, by using a well-defined interchange format, development Good June 22, 1999 [Page 1] INTERNET-DRAFT LDAP Data Interchange Format 22 June 1999 of data import tools from legacy systems is facilitated. A fairly simple set of tools written in awk or perl can, for example, convert a database of personnel information into an LDIF file. This file can then be imported into a directory server, regardless of the internal database representation the target directory server uses. The LDIF format was originally developed and used in the University of Michigan LDAP implementation. The first use of LDIF was in describing directory entries. Later, the format was expanded to allow representation of changes to directory entries. Relationship to the application/directory MIME content-type: The application/directory MIME content-type [1] is a general framework and format for conveying directory information, and is independent of any particular directory service. The LDIF format is a simpler format which is perhaps easier to create, and may also be used, as noted, to describe a set of changes to be applied to a directory. The key words "MUST", "MAY", and "SHOULD" used in this document are to be interpreted as described in [7]. Definition of the LDAP Data Interchange Format The LDIF format is used to convey directory information, or a description of a set of changes made to directory entries. An LDIF file consists of a series of records separated by line separators. A record consists of a sequence of lines describing a directory entry, or a sequence of lines describing a set of changes to a directory entry. An LDIF file specifies a set of directory entries, or a set of changes to be applied to directory entries, but not both. There is a one-to-one correlation between LDAP operations that modify the directory (add, delete, modify, and modrdn), and the types of changerecords described below ("add", "delete", "modify", and "modrdn" or "moddn"). This correspondence is intentional, and permits a straightforward translation from LDIF changerecords to protocol operations. Formal Syntax Definition of LDIF The following definition uses the augmented Backus-Naur Form specified in RFC 822 [2]. ldif-file = ldif-content / ldif-changes Good June 22, 1999 [Page 2] INTERNET-DRAFT LDAP Data Interchange Format 22 June 1999 ldif-content = version-spec 1*(1*SEP ldif-attrval-record) ldif-changes = version-spec 1*(1*SEP ldif-change-record) ldif-attrval-record = dn-spec SEP 1*attrval-spec ldif-change-record = dn-spec SEP *control changerecord version-spec = "version:" FILL version-number version-number = 1*DIGIT ; version-number MUST be "1" for the ; LDIF format described in this document. dn-spec = "dn:" (FILL dn / ":" FILL base64-dn) dn = base64-dn = rdn = base64-rdn = control = "control: FILL ldap-oid ; controlType 0*1 (FILL ("true" / "false")) ; criticality 0*1(value-spec) ; controlValue SEP ; (See note 10, below) ldap-oid = attrval-spec = attribute-description value-spec value-spec = ":" ( FILL 0*1(value) / ":" FILL base64-value / "<" FILL url) SEP url = ; (See Note 6, below) attribute-description = value = safe-initval *safe ; (See Note 9, below) Good June 22, 1999 [Page 3] INTERNET-DRAFT LDAP Data Interchange Format 22 June 1999 safe = safe-initval = base64-value = changerecord = "changetype:" FILL (change-add / change-delete / change-modify / change-moddn) change-add = "add" SEP 1*attrval-spec change-delete = "delete" SEP change-moddn = ("modrdn" / "moddn") SEP "newrdn:" ( FILL rdn / ":" FILL base-64-rdn) SEP "deleteoldrdn:" FILL ("0" / "1") SEP 0*1("newsuperior:" FILL (dn / ":" FILL base-64-dn) SEP) change-modify = "modify" SEP *mod-spec mod-spec = (mod-add-spec / mod-delete-spec / mod-replace-spec) "-" SEP mod-add-spec = "add:" FILL attribute-description SEP *attrval-spec mod-delete-spec = "delete:" FILL attribute-description SEP *attrval-spec mod-replace-spec = "replace:" FILL attribute-description SEP *attrval-spec SPACE = FILL = 1*SPACE SEP = (CR LF / LF) CR = LF = NUL = Good June 22, 1999 [Page 4] INTERNET-DRAFT LDAP Data Interchange Format 22 June 1999 DIGIT = Notes on LDIF Syntax 1) For the LDIF format described in this document, the version number MUST be "1". If the version number is absent, implementations MAY choose to interpret the contents as an older LDIF file format, supported by the University of Michigan ldap-3.3 implementation [8]. 2) Any non-emptry line, including comment lines, in an LDIF file MAY be folded by inserting a line separator (SEP) and a SPACE. Folding MUST NOT occur before the first character of the line. In other words, folding a line into two lines, the first of which is empty, is not permitted. Any line that begins with a single space MUST be treated as a continuation of the previous (non-empty) line. When joining folded lines, exactly one space character at the beginning of each continued line must be discarded. 3) Any line that begins with a pound-sign ("#", ASCII 35) is a comment line, and MUST be ignored when parsing an LDIF file. 4) Any dn or value that contains characters other than those defined as "safe", or begins with a character other than those defined as "safe-initval", above, MUST be base-64 encoded. Other values MAY be base-64 encoded. 5) When a zero-length attribute value is to be included directly in an LDIF file, it MUST be represented as attribute-description ":" FILL SEP. For example, "seeAlso:" followed by a newline represents a zero-length "seeAlso" attribute value. It is also permissible for the value referred to by a URL to be of zero length. 6) When a URL is specified in an attrval-spec, the following conventions apply: a) Implementations SHOULD support the file:// URL format. The contents of the referenced file are to be included verbatim in the interpreted output of the LDIF file. b) Implementations MAY support other URL formats. The semantics associated with each supported URL will be documented in an associated Applicability Statement. 7) While it is permissible for character values larger than 126 to be contained in an attribute value, implementations SHOULD base-64 encode any value that contains such characters when generating LDIF. However, implementations MAY leave the values unencoded. This relaxation is designed to allow editing of LDIF files containing UTF-8 data. Good June 22, 1999 [Page 5] INTERNET-DRAFT LDAP Data Interchange Format 22 June 1999 8) Attribute values contained in LDIF files represent directory data, and therefore MUST be valid UTF-8 strings. Implementations that read LDIF MAY interpret files in which the values are stored in some other character set encoding, but implementations MUST NOT generate LDIF content which does not contain valid UTF-8 data. 9) Values or distinguished names that end with SPACE SHOULD be base- 64 encoded. 10) When controls are included in an LDIF file, implementations MAY choose to ignore some or all of them. This may be necessary if the changes described in the LDIF file are being sent on an LDAPv2 connection (LDAPv2 does not support controls), or the particular controls are not supported by the remote server. If the criticality of a control is "true", then the implementation MUST either include the control, or MUST NOT send the operation to a remote server. Examples of LDAP Data Interchange Format Example 1: An simple LDAP file with two entries version: 1 dn: cn=Barbara Jensen, ou=Product Development, dc=airius, dc=com objectclass: top objectclass: person objectclass: organizationalPerson cn: Barbara Jensen cn: Barbara J Jensen cn: Babs Jensen sn: Jensen uid: bjensen telephonenumber: +1 408 555 1212 description: A big sailing fan. dn: cn=Bjorn Jensen, ou=Accounting, dc=airius, dc=com objectclass: top objectclass: person objectclass: organizationalPerson cn: Bjorn Jensen sn: Jensen telephonenumber: +1 408 555 1212 Example 2: A file containing an entry with a folded attribute value version: 1 dn:cn=Barbara Jensen, ou=Product Development, dc=airius, dc=com objectclass:top Good June 22, 1999 [Page 6] INTERNET-DRAFT LDAP Data Interchange Format 22 June 1999 objectclass:person objectclass:organizationalPerson cn:Barbara Jensen cn:Barbara J Jensen cn:Babs Jensen sn:Jensen uid:bjensen telephonenumber:+1 408 555 1212 description:Babs is a big sailing fan, and travels extensively in sea rch of perfect sailing conditions. title:Product Manager, Rod and Reel Division Example 3: A file containing a base-64-encoded value version: 1 dn: cn=Gern Jensen, ou=Product Testing, dc=airius, dc=com objectclass: top objectclass: person objectclass: organizationalPerson cn: Gern Jensen cn: Gern O Jensen sn: Jensen uid: gernj telephonenumber: +1 408 555 1212 description:: V2hhdCBhIGNhcmVmdWwgcmVhZGVyIHlvdSBhcmUhICBUaGlzIHZhbHVlIGlzIGJ hc2UtNjQtZW5jb2RlZCBiZWNhdXNlIGl0IGhhcyBhIGNvbnRyb2wgY2hhcmFjdGVyIGluIGl0ICh hIENSKS4NICBCeSB0aGUgd2F5LCB5b3Ugc2hvdWxkIHJlYWxseSBnZXQgb3V0IG1vcmUu Example 4: A file containing an entries with UTF-8-encoded attribute values, including language tags. Comments indicate the contents of UTF-8-encoded attributes and distinguished names. version: 1 dn:: b3U95Za25qWt6YOoLG89QWlyaXVz # dn:: ou=,o=Airius objectclass: top objectclass: organizationalUnit ou:: 5Za25qWt6YOo # ou:: ou;lang-ja:: 5Za25qWt6YOo # ou;lang-ja:: ou;lang-ja;phonetic:: 44GI44GE44GO44KH44GG44G2 # ou;lang-ja:: ou;lang-en: Sales description: Japanese office dn:: dWlkPXJvZ2FzYXdhcmEsb3U95Za25qWt6YOoLG89QWlyaXVz # dn:: uid=,ou=,o=Airius Good June 22, 1999 [Page 7] INTERNET-DRAFT LDAP Data Interchange Format 22 June 1999 userpassword: {SHA}O3HSv1MusyL4kTjP+HKI5uxuNoM= objectclass: top objectclass: person objectclass: organizationalPerson objectclass: inetOrgPerson uid: rogasawara mail: rogasawara@airius.co.jp givenname;lang-ja:: 44Ot44OJ44OL44O8 # givenname;lang-ja:: sn;lang-ja:: 5bCP56yg5Y6f # sn;lang-ja:: cn;lang-ja:: 5bCP56yg5Y6fIOODreODieODi+ODvA== # cn;lang-ja:: title;lang-ja:: 5Za25qWt6YOoIOmDqOmVtw== # title;lang-ja:: preferredlanguage: ja givenname:: 44Ot44OJ44OL44O8 # givenname:: sn:: 5bCP56yg5Y6f # sn:: cn:: 5bCP56yg5Y6fIOODreODieODi+ODvA== # cn:: title:: 5Za25qWt6YOoIOmDqOmVtw== # title:: givenname;lang-ja;phonetic:: 44KN44Gp44Gr44O8 # givenname;lang-ja;phonetic:: sn;lang-ja;phonetic:: 44GK44GM44GV44KP44KJ # sn;lang-ja;phonetic:: cn;lang-ja;phonetic:: 44GK44GM44GV44KP44KJIOOCjeOBqeOBq+ODvA== # cn;lang-ja;phonetic:: title;lang-ja;phonetic:: 44GI44GE44GO44KH44GG44G2IOOBtuOBoeOCh+OBhg== # title;lang-ja;phonetic:: givenname;lang-en: Rodney sn;lang-en: Ogasawara cn;lang-en: Rodney Ogasawara title;lang-en: Sales, Director Example 5: A file containing a reference to an external file version: 1 dn: cn=Horatio Jensen, ou=Product Testing, dc=airius, dc=com objectclass: top objectclass: person objectclass: organizationalPerson cn: Horatio Jensen cn: Horatio N Jensen sn: Jensen Good June 22, 1999 [Page 8] INTERNET-DRAFT LDAP Data Interchange Format 22 June 1999 uid: hjensen telephonenumber: +1 408 555 1212 jpegphoto:< file:///usr/local/directory/photos/hjensen.jpg Example 6: A file containing a series of change records and comments version: 1 # Add a new entry dn: cn=Fiona Jensen, ou=Marketing, dc=airius, dc=com changetype: add objectclass: top objectclass: person objectclass: organizationalPerson cn: Fiona Jensen sn: Jensen uid: fiona telephonenumber: +1 408 555 1212 jpegphoto:< file:///usr/local/directory/photos/fiona.jpg # Delete an existing entry dn: cn=Robert Jensen, ou=Marketing, dc=airius, dc=com changetype: delete # Modify an entry's relative distinguished name dn: cn=Paul Jensen, ou=Product Development, dc=airius, dc=com changetype: modrdn newrdn: cn=Paula Jensen deleteoldrdn: 1 # Rename an entry and move all of its children to a new location in # the directory tree (only implemented by LDAPv3 servers). dn: ou=PD Accountants, ou=Product Development, dc=airius, dc=com changetype: modrdn newrdn: ou=Product Development Accountants deleteoldrdn: 0 newsuperior: ou=Accounting, dc=airius, dc=com # Modify an entry: add an additional value to the postaladdress attribute, # completely delete the description attribute, replace the telephonenumber # attribute with two values, and delete a specific value from the # facsimiletelephonenumber attribute dn: cn=Paula Jensen, ou=Product Development, dc=airius, dc=com changetype: modify add: postaladdress postaladdress: 123 Anystreet $ Sunnyvale, CA $ 94086 - delete: description - Good June 22, 1999 [Page 9] INTERNET-DRAFT LDAP Data Interchange Format 22 June 1999 replace: telephonenumber telephonenumber: +1 408 555 1234 telephonenumber: +1 408 555 5678 - delete: facsimiletelephonenumber facsimiletelephonenumber: +1 408 555 9876 - Example 7: An LDIF file containing a change record with a control version: 1 # Delete an entry. The operation will attach the LDAPv3 # Tree Delete Control defined in [9]. The criticality # field is "true" and the controlValue field is # absent, as required by [9]. dn: ou=Product Development, dc=airius, dc=com control: 1.2.840.113556.1.4.805 true changetype: delete Security Considerations Given typical directory applications, an LDIF file is likely to contain sensitive personal data. Appropriate measures should be taken to protect the privacy of those persons whose data is contained in an LDIF file. Since ":<" directives can cause external content to be included when processing an LDIF file, one should be cautious of accepting LDIF files from external sources. A "trojan" LDIF file could name a file with sensitive contents and cause it to be included in a directory entry, which a hostile entity could read via LDAP. LDIF does not provide any method for carrying authentication information with an LDIF file. Users of LDIF files must take care to verify the integrity of an LDIF file received from an external source. Appendix A: Differences from previous versions of this document This section summarizes the differences between previous revisions of this draft, as an aid to document reviewers. This section will be deleted prior to publication as an RFC. Differences between draft-ietf-asid-ldif-00.txt and draft-ietf-asid- ldif-01.txt 1) The BNF has been modified to explicitly disallow ldif content and change records in the same file. In other words, a given LDIF file Good June 22, 1999 [Page 10] INTERNET-DRAFT LDAP Data Interchange Format 22 June 1999 is either a series of directory entries, or a series of modifications. An LDIF file MUST NOT contain both types of records. 2) External references are now URLs, instead of simple filenames. 3) The BNF has been modified to allow base-64-encoded distinguished names. 4) Multiple separators are now permitted between records. Differences between draft-ietf-asid-ldif-01.txt and draft-ietf-asid- ldif-02.txt 1) The BNF has been modified such that a simple attribute name ("attrname") has been replaced with an "attribute-description" as defined in the LDAPv3 protocol document [4]. This permits language codes and other attribute options to be carried in an LDIF file. 2) A new option, "charset", may be used in attribute descriptions. This facilitates multi-lingual character set conversion. 3) The definition of the "safe" and "safe-initval" productions has been relaxed to allow non-ASCII characters with values greater than 126. This permits more natural expression of character sets such as Latin-1 in LDIF files. Differences between draft-ietf-asid-ldif-02.txt and draft-good-ldap- ldif-00.txt 1) The "charset-option" and "charset-name" productions were removed from the BNF, due to objections within the working group. UTF-8 is the only character set that may be used in LDIF. 2) Examples were reworked to reflect the above change, and to include an example of a non-western language represented in UTF-8. Differences between draft-ietf-good-ldif-00.txt and draft-good-ldap- ldif-01.txt 1) Added version identifiers to the examples - they were missing. 2) Clarified that LDIF files must use UTF-8. Differences between draft-good-ldap-ldif-01.txt and draft-good-ldap- ldif-02.txt 1) Added a recommendation that values ending in SPACE should be base-64 encoded. Good June 22, 1999 [Page 11] INTERNET-DRAFT LDAP Data Interchange Format 22 June 1999 2) Clarified the procedure for joining folded lines. 3) Updated header to reflect new IETF I-D guidelines. Differences between draft-good-ldap-ldif-02.txt and draft-good-ldap- ldif-03.txt 1) Fixed reference from RFC 1779 to RFC 2253. 2) Version string is now required. 3) Comment lines may be folded (this is now explicitly mentioned in note 2). 4) Moved this section (differences between draft versions) to an appendix. 5) Updated examples to use "dc=airius, dc=com" instead of "o=Ace Industry, c=US" 6) Cleaned up references section. Differences between draft-good-ldap-ldif-03.txt and draft-good-ldap- ldif-04.txt 1) The grammar now requires that an LDIF file end with one or more SEP sequences (newlines). This was inadvertantly prohibited in earlier revisions of the grammar. 2) Several minor spelling and typographical errors were fixed. 3) Reworked the grammar to make it more readable. Hallvard Furuseth (University of Oslo) provided the new BNF. 4) Excluded NUL from "safe" production. 5) Changed "0,1*xxx" "0*1xxx" in compliance with RFC822. 6) Fixed a glitch in the grammar that allowed multiple changetypes within a single LDIF change record. The intent is that only one changetype per change record is permitted. 7) Fixed a mistake in example 2 (folded attribute value). 8) The BNF now explicitly requires that zero-length attribute values be encoded as attribute-description ":" FILL SEP. 9) Factored "changetype: FILL" out of the productions for change-add, Good June 22, 1999 [Page 12] INTERNET-DRAFT LDAP Data Interchange Format 22 June 1999 change-delete, change-moddn, and change-modify. 10) RFC 2251 permits an LDAP modify operation with no modifications, and also permits an attribute with no values. Although it's unclear what the purpose of these constructs might be, I altered the BNF to allow these to be described in LDIF. 11) The BNF may now carry LDAP v3 controls in ldif-change-records. The "value-spec" production was factored out to allow it to be used in the definition of a control. 12) Clarified the rules for line-folding to prohibit a line from being folded into two lines, the first of which is empty. This guarantees that the sequence SEP SEP terminates an LDIF record, and allows, for example, "perl -n00" to be used to read an entire LDIF record into the $_ variable. Acknowledgments The LDAP Interchange Format was developed as part of the University of Michigan LDAP reference implementation, and was developed by Tim Howes, Mark Smith, and Gordon Good. It is based in part upon work supported by the National Science Foundation under Grant No. NCR- 9416667. Members of the IETF LDAP Extensions Working group provided many helpful suggestions. In particular, Hallvard B. Furuseth of the University of Oslo made many significant contributions to this document, including a thorough review and rewrite of the BNF. References [1] Howes, T., Smith, M., "A MIME Content-Type for Directory Infor- mation", RFC 2425, September 1998, [2] Crocker, D.H., "Standard for the Format of ARPA Internet Text Messages", RFC 822, August 1982, [3] Wahl, M., Kille, S., Howes, T., "A String Representation of Dis- tinguished Names", RFC 2253, [4] Wahl, M., Howes, T., Kille, S., "Lightweight Directory Access Protocol (v3)", RFC 2251, July, 1997, Good June 22, 1999 [Page 13] INTERNET-DRAFT LDAP Data Interchange Format 22 June 1999 [5] Borenstein, N., Freed, N., "MIME (Multipurpose Internet Mail Extensions) Part One: Mechanisms for Specifying and Describing the Format of Internet Message Bodies", section 5.2, "Base64 Content-Transfer-Encoding", RFC 1521, December 1993, [6] T. Berners-Lee, L. Masinter, M. McCahill, "Uniform Resource Locators (URL)", RFC 1738, December 1994, [7] S. Bradner, "Key Words for use in RFCs to Indicate Requirement Levels", Harvard University, RFC 2119, March 1997, [8] The SLAPD and SLURPD Administrators Guide. University of Michi- gan, April 1996. [9] M. P. Armijo, "Tree Delete Control", Microsoft Corporation, INTERNET-DRAFT November 1998, Author's Address Gordon Good Netscape Communications Corp. 501 E. Middlefield Rd. Mailstop MV068 Mountain View, CA 94043, USA Phone: +1 650 937-3825 EMail: ggood@netscape.com This Internet Draft expires December 22, 1999. Good June 22, 1999 [Page 14]