Internet DRAFT - draft-iab-rfcv3-preptool

draft-iab-rfcv3-preptool







Network Working Group                                         P. Hoffman
Internet-Draft                                                     ICANN
Intended status: Informational                             J. Hildebrand
Expires: August 13, 2016                                           Cisco
                                                       February 10, 2016


                      RFC v3 Prep Tool Description
                      draft-iab-rfcv3-preptool-01

Abstract

   This document describes some aspects of the "prep tool" that is
   expected to be created when the new RFC v3 specification is deployed.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on August 13, 2016.

Copyright Notice

   Copyright (c) 2016 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.





Hoffman & Hildebrand     Expires August 13, 2016                [Page 1]

Internet-Draft        RFC v3 Prep Tool Description         February 2016


Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  v3 Prep Tool Usage Scenarios  . . . . . . . . . . . . . . . .   2
   3.  Internet-Draft Submission . . . . . . . . . . . . . . . . . .   3
   4.  Canonical RFC Preparation . . . . . . . . . . . . . . . . . .   4
   5.  What the v3 Prep Tool Does  . . . . . . . . . . . . . . . . .   4
   6.  Additional Uses for the Prep Tool . . . . . . . . . . . . . .  10
   7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  11
   8.  Security Considerations . . . . . . . . . . . . . . . . . . .  11
   9.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  11
   10. Informative References  . . . . . . . . . . . . . . . . . . .  11
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  11

1.  Introduction

   For the future of the RFC format, the RFC Editor has decided that XML
   (using the XML2RFCv3 vocabulary [I-D.iab-xml2rfc]) is the canonical
   format, in the sense that it is the data that is blessed by the
   process as the actual RFC.  See [RFC6949] for more detail on this.

   Most people will read other formats, such as HTML, PDF, ASCII text,
   or other formats of the future, however.  In order to ensure each of
   these formats is as similar as possible to one another as well as the
   canonical XML, there is a desire that the translation from XML into
   the other formats will be straightforward syntactic translation.  To
   make that happen, a good amount of data will need to be in the XML
   format that is not there today.  That data will be added by a program
   called the "prep tool", which will often run as a part of the xml2rfc
   process.

   This draft specifies the steps that the prep tool will have to take.
   As changes to [I-D.iab-xml2rfc] are made, this document will be
   updated.

   The details (particularly any vocabularies) described in this
   document are expected to change based on experience gained in
   implementing the RFC production center's toolset.  Revised documents
   will be published capturing those changes as the toolset is
   completed.  Other implementers must not expect those changes to
   remain backwards-compatible with the details described in this
   document.

2.  v3 Prep Tool Usage Scenarios

   The prep tool will have several settings:

   o  Internet-Draft preparation



Hoffman & Hildebrand     Expires August 13, 2016                [Page 2]

Internet-Draft        RFC v3 Prep Tool Description         February 2016


   o  Canonical RFC preparation

   There are only a few differences between the two settings.  For
   example, the boilerplate output will be different, as will the date
   output on the front page.

   Note that this only describes what the IETF-sponsored prep tool does.
   Others might create their own work-alike prep tools for their own
   formatting needs.  However, an output format developer does not need
   to change the prep tool in order to create their own formatter: they
   only need to be able to consume prepared text.  The IETF-sponsored
   prep tool runs in two different modes: "I-D" mode when the tool is
   run during Internet-Draft submission and processing, and "RFC
   production mode" when the tool is run by the RFC Production Center
   (RPC) while producing an RFC.

   This tool is described as if it is a separate tool so that we can
   reason about its architectural properties.  In actual implementation,
   it might be a part of a larger suite of functionality.

3.  Internet-Draft Submission

   When the IETF draft submission tool accepts v3 XML as an input
   format, the submission tool runs the submitted file through the prep
   tool.  This is called "I-D mode" in this document.  If the tool finds
   no errors, it keeps two XML files: the submitted file and the prepped
   file.

   The prepped file provides a record of what a submitter was attesting
   to at the time of submission.  It represents a self-contained record
   of what any external references resolved to at the time of
   submission.

   The prepped file is used by the IETF formatters to create outputs
   such as HTML, PDF, and text (or the tools act in a way
   indistinguishable from this).  The message sent out by the draft
   submission tool includes a link to the original XML as well as the
   other outputs, including the prepped XML.

   The prepped XML can be used by tools not yet developed to output new
   formats that have as similar output as possible to the current IETF
   formatters.  For example, if the IETF creates a .mobi output renderer
   later, it can run that renderer on all of the prepped XML that has
   been saved, ensuring that the content of included external references
   and all of the part numbers and boilerplate will be the same as what
   was produced by the previous IETF formatters at the time the document
   was first uploaded.




Hoffman & Hildebrand     Expires August 13, 2016                [Page 3]

Internet-Draft        RFC v3 Prep Tool Description         February 2016


4.  Canonical RFC Preparation

   During AUTH48, the RPC will run the prep tool in canonical RFC
   preparation mode and make the results available to the authors so
   they can see what the final output might look like.  When the
   document is done with AUTH48 review, the RPC runs the prep tool in
   canonical RFC preparation mode one last time, locks down the
   canonicalized XML, runs the formatters for the publication formats,
   and publishes all of those.  It is probably a good idea for the RPC
   to keep a copy of the input XML file from the various steps of the
   RFC production process.

   This document assumes that the prep tool will be used in the
   following manner by the RFC Production Center; they may use something
   different, or with different configuration.

   Similarly to I-D's, the prepped XML can be used later to re-render
   the output formats, or to generate new formats.

5.  What the v3 Prep Tool Does

   The steps listed here are in order of processing.  In all cases where
   the prep tool would "add" an attribute or element, if that attribute
   or element already exists, the prep tool will check that the
   attribute or element is correct.  If the value is incorrect, the prep
   tool will warn with the old and new values, then replace the
   incorrect value with the new value.

   1.   Fully process any DTDs in the input document, then remove the
        DTD.  At a minimum, this entails processing the entityrefs and
        includes for external files.

   2.   Process all <x:include> elements.  Note: <x:include>d XML may
        include more <x:include>s (with relative URLs rooted at the
        xml:base).  The tool may be configurable with a limit on the
        depth of recursion.

   3.   Run idnits. idnits will indicate if it encountered any errors,
        and will also provide text with all of the warnings and errors
        in a human-readable form.  The prep tool displays all the
        warnings and errors, and stops if there was an error.

   4.   Remove processing instructions.

   5.   If in RFC production mode, remove comments.

   6.   Add the "Status of this Memo" text with current values.
        However, if different boilerplate text already exists in the



Hoffman & Hildebrand     Expires August 13, 2016                [Page 4]

Internet-Draft        RFC v3 Prep Tool Description         February 2016


        input, produce a warning that says that other tools,
        specifically the draft submission tool, will treat that
        condition as an error.  The application will use the
        "submissionType", and "consensus" attributes of the <rfc>
        element, and the "status" and "stream" attributes of the
        <seriesInfo> element, to determine which [RFC5741] boilerplate
        to include, as described in Appendix A of [I-D.iab-xml2rfc].

   7.   Add the "Copyright Notice" text.  The application will use the
        "ipr" and "submissionType" attributes of the <rfc> element and
        the <date> element to determine which portions and which version
        of the TLP to use, as described in A.1 of [I-D.iab-xml2rfc].

   8.   Fill in the "prepTime" attribute of <rfc> with the current
        datetime.

   9.   Fill in the "name" attribute of the <seriesInfo> in the <rfc>
        with a string indicating the type of prepping that was done (RFC
        production or I-D mode).

   10.  If in I-D mode, if there is a <note> element with a
        "removeInRFC" attribute that has the value "true", add a
        paragraph to the top of the <note> element that says "This note
        is to be removed before publishing as an RFC.", if such a
        paragraph does not yet exist.

   11.  If in I-D mode, fill in "expiresDate" attribute of <rfc> based
        on the <date> element of the document's <front> element, if
        there is one.  Use the same expiry date (if needed) in the
        boilerplate.

   12.  Fill in any default values for attributes on elements, except
        "keepWithNext" and "keepWithPrevious" of <t>, and "toc" of
        <section>.

   13.  For any <reference> element that does not already have a
        "target" attribute, fill that attribute in if the element has
        one or more <seriesinfo> child element(s).  The particular URLs
        for RFCs and Internet-Drafts for this step will be specified
        later by the RFC Editor and the IESG.  These URLs might also be
        different before and after the v3 format is adopted.

   14.  For each <reference> or <referencegroup> that has an associated
        <displayreference>, replace the value of the "anchor" attribute
        with the value of the "to" attribute of the associated
        <displayreference>.





Hoffman & Hildebrand     Expires August 13, 2016                [Page 5]

Internet-Draft        RFC v3 Prep Tool Description         February 2016


   15.  Add a "slugifiedName" attribute to each <name> element that does
        not contain one; replace the attribute if it contains a value
        that begins with "n-".

   16.  Add "pn" attributes for all parts.  Parts are:

        *  <section>: pn='s-1.4.2'

        *  <abstract>: pn='s-abstract'

        *  <note>: pn='s-note-2'

        *  <boilerplate>: pn='s-boilerplate'

        *  <table>: pn='t-3'

        *  <figure>: pn='f-4'

        *  <artwork>, <aside>, <blockquote>, <dl>, <dt>, <li>, <ol>,
           <references>, <sourcecode>, <t>, <ul>:
           pn='p-[section]-[counter]'

   17.  Add a "start" attribute to every <ol> element containing a group
        that does not already have a start.

   18.  If the "sortRefs" attribute of the <rfc> element is true, sort
        the <reference>s and <referencegroup>s lexically by the value of
        the "anchor" attribute, as modified by the "to" attribute of any
        <displayreference> element.

   19.  For each <xref> element referring to a <reference> or
        <referencegroup> that has an associated <displayreference>,
        modify the "target" attribute of the <xref> to match the "to"
        attribute of the associated <displayreference>.

   20.  For each <xref> element that has content, fill the
        "derivedContent" with the element content, having first trimmed
        the whitespace from ends of content text.  Issue a warning if
        the "derivedContent" attribute already exists and has a
        different value from what was being filled in.

   21.  For each <xref> element that does not have content, fill the
        "derivedContent" based on the "format" attribute.

        *  For format='counter', the "derivedContent" is the section,
           figure, table, or ordered list number of the element with
           anchor equal to the xref target.




Hoffman & Hildebrand     Expires August 13, 2016                [Page 6]

Internet-Draft        RFC v3 Prep Tool Description         February 2016


        *  For format='default' and the "target" attribute points to a
           <reference> or <referencegroup> element, the "derivedContent"
           is the value of the "target" attribute (or the "to" attribute
           of a <displayreference> element for the targeted
           <reference>).

        *  For format='default' and the "target" attribute points
           something else, the "derivedContent" is the title of the
           thing pointed to, such as "Section 2.3" or "Table 4".

        *  For format='title', if the target is a <reference> element,
           the "derivedContent" attribute is the name of the reference,
           extracted from the <title> child of the <front> child of the
           reference.

        *  For format='title', if the target element has a <name> child
           element, the "derivedContent" attribute is the text content
           of that <name> element concatenated with the text content of
           each descendant node of <name> (that is, stripping out all of
           the XML markup, leaving only the text).

        *  For format='title', if the target element does not contain a
           <name> child element, the "derivedContent" attribute is the
           value of the "target" attribute with no other adornment.
           Issue a warning if the "derivedContent" attribute already
           exists and has a different value from what was being filled
           in.

   22.  For each <relref> element referring to a <reference> or
        <referencegroup> that has an associated <displayreference>,
        modify the "target" attribute of the <relref> to match the "to"
        attribute of the associated <displayreference>.

   23.  For each <relref> element, fill in the "derivedLink" attribute.

   24.  For each <relref> element that does not have content, fill the
        "derivedRemoteContent" based on the content of the target
        reference.

        *  If the target reference is an RFC or Internet-Draft in the v3
           format, find the anchor given in the "relative" attribute or
           derived from the "section" attribute, and use the identifier
           of that element (such as "Section 2.3" or "Table 4") for the
           "derivedRemoteContent".

        *  If the target reference is not a RFC or Internet-Draft in the
           v3 format, use the value of the "relative" or "section"
           attribute for the "derivedRemoteContent".



Hoffman & Hildebrand     Expires August 13, 2016                [Page 7]

Internet-Draft        RFC v3 Prep Tool Description         February 2016


        *  Issue a warning if the "derivedRemoteContent" attribute
           already exists and has a different value from what was being
           filled in.

   25.  For each <relref> element that has content, fill the
        "derivedRemoteContent" with the element content, having first
        trimmed the whitespace from ends of content text.  Issue a
        warning if the "derivedRemoteContent" attribute already exists
        and has a different value from what was being filled in.

   26.  Remove every <displayreference> element.

   27.  If an <artwork> element has a "src" attribute where no scheme is
        specified, treat the scheme as "file:" in a path relative to the
        file being processed.  This will likely be one of the most
        common authoring approaches.

   28.  If an <artwork> element has a "src" attribute with a "file:"
        scheme, and if processing the URL would cause the processor to
        retrieve a file that is not in the same directory, or a
        subdirectory, as the file being processed, give an error.  This
        rule attempts to prevent <artwork src='file:///etc/passwd'> and
        similar security issues.

   29.  If an <artwork> element has type='svg' and there is a "src"
        attribute, the data needs to be moved into the content of the
        <artwork> element.

        *  If the "src" URI scheme is "data:", fill the content of the
           <artwork> element with that data and remove the "src"
           attribute.

        *  If the "src" URI scheme is "file:", "http:", or "https:",
           fill the content of the <artwork> element with the resolved
           XML from the URI in the "src" attribute.  Add an
           "originalSrc" attribute with the value of the URI and remove
           the "src" attribute.

   30.  If an <artwork> element has type='binary-art', the data needs to
        be in a "src" attribute with a URI scheme of "data:".  If the
        "src" URI scheme is "file:", "http:", or "https:", resolve the
        URL.  Replace the "src" attribute with a "data:" URI, add an
        "originalSrc" attribute with the value of the URI, and remove
        the "src" attribute.  For the "http:" and "https:" URI schemes,
        the mediatype of the "data:" URI will be the Content-Type of the
        HTTP response.  For the "file:" URI scheme, the mediatype of the
        "data:" URI needs to be guessed with heuristics (this is
        possibly a bad idea).  Note: since this feature can't be used



Hoffman & Hildebrand     Expires August 13, 2016                [Page 8]

Internet-Draft        RFC v3 Prep Tool Description         February 2016


        for RFCs at the moment, this entire feature might be de-
        prioritized.

   31.  If an <artwork> element does not have type='svg' or
        type='binary-art' and there is a "src" attribute, the data needs
        to be moved into the content of the <artwork> element.  Note
        that this step assumes that all of the preferred types other
        than "binary-art" are text.

        *  If the "src" URI scheme is "data:", fill the content of the
           <artwork> element with the correctly-escaped form of that
           data and remove the "src" attribute.

        *  If the "src" URI scheme is "file:", "http:", or "https:",
           fill the content of the <artwork> element with the correctly-
           escaped form of the resolved text from the URI in the "src"
           attribute.  Add an "originalSrc" attribute with the value of
           the URI and remove the "src" attribute.

   32.  If a <sourcecode> element has a "src" attribute where no scheme
        is specified, treat the scheme as "file:" in a path relative to
        the file being processed.

   33.  If a <sourcecode> element has a "src" attribute with a "file:"
        scheme, and if processing the URL would cause the processor to
        retrieve a file that is not in the same directory, or a
        subdirectory, as the file being processed, give an error.  This
        rule attempts to prevent <sourcecode src='file:///etc/passwd'>
        and similar security issues.

   34.  If a <sourcecode> element has a "src" attribute, the data needs
        to be moved into the content of the <sourcecode> element.

        *  If the "src" URI scheme is "data:", fill the content of the
           <sourcecode> element with the appropriately-escaped data and
           remove the "src" attribute.

        *  If the "src" URI scheme is "file:", "http:", or "https:",
           fill the content of the <sourcecode> element with the
           appropriately-escaped resolved text from the URI in the "src"
           attribute.  Add an "originalSrc" attribute with the value of
           the URI and remove the "src" attribute.

   35.  Determine all the characters used in the document, and fill in
        the "scripts" attribute for <rfc>.

   36.  Ensure that the output has the "version" attribute of <rfc>, and
        that it is set to "3".



Hoffman & Hildebrand     Expires August 13, 2016                [Page 9]

Internet-Draft        RFC v3 Prep Tool Description         February 2016


   37.  If in RFC production mode, remove all <link> elements whose
        "rel" attribute has the value "alternate".

   38.  If in RFC production mode, check if there is a <link> element
        with an ISSN for the RFC series; if not, add one.

   39.  If in RFC production mode, check if there is a <link> element
        with a DOI for this RFC; if not, add one.

   40.  If in RFC production mode, check if there is a <link> element
        with the file name of the Internet-Draft that became this RFC;
        if not, add one.

   41.  If in RFC production mode, remove all "xml:base" or
        "originalSrc" attributes from all elements.

   42.  Pretty-format the XML output.  (Note: tools like
        https://github.com/hildjj/dentin do an adequate job.)

   43.  If in RFC production mode, ensure that the result is in full
        compliance to v3 schema, without any deprecated elements or
        attributes, and give an error if any issues are found.

6.  Additional Uses for the Prep Tool

   There will be a need for Internet-Draft authors who include files
   from their local disk (such as for <artwork src="mydrawing.svg"/>) to
   have the contents of those files inlined to their drafts before
   submitting them to the Internet-Draft processor.  (There is a
   possibility that the Internet-Draft processor will allow XML files
   and accompanying files to be submitted at the same time, but this
   seems troublesome from a security, portability, and complexity
   standpoint.)  For these users, having a local copy of the prep tool
   that has an option to just inline all local files would be terribly
   useful.  That option would be a proper subset of the steps given in
   Section 5.

   A feature that might be useful in a local prep tool would be the
   inverse of the "just inline" option would be "extract all".  This
   would allow a user who has a v3 RFC or Internet-Draft to dump all of
   the <artwork> and <sourcecode> elements into local files instead of
   having to find each one in the XML.  This option might even do as
   much validation as possible on the extracted <sourcecode> elements.
   This feature might also remove some of the features added by the prep
   tool (such as part numbers and slugifiedName's starting with "n-") in
   order to make the resulting file easier to edit.





Hoffman & Hildebrand     Expires August 13, 2016               [Page 10]

Internet-Draft        RFC v3 Prep Tool Description         February 2016


7.  IANA Considerations

   None.

8.  Security Considerations

   Steps in this document attempt to prevent the <artwork> and
   <sourcecode> entities from exposing the contents of files outside the
   directory in which the document being processed resides.

9.  Acknowledgements

   Many people contributed valuable ideas to this document.  Special
   thanks go to Robert Sparks for his in-depth review and contributions
   early in the development of this document.

10.  Informative References

   [I-D.iab-xml2rfc]
              Hoffman, P., "The "xml2rfc" version 3 Vocabulary", draft-
              iab-xml2rfc-02 (work in progress), January 2016.

   [RFC5741]  Daigle, L., Ed., Kolkman, O., Ed., and IAB, "RFC Streams,
              Headers, and Boilerplates", RFC 5741, DOI 10.17487/
              RFC5741, December 2009,
              <http://www.rfc-editor.org/info/rfc5741>.

   [RFC6949]  Flanagan, H. and N. Brownlee, "RFC Series Format
              Requirements and Future Development", RFC 6949, DOI
              10.17487/RFC6949, May 2013,
              <http://www.rfc-editor.org/info/rfc6949>.

Authors' Addresses

   Paul Hoffman
   ICANN

   Email: paul.hoffman@icann.org


   Joe Hildebrand
   Cisco

   Email: jhildebr@cisco.com







Hoffman & Hildebrand     Expires August 13, 2016               [Page 11]