TOOLS team A. Rousskov Internet-Draft The Measurement Factory Expires: April 6, 2005 October 06, 2004 Requirements for IETF Draft Submission Toolset draft-ietf-tools-draft-submission-03 Status of this Memo This document is an Internet-Draft and is subject to all provisions of section 3 of RFC 3667. By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she become aware will be disclosed, in accordance with RFC 3668. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on April 6, 2005. Copyright Notice Copyright (C) The Internet Society (2004). Abstract This document specifies requirements for an IETF toolset facilitating Internet-Draft submission, validation, and posting. Rousskov Expires April 6, 2005 [Page 1] Internet-Draft Draft Submission Toolset: Requirements October 2004 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. State of this draft . . . . . . . . . . . . . . . . . . . . . 3 3. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 4. Notation and Terminology . . . . . . . . . . . . . . . . . . . 4 5. Status quo . . . . . . . . . . . . . . . . . . . . . . . . . . 4 6. Overall toolset operation . . . . . . . . . . . . . . . . . . 6 7. Upload page . . . . . . . . . . . . . . . . . . . . . . . . . 8 8. Check action . . . . . . . . . . . . . . . . . . . . . . . . . 9 8.1 Preprocessing . . . . . . . . . . . . . . . . . . . . . . 9 8.2 Storage . . . . . . . . . . . . . . . . . . . . . . . . . 10 8.3 Extraction . . . . . . . . . . . . . . . . . . . . . . . . 10 8.4 Validation . . . . . . . . . . . . . . . . . . . . . . . . 11 8.4.1 Absolute requirements . . . . . . . . . . . . . . . . 12 8.4.2 Desireable features . . . . . . . . . . . . . . . . . 13 9. Check page . . . . . . . . . . . . . . . . . . . . . . . . . . 13 9.1 External meta-data . . . . . . . . . . . . . . . . . . . . 14 10. Post Now action . . . . . . . . . . . . . . . . . . . . . . 14 11. Adjust action . . . . . . . . . . . . . . . . . . . . . . . 15 12. Adjust page . . . . . . . . . . . . . . . . . . . . . . . . 15 13. Post Manually action . . . . . . . . . . . . . . . . . . . . 16 14. Receipt page . . . . . . . . . . . . . . . . . . . . . . . . 16 15. Bypassing the toolset . . . . . . . . . . . . . . . . . . . 16 16. E-mail interface . . . . . . . . . . . . . . . . . . . . . . 17 17. Implementation stages . . . . . . . . . . . . . . . . . . . 19 18. Security Considerations . . . . . . . . . . . . . . . . . . 20 19. IANA Considerations . . . . . . . . . . . . . . . . . . . . 20 20. Compliance . . . . . . . . . . . . . . . . . . . . . . . . . 20 A. Comparison with current procedures . . . . . . . . . . . . . . 21 B. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 21 C. Change log . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Author's Address . . . . . . . . . . . . . . . . . . . . . . . 25 Intellectual Property and Copyright Statements . . . . . . . . 26 Rousskov Expires April 6, 2005 [Page 2] Internet-Draft Draft Submission Toolset: Requirements October 2004 1. Introduction Public Internet-Drafts are primary means of structured communication within IETF. Current Internet-Draft submission and posting mechanisms hinder efficient and timely communication while creating unnecessary load on the IETF Secretariat. IETF TOOLs team recommends formalization and automation of the current mechanisms. This document contains specific automation requirements. IETF Secretariat and many IETF participants have long been proponents of automation. This document attempts to reflect their known needs and wishes, as interpreted by the TOOLs team. 2. State of this draft While nothing has been set in stone yet, this draft approaches its stable state. Text marked with "XXX" usually contains informal descriptions of known problems the team still needs to solve. These "XXXs" are uniquely numbered for ease of reference. Please review this draft. Comments on substance and XXX issues are requested. Editorial comments would be most useful at a later stage, when most XXXs are resolved. Please post comments on tools-discuss@ietf.org mailing list or e-mail them directly to the author. RFC Editor Note: Please remove this section for the final publication of the document. It has been inspired by draft-rousskov-newtrk-id-state and related NEWTRK WG discussions. 3. Scope Draft Submission Toolset is about getting a single new revision of Internet-Draft from an IETF participant to the IETF draft repository. A single draft revision may include several formats, and dealing with those formats is in Toolset scope. Definition and sources of draft meta-information (to be used in Secretariat databases and elsewhere) are in scope. Submitter authentication and submission authorization are in scope. Draft posting may result in various notifications sent to interested parties. While this document recommends a subset of notification targets, details of notifications are out of scope. Creation of new drafts or new draft revisions as well as manipulation, visualization, and interaction with the drafts already in the repository are out of scope. Draft expiration and archiving of old draft revisions are out of scope. Rousskov Expires April 6, 2005 [Page 3] Internet-Draft Draft Submission Toolset: Requirements October 2004 4. Notation and Terminology The following terms are to be interpreted according to their definitions below. [[XXX48: delete unused terms, if any. --Alex]] Posted draft: A draft accepted into public IETF draft repository and, hence, publicly available on IETF web site. Posting (a.k.a., publication) of a draft does not imply any IETF or IESG review and endorsement. Submitter: A human or software submitting an Internet-Draft for validation or posting. Authorized Submitter: Any of the draft authors or editors listed in the draft text and any member of IETF Secretariat staff. Immediately: without human interaction or artificial software delays. The Toolset is specified using a set of normative requirements. These requirements are English phrases ending with an "(Rnnn)" mark, where "nnn" is a unique requirement number. This document specifies interface and functionality of the toolset, not details of the toolset implementation. However, implementation hints or examples are often useful. To avoid mixup with toolset requirements, such hints and examples are often marked with a "Hint:" prefix. Implementation hints do not carry any normative force, and a different implementation may be the best choice. 5. Status quo This section summarizes the process for draft submission and posting as it exists at the time of writing. To get an Internet-Draft posted on IETF web site, an IETF participant e-mails draft text to the IETF Secretariat, along with an informal note asking to post the draft. Secretariat staff reads the note, reviews the draft according to a checklist, and then approves or rejects the submission. Draft approval triggers the corresponding announcement to be sent to appropriate IETF mailing lists. Every 4 hours, approved drafts are automatically copied to the IETF drafts repository and become available on IETF web site. Collectively, IETF participants submit thousands of Internet-Drafts per year (in the year 2000, about three thousand drafts were submitted; 2002: 5K; 2004: 7K). About 30-50% of posted drafts are Working Group drafts (among some 2,100 drafts, there were about 380 new and 290 updated WG drafts posted in 2003). While no statistics Rousskov Expires April 6, 2005 [Page 4] Internet-Draft Draft Submission Toolset: Requirements October 2004 is available, the vast majority of submitted drafts are approved by Secretariat for posting. It usually takes the Secretariat a few minutes to review a given draft. However, since the Secretariat staff does not work 24/7, does not work in all time zones, and since approved drafts are posted in batches every 4 hours, it may take from several hours to a couple of days to get a draft posted. Due to much higher demand and fixed processing capacity, postings during the last weeks before IETF face-to-face meetings take much longer, creating a long queue of unprocessed drafts that are then announced nearly simultaneously. To give IETF face-to-face meeting participants time to review relevant documents, Secretariat does not accept Internet-Draft submissions close to IETF meetings (regardless of whether a draft is relevant to the upcoming meeting). Many Working Groups have come up with ad hoc solutions to cope with posting delays. For example, many draft subversions are "temporary" published on personal web sites or sent (completely or in part) to the group list. Alternative means of publication may effectively replace official IETF interfaces, with only a few major draft revisions end up being posted on IETF web site. Informal interfaces for submitting and posting drafts discourage automation. Lack of submission automation increases Secretariat load, complicates automated indexing and cross-referencing of the drafts, and, for some authors, leads to stale drafts not being updated often enough. Beyond a short Secretariat checklist, submitted drafts are not checked for compliance with IETF requirements for archival documents, and submitters are not notified of any violations. As a result, IESG and RFC Editor may have to spend resources (and delay standard approval) resolving violations with draft authors. Often, these violations can be detected automatically and would have been fixed by draft authors if authors knew about them before requesting to publish the draft as a standard. Technically, anybody and anything can submit a draft to the Secretariat. There is no reliable authentication mechanism in place. Initial submissions of WG drafts require WG Chair approval, which can be faked just like the submission request itself. No malicious impersonations or fake approvals have been reported to date however. Lack of authentication is not perceived as a serious problem, possibly because serious falsification are likely to be noticed before serious damage can be done. Due to informal and manual nature Rousskov Expires April 6, 2005 [Page 5] Internet-Draft Draft Submission Toolset: Requirements October 2004 of the submission mechanism, its massive automated abuse is unlikely to cause anything but a short denial of draft posting service and, hence, is probably not worth defending against. However, future automation may result in a different trade off. 6. Overall toolset operation This section provides high-level description for the proposed toolset. The description is meant to show overall operation and order; please refer to other sections for details specific to each step. A typical submitter goes through a sequence of 2-4 web pages and associated actions. The number of pages depends on the draft validation and meta-data extraction results. For example, validating the draft without posting it requires interacting with two web pages: Upload and Check. The common case of posting a valid draft without manual meta-data adjustments takes three web pages (Upload, Check, Receipt). Here is a brief overview of pages and actions: Upload page: Interface to copy draft from submitters computer into the toolset staging area (Section 7). Multiple formats are accepted. The draft is sent to the Check action. Check action: Stores the draft in the toolset staging area, extracts draft meta-data, validates the submission (Section 8). Produces the Check page. Check page: Displays draft interpretation and validation results (Section 9). Draft rendering preview may also be given on this page. After reviewing draft interpretation and validation results, the submitter has four basic choices (a) auto-post draft "as is" now; (b) make manual corrections and submit the draft to Secretariat for manual posting later; (c) cancel submission; or (d) do nothing. Automated posting option may not be available for drafts that failed validation. Automated posting: If submitter decides to proceed with automated posting from the Check page, the system authenticates the submitter and checks whether the submitter is allowed to post the draft. If submitter is authorized, the draft is immediately posted, deleted from the staging area, and the submitter is notified of the result via e-mail and Receipt page (Section 10). Rousskov Expires April 6, 2005 [Page 6] Internet-Draft Draft Submission Toolset: Requirements October 2004 Manual adjustment and posting: If submitter decides to adjust meta-data, the draft remains in the toolset staging area, and the Adjust action (Section 11) presents the submitter with an Adjust page (Section 12). When submitter makes the adjustments and proceeds with manual posting, a pointer to the stored draft and its adjusted meta-data is sent to the secretariat for manual processing (Section 13). The submitter is notified of the pending Secretariat request via e-mail and Receipt page. Cancellation: If submitter decides to explicitly cancel the submission, the draft is deleted from the toolset staging area and an appropriate Receipt page is generated with no further actions. Receipt page: Contains details of a successful or failed draft submission and informs the submitter of the next appropriate step(s) related to submission result. The following diagram illustrates basic submission logic: /---> Post Now / Upload --> Check -+-----> Adjust ---> Send to Secretariat \ \---> Cancel If post-if-valid option is used, the submission logic becomes simpler as draft checking becomes implicit: /---> Post Now Upload -+ \---> Error Receipt If the submitter does not select any explicit action for 1 hour, then the submission is considered abandoned, and all corresponding state may be deleted by garbage collection (R66). The staging area maintenance algorithms must be robust in the presence of DoS attacks attempting to overwhelm the area with fake submissions in various stages (R67). The "web pages" this text is referring to are distinct dialogs, that may be visible to the submitter under the same or different URL, and supported by a single or several CGI scripts (or even applets). The Toolset must handle multiple submitters simultaneously submitting the same draft (R72) and a single submitter simultaneously submitting two drafts (R73). The latter might happen, for example, when the submitter is using several browser windows to submit several drafts or is submitting drafts via e-mail interface. The term Rousskov Expires April 6, 2005 [Page 7] Internet-Draft Draft Submission Toolset: Requirements October 2004 "simultaneously" means that submission processing times overlap. Hint: Except for the Upload page, pages contain submission session identifier to provide actions with access to stored information. The identifier is specific to the submission rather than draft version or the submitter. While the nature of web interface allows the session identifier to be invisible to the submitter, e-mail communication would need to identify the session so that the recipient (and Toolset) know the context. Hint: A single action may correspond to multiple programs and, vice versa, a single CGI program may implement several actions. Actions preserve and exchange state by storing it along with draft. Grouping all submission-specific information in one subdirectory named using session identifier may increase robustness and simplify debugging. Session creation and destruction can then be logged in a global index. Ways to partially or completely bypass the toolset are documented in Section 15 [[XXX4: Need to add details on how each action interacts with IETF infrastructure. --Alex]] 7. Upload page To upload the draft, the submitter goes to a well-known page on IETF web site (R1). There, the draft text can be uploaded using HTML file input form. This form provides input fields to upload plain text format of the draft (R2) and all other formats allowed by IETF draft publication rules (R3). At the time of writing, these formats are: XML (RFC 2629 and draft-mrose-writing-rfcs), Postscript, and PDF [[XXX66: is XML currently allowed? --Alex]]. The Upload page exposes the "post-if-valid" knob of the Check action (Section 8) (R80). Hint: a corresponding "post if valid" checkbox next to the "upload" or "validate" button may suffice. Submitted forms are handled by the Check action (R5). The Toolset should have a Validation page, identical to the Upload page with the exception of its action. Submitting a draft via the Validation page causes the draft to be validated exactly like using Upload page would (R74). Regardless of the validation results, the stored draft meta-data is marked so that validation-only drafts can be identified and deleted first by garbage collector for the toolset staging area (R75). Drafts uploaded via the Validation page cannot be be posted (R76); they would need to be uploaded via Upload page Rousskov Expires April 6, 2005 [Page 8] Internet-Draft Draft Submission Toolset: Requirements October 2004 for that, and the validation results page should explain that (R77). This page, hosted at a well-known location, would be useful for tools using online validation, especially for bulk draft processing. 8. Check action The Check action preprocesses XML draft source (if any), stores submitted draft (all formats) in the staging area, and then extracts meta-data and validates each format (R6). Failure of each operation results in action failure, indicated to submitter via a non-successful HTTP response status code with a human-friendly explanation in the response body (R7). The toolset must reject submissions lacking plain text or XML sources format (R68). XML sources are acceptable without plain text only if the tool successfully generates plain text version of the draft from those XML sources, as a part of the preprocessing step (R69). XML source needs to be preprocessed to resolve XML processor instructions to include references. The action needs to store the draft so that successfully validated drafts can later be auto-posted at submitter request. The action needs to extract meta-data to perform validation and posting. Drafts need to be validated to catch broken submissions and to block automated submissions of malformed final draft versions for IETF Last Call and IESG review. The Check interface includes a "post-if-valid" parameter. When set, the action proceeds with posting of valid drafts (i.e., the Post Now action documented in Section 10), without asking for an explicit confirmation (R78). The parameter is ignored when draft validation fails (R79). This functionality is similar to using e-mail interface. 8.1 Preprocessing XML source, if any, is preprocessed in an attempt to resolve all XML processor instructions (PIs) (R8). The XML preprocessor uses public database(s) to resolve PI references (R85). The toolset documentation specifies what databases are used and how PIs are mapped to database entries (R86). The toolset must not rely on PIs existence (R87) because some XML sources will be preprocessed before the submission or will be written without PIs. Hint: Local up-to-date copies of Marshall Rose's reference databases at xml.resource.org can be used. Both original and preprocessed XML sources may be posted later. The original source with include PIs may be useful to the RFC Editor and diffs. The preprocessed version without PIs becomes the default Rousskov Expires April 6, 2005 [Page 9] Internet-Draft Draft Submission Toolset: Requirements October 2004 public XML source of the posted draft (R10). If any of the include PIs cannot be handled, the entire action fails (R11). When no plain text version of the draft is submitted, but XML sources are available, the toolset attempts to generate plain text version from submitted XML sources (R70). Draft formats other than XML are not preprocessed. 8.2 Storage The Check action stores all submitted formats of the draft in a staging area dedicated to the Toolset (R12). If, after garbage collection, the staging area is full (i.e., the total used size reached configured maximum capacity), the action fails (R13). 8.3 Extraction Each stored draft format is interpreted to extract draft meta-data (R14). If a given format is interpreted and meta-data extraction fails, the format is not accepted for automated posting (R15). In this case, the submitter can either delete the offending format(s) from the submission and auto-post OR proceed along the manual submission route so that the Secretariat can verify all formats. Section 17 documents non-obvious implementation schedule related to the above two requirements. When only partial support for format interpretation is available, only interpreted formats are subject to extraction and validation requirements. In other words, if the toolset does not yet support interpretation of a given format, then the corresponding information is stored and made available "as is", regardless of the actual content. The draft interpreter extracts the following meta-data from each draft format (R16). identifier: Also known as draft "filename". For example, draft-ietf-tools-draft-submission-13. revision: A non-negative integer also known as draft version. For example, 13 in draft-ietf-tools-draft-submission-13. name: Common part of all draft identifiers for all revisions of the same draft. For example, draft-ietf-tools-draft-submission in draft-ietf-tools-draft-submission-13. Rousskov Expires April 6, 2005 [Page 10] Internet-Draft Draft Submission Toolset: Requirements October 2004 WG ID: IETF working group identifier. WG value is empty for individual drafts not meant to be work items of a known WG and for non-IETF drafts. For example, "tools" in "draft-ietf-tools-draft-submission-13" and "opes" in "draft-rousskov-opes-ocp-00" are both WG IDs. WG flag: True for IETF WG drafts and false for all other drafts. For example, "true" for "draft-ietf-tools-draft-submission-13". title: A human-friendly draft title. For example, the title of this draft is "Requirements for IETF Draft Submission Toolset" authors: A list of all draft authors. For each author, their first name, last name, and e-mail are extracted. [[XXX53: What if e-mail is not available? Should auto-posting be allowed for drafts without author e-mails? Even though this will make it easier to post drafts without co-author knowledge/consent? --Alex]] abstract: Draft abstract text. submission date: Draft submission date. expiration date: Draft expiration date. Initially, the toolset uses draft name to extract WG ID (R88) and to detect WG drafts (R89). Currently, WG drafts do not have to contain WG name as a third component. If IETF policy is not changed to require uniform naming of WG drafts, the toolset can eventually consult IETF databases to check WG status of individual-looking drafts (R90). [[XXX11: number of pages and size would depend on format version; they probably should not (do not need to) be specified manually; do we need to extract them at this stage? --Alex]] 8.4 Validation IETF standards have to follow a set of syntax and semantics requirements [[XXX12: provide references to the nits document and IP policies --Alex]]. Most of those requirements are not enforced for Internet-Drafts. However, following them may improve draft quality, reduce IESG load, and increase the chances of the draft being approved as an RFC. When validating a given draft, it is important to distinguish between absolute requirements and desirable draft properties. Both categories are checked for, but violations have different effects depending on the category. The two categories are detailed in the Rousskov Expires April 6, 2005 [Page 11] Internet-Draft Draft Submission Toolset: Requirements October 2004 following subsections. 8.4.1 Absolute requirements Violating any of these requirements would prevent a draft to be automatically posted (R17). The offending draft would have to be fixed or submitted for manual posting, with an explanation why the absolute requirements need to be violated (or why Validator mis-detected violations). These explanations may speedup Secretariat posting decision and may help help Secretariat to improve draft submission toolset. 1. All available meta-data entries must match across all submitted draft formats (R18). For example, if the interpreter managed to extract draft title from plain text and PDF version, both titles must match. This requirement prevents accidental submission of mismatching formats. 2. A draft must be submitted by an authorized submitter (R19). [[XXX13: move this lower, we do not know the submitter at this stage --Alex]][[XXX14: Don't forget the co-author, changed editor, and drafts being posted anonymously (secretariat needs to know who submitter is, but name doesn't necessarily appear on draft) --Harald]] 3. A Working Group draft must be approved as a WG draft by the corresponding Working Group (R20). This approval is usually relayed before or after the submission of the -00 version of the draft, by a Chair or Secretary of the corresponding WG. [[XXX58: Document what the toolset must do if no approval exists at the time of the submission. --Alex]] 4. Current draft state must allow new revisions to be posted (R21). [[XXX15: document IESG review states when new revisions are allowed. Secretariat, Harald, and Pekka opinions seem to differ here. Secretariat: No revisions are allowed in any state except for "I-D exists", "AD watching", or an explicit IESG request for a new revision. Harald: no revisions once submitted for publication. Pekka: In practice, revisions are allowed in any state; don't try to solve a process knowledge problem (the I-D author should not submit new versions) using a technical means. Need further clarification. If no clarification, always allow until approved, but issue a warning when draft is under IESG control. --Alex]]. 5. Correct draft ID (including correct revision number with respect to already published revisions, if any) must appear in the draft text (R22). Rousskov Expires April 6, 2005 [Page 12] Internet-Draft Draft Submission Toolset: Requirements October 2004 6. An IETF IPR statement must appear in the draft text (R23). [[XXX16: add the applicable parts of RFC 2026, 3667 and 3668 - this is mostly a matter of checking the presence of boilerplate text. --Henrik]][[XXX57: Do you mean that the I-D boilerplate must be present, or that the entire IPR and Copyright text must be present, or both? --Brian E. Carpenter]] 8.4.2 Desireable features Violating any of the following requirements does not prevent the submitter to auto-post the draft (R24). TBD: list testable nits here or refer to the nits document. Henrik's idnits tool is a starting point: http://ietf.levkowetz.com/tools/idnits/ [[XXX33: What to do when two versions are submitted with a suspiciously small gap? How small should the gap be to warrant warnings/actions? --Stanislav]][[XXX59: to prevent both DOS attacks and human error, anything less than 48 hours should be kicked over for manual handling. --Brian E. Carpenter]] When a valid draft is being posted and submitter authorization or co-author notification is performed, validation results should be included in the e-mail (R81) so that the submitter can see meta-data extraction and validation warnings. Note that the results cannot include errors since only valid drafts can be posted. 9. Check page The Check page, created by the Check action displays extracted draft meta-data and validation results (R25). The purpose of the page is to allow the submitter to verify whether stored draft and automatically extracted meta-data match submitter's intent and to be informed of validation problems. Extracted meta-data items that were not successfully extracted or that failed validation checks must be marked specially (rather than silently omitted) (R26). Validation messages include both errors and warnings. Each validation message should refer to normative document(s) containing corresponding validation rules (R27). The submitter can also enter external meta-data (Section 9.1), which is required for automated posting of the draft (R28). If validation was successful, an "automatically post the draft now" button is provided (R29). Regardless of validation results, "adjust and post manually" and "cancel" buttons are provided (R30). Rousskov Expires April 6, 2005 [Page 13] Internet-Draft Draft Submission Toolset: Requirements October 2004 The Check page provides a preview of the draft plain text format rendering (R31), with a link to see how the entire draft (with all its formats) would be rendered if posted (R82). Hint: the Check page preview should be sufficiently long to let authors detect obvious draft mismatch or misinterpretation errors but short enough to avoid dominating the page. Displaying just the draft "prefix", including abstract, may be sufficient. For draft updates, the Check page reports the the time and the submitter of the last update (R83). This information is especially useful when multiple authors are working on the same draft. The page also provides a link to generate a diff against the last posted version (R84). 9.1 External meta-data The Check page solicits the following meta-data from the submitter. This information must be supplied by submitter because it cannot be extracted from the draft: Which author is the submitter? (R32) [[XXX30: with the exception for Secretariat manual submission? No. Secretariat submits on behalf of one of the authors. --Alex]][[XXX31: Are there any situations when a draft is submitted by somebody other than author and not on explicit authors behalf? What happens to IPR "by submitting this draft the authors ..." statement then? --Alex]]; List of drafts obsoleted by this draft (R33). This is useful to make obsoleted drafts invisible. [[XXX43: Can non -00 draft obsolete other drafts? Or is this reserved for -00 versions for some reason? Is this reserved for WG drafts only? --Alex]]. [[XXX44: This may open a bigger can of worms than we want to deal with, as it allows anybody to "kill" any other draft. The toolset would probably need a manual check by Secretariat and/or approvals from both(?) WG Chairs. A lot of work for something relatively rare? --Alex]][[XXX60: This is too complex and would be a luxury in Version 1. Maybe for Version 2. --Brian E. Carpenter]] E-mail address for discussion of this draft (R71). Hint: The toolset can suggest WG mailing list address for WG drafts, [submitting] author address for individual drafts, or even the first e-mail address in draft text. This information is optional. 10. Post Now action The Post Now action checks that the draft has been successfully validated (R34), validates external meta-data (including submitter Rousskov Expires April 6, 2005 [Page 14] Internet-Draft Draft Submission Toolset: Requirements October 2004 e-mail) (R35), and posts the draft (R36). Submitter is notified of the action progress and final result (R37). External data contains submitter e-mail address. As a part of the validation procedure, the Post Now action checks that the submitter has access to e-mail sent to that address (R38). If access rights are not confirmed, the submission eventually and silently times out (R39). Hint: To check submitter's access to e-mail the tool can e-mail a hard-to-guess cookie or token to the submitter's address. The submitter is then requested to go to the token-holding URL in order to continue with the submission. [[XXX2: should the posting program inform all authors that their draft is being posted (since the draft says they all made an IPR statement)? Default answer: yes. Obtaining consent from all authors seems impractical. --Alex]][[XXX26: responding to e-mail should also be supported (as an alternative to going back to cut-and-paste the URL with a token) --Alex]] If draft posting is successful, toolset state information may be deleted from the toolset storage area [[XXX21: on-demand garbage collection may be better from debugging point of view; what may seem like a successful post may not be that successful --Alex]]. 11. Adjust action The Adjust action generates the Adjust page (R40), populating it with available extracted meta-data and external meta-data as well as validation results and preview. Some or all of the information may be missing, depending on draft interpretation and rendering success. 12. Adjust page The Adjust page includes the same information as the Check page, but allows the submitter to adjust all extracted draft meta-data (and, naturally, external meta-data) at will (R41). Such adjustment is necessary when automated extraction failed to extract [correct] information. To avoid mismatch between draft and its meta-data, adjusted drafts cannot be automatically posted and require manual validation by Secretariat (R42). Secretariat staff can post drafts with adjusted meta-data as described in Section 15. The Adjust page allows the submitter to enter an informal comment explaining why adjustments are necessary and automated posting mode cannot be used (R48). Rousskov Expires April 6, 2005 [Page 15] Internet-Draft Draft Submission Toolset: Requirements October 2004 The "post manually" and "cancel" buttons are provided (R43). The former is backed by the "Post Manually" action (Section 13). 13. Post Manually action The Post Manually action sends adjusted meta-data and draft pointer to the Secretariat for manual validation and posting (R44). A receipt page is generated instruction the submitter to wait (R45). Secretariat will notify the submitter once the draft is posted or rejected. This notification is sent by the toolset if Secretariat is using the toolset to post the draft (R46). 14. Receipt page The Receipt page is generated by various actions to inform the submitter of current submission status and further actions. The contents of the page is likely to be highly dependent on the action and state for which receipt is being generated. This section documents general requirements applicable to all actions and states. The Receipt page should give the submitter a URI or another identifier that can be used by Secretariat for manual troubleshooting of the submission (R63). The identifier should be perpetual (R64) even though the associated details are likely to be eventually lost (e.g., draft submission data and logs are deleted from the staging area as a part of the garbage collection routine). The Receipt page should give the submitter a Secretariat point-of-contact to report submission problems (R65). 15. Bypassing the toolset A buggy toolset or unusual circumstances may force a submitter to submit draft to Secretariat for manual processing. This can be done by choosing the "manual posting" route supported by the toolset (R47) or, as a last resort, by e-mailing the draft directly to Secretariat. In either case, an informal "cover letter" has to accompany the draft. The letter should explain why the automated interface cannot be used. When processing manual submissions, the Secretariat may be able to use the toolset (R49). A Manual Check page similar to the default Check page provides authenticated Secretariat staff with editable meta-data fields and a "force posting" action (R50). The forced posting action accepts meta-data fields "as is" and proceeds with the regular posting algorithm (R51) [[XXX22: Should we document the details even though this page is for internal Secretariat use? --Alex]][[XXX61: Yes. There is no reason for it to be secret. Rousskov Expires April 6, 2005 [Page 16] Internet-Draft Draft Submission Toolset: Requirements October 2004 --Brian E. Carpenter]] [[XXX23: Should forced posting script obtain authors consent or do we want to be able to bypass that as well? --Alex]][[XXX62: I think we need the flexibility - situations can arise where an author is on sabbatical or medical leave or otherwise unavailable for a long period. --Brian E. Carpenter]] Using manual processing may result in significant posting delays. Generated submission receipts or notifications ought to give the submitter an expected processing time estimate (R53). The intent of this mode is to provide a way for submitters to bypass bugs or limitations of automated mechanisms in order to post an "unusual" draft or to post a draft under "unusual" circumstances. One example would be a draft that does not contain standard IETF boilerplate but has a special IESG permission to post the draft with the experimental boilerplate. Another example is a draft that fails automated validation tests due to a validator bug. The bypass mode is also likely to be used (effectively) by the majority of submitters during the Beta stage of the toolset implementation, when few submitters will know about (or be allowed to use) the toolset. [[XXX24: mention that sometimes submitter is not the author or chair so default authentication may not work -- unless we provide an interface to authorize "other" submitters? --Alex]] 16. E-mail interface The toolset should have e-mail interface for automated posting of valid drafts (R55). While virtually every documented toolset functionality can, technically, be implemented behind an e-mail interface, features other than posting of valid drafts are believed to be prohibitively awkward to implement or use via e-mail. E-mail interface accepts a draft as a set of e-mail attachment(s) (one per draft format) (R56), uses sender's e-mail address to select submitters identity (R57), checks the submission as if the draft was submitted via the web interface (R58), and posts the draft if the check is successful (R59). The submitter should be notified of the outcome of the draft submission via e-mail (R60). Notification includes detected errors and warnings (if any) (R61). Toolset actions should be implemented to support e-mail and web interfaces without code duplication. Rousskov Expires April 6, 2005 [Page 17] Internet-Draft Draft Submission Toolset: Requirements October 2004 [[XXX54: What about external meta-data other than submitter's e-mail? Check that no required meta-data input is missing in the e-mail interface. --Alex]] [[XXX37: If there is only one draft format, can e-mail body be used for submission? --Alex]] [[XXX38: To easily filter out spam and reduce human errors, should we require a special keyword in the subject or body of the submission e-mail? For example, we can require and check that the subject starts with "Please post" --Alex]] While both web and e-mail interfaces allow for fast posting of valid drafts, there are significant differences between the two interfaces. Primary advantages of e-mail interface are: off-line mode: A submitter can do all the manual work required to submit a draft while being disconnected from the network. E-mail client actually submits the draft when connectivity is regained. [[XXX39: This advantage would be significantly reduced if we require submitter e-mail verification for each draft submission. Such verification would mean that valid drafts cannot be posted automatically when connectivity is regained -- sending an e-mail just starts the process. --Alex]][[XXX63: Nevertheless, I think the verification emails are required - otherwise we are very exposed to bogons. --Brian E. Carpenter]] poor connectivity: E-mail systems are often better suited for automated transmission and re-transmission of e-mails when network connectivity is poor due to high packet loss ratios, transmission delays, and other problems. convenience: Some IETFers consider e-mail interfaces as generally "more convenient". Primary advantages of web interface are: confirmation: A submitter is given a chance to verify that automated extraction of meta-data produced reasonable results. Other useful confirmations are possible (e.g., "Are you sure you want to post a revision of the draft that was updated 30 seconds ago by your co-author?"). validation: A submitter can validate the draft without posting it. quality: Non-critical warnings may prompt the submitter to postpone posting to improve draft quality. Rousskov Expires April 6, 2005 [Page 18] Internet-Draft Draft Submission Toolset: Requirements October 2004 manual adjustments: The submitter can adjust extracted meta-data and ease Secretariat work on manually posting an unusual draft. context help: Web interface makes it easy to provide links to extra information about input fields, errors, posting options, deadlines, etc. convenience: Some IETFers consider web interfaces as generally "more convenient". 17. Implementation stages This section defines implementation schedule for documented toolset requirements. The schedule is divided into three consecutive phases. Requirements listed in later stages may be covered in earlier stages, but do not have to be. Beta Stage: Initial basic implementation to test major concepts and relieve the Secretariat from handling the most common submission case. This stage should take about [[XXX49: 30? --Alex]] calendar days to implement. 1. R14 and R15 for plain text format. Other formats are accepted but may not be interpreted. 2. R69, R70, and related requirements can be ignored, in which case plain text version is always required. 3. R88, R89 (WG info extraction from draft name) Production Stage: Support for all major features. Once this stage is completed, the Secretariat should only handle unusual draft submissions. This stage should take about [[XXX52: 90? --Alex]] calendar days to implement. 1. R14 and R15 for plain text and XML formats. Other formats are accepted but may not be interpreted. 2. R69, R70. 3. R71 4. R74 - R77. 5. R83 (report last update author/timestamp). Rousskov Expires April 6, 2005 [Page 19] Internet-Draft Draft Submission Toolset: Requirements October 2004 Enhancement Stage: A never-ending stage focusing on sophisticated features that improve draft interpretation and validation. 1. R14 and R15 for all formats. 2. R78 - R80 (post-if-valid functionality). 3. R81 (show warnings to co-authors). 4. R82 (complete draft preview). 5. R84 (a link to generate a diff). 6. R90 (WG info extraction from draft databases) [[XXX50: Is it appropriate for us to specify stage deadlines for the first two stages? Should IESG do that instead? Would be nice to have at least a rough number to illustrate the relative duration of stages though. --Alex]] Implementation experience is likely to result in changes of the toolset requirements. Such changes should be documented as a part of stage evaluation activities. 18. Security Considerations Some. TBD: Talk about why authentication and anti-DoS measures become important once things become automated. When everybody is using an informal e-mail interface, an automated attack will last only until the interface is changed. The informal interface can be changed very quickly. Only the attacker would be suffering from the change, since others do not automate and, hence, are flexible. Once things are automated and interfaces are documented, substantially changing an interface would require rewriting many software agents that use current interfaces. [[XXX25: do we need an explicit IESG approval to require stronger-then-current submitter authentication? --Alex]] 19. IANA Considerations None. 20. Compliance TBD: What does it mean to be compliant with (to satisfy) our requirements? The definition must be usable in the context of IESG evaluation of Secretariat work on implementing the proposed toolset. Rousskov Expires April 6, 2005 [Page 20] Internet-Draft Draft Submission Toolset: Requirements October 2004 Appendix A. Comparison with current procedures TBD: This section summarizes major differences between draft submission approach currently in use by IETF and the proposed toolset, including violations of the current IETF rules. o The toolset allows posting of XML and PDF draft formats. XML format is not currently accepted by the Secretariat, and legality of PDF acceptance by the Secretariat has been questioned. XML sources should be accepted to enable IETF tools and participants to have access to raw draft meta-data and content. They are also useful to the RFC Editor and, hence, it is a good idea to validate and get them "into the system" early. The latter argument applies to PDF drafts as well, although the first Toolset versions are not expected to interpret PDF drafts. [[XXX64: Check whether there are any rules allowing the Secretariat to accept PDF drafts today. --Alex]] o The toolset will automatically notify authors of posted drafts. Currently, niether the submitter nor any of the co-authors are notified when the draft is posted. Notification is meant, in part, to allow co-authors detect cases where their name is put on the authors list without permission. Eventually, there will be a general IETF mechanism to allow 3rd parties such as ADs, chairs, or reviewers to register for draft notifications. Appendix B. Acknowledgments The author gratefully acknowledges the contributions of Harald Tveit Alvestrand (Cisco), Brian E. Carpenter (IBM), Barbara B. Fuller (Foretec), Henrik Levkowetz, Larry Masinter (Adobe), Pekka Savola (Netcore), and Stanislav Shalunov (Internet2). Special thanks to Marshall Rose for his xml2rfc tool. Appendix C. Change log RFC Editor Note: This section is to be removed during the final publication of the document. Internal WG revision control ID: $Id: id.xml,v 1.18 2004/10/06 15:51:44 rousskov Exp $ 2004/10/05 * Resolved XXX9: The toolset should eventually offer a Validation-only page. Rousskov Expires April 6, 2005 [Page 21] Internet-Draft Draft Submission Toolset: Requirements October 2004 * Resolved XXX19: The toolset should eventually provide the submitter with a way to preview the entire draft, with all formats. * Resolved XXX40, XXX41, and XXX56: first use draft name to extract WG flag and WG name and hope for an IETF policy change. If IETF policy on naming drafts does not change soon, add code to query some databases to map individual-looking drafts to WG names. * Resolved XXX46 and XXX47: store and make public both original and preprocessed XML sources. Most tools are likely to use preprocessed XML format. Humans and some diff tools may prefer the original. 2004/09/30 * Added requirements R72 and R73 to handle multiple submitters submitting the same draft and a single submitter submitting two drafts at the same time, addressing XXX27. * Resolved XXX7: There seems to be no good reason to support cut-and-paste mode. Submission via file upload interface should suffice. * Semi-resolved XXX53: Toolset should accept PDFs because RFC Editor does. Still need to check whether the Secretariat accepts PDFs legally today (XXX64). 2004/09/29 * Clarified and polished the "Scope" section. * Updated "State of this draft" to document approaching-last-call state of the draft and to solicit editorial-level feedback. 2004/09/27 * Marked formal toolset requirements using a Rnnn notation to (a) document implementation schedule, and (b) make compliant implementation and compliance evaluation easier. * Marked informal implementation hints with a "Hint:" tag, to avoid possible confusion with formal requirements. * Started documenting implementation schedule. For example, only plain text formats are interpreted during the first stage, then XML support is added, then other formats. Meanwhile, Rousskov Expires April 6, 2005 [Page 22] Internet-Draft Draft Submission Toolset: Requirements October 2004 un-interpreted formats are accepted and posted as is as long as plain text version validates. * Added explicit requirements for managing abandoned submissions (Brian E. Carpenter) * Plain text or XML formats are always required (Brian E. Carpenter) * Added XXX55: Accepting PDFs is a change of current documented procedures? (Brian E. Carpenter) * Added an optional "discussion address" to the external meta-data to help reviewers know where to send comments (inspired by Brian E. Carpenter suggestion; Brian wanted this to be a required extractable meta-data) * Resolved XXX17, XXX28, and XXX29: Today, -00 WG drafts are approved by the Chair either before and after submission, depending on several factors. Based on WG chairs feedback we still need to support both modes. Thus, there is no policy change to talk about (and more work for the tool implementors to support both modes). Still need to add specific toolset requirements in case there is no approval recorded. * Resolved XXX18, XXX32, and XXX45: We are going to move "request for publication" functionality to a separate [simple] tool that works with an existing/posted draft. * Resolved XXX6: We are going to move the "withdraw this ID" functionality desired by Secretariat to a separate [simple] tool that works with an existing/posted draft. * Added a "comment" field to the Adjust page so that the submitter can tell Secretariat why manual action is necessary. This may both save time Secretariat and let them improve the toolset to minimize manual submissions (including fixing validation/extraction bugs). * Added the Receipt page to the list of documented pages, for completeness. * Emphasized that common sequence of pages to go through is as short as possible for a given set of features, and that "page" means "distinct dialog", not necessarily a "distinct URL". Some reviewers thought "there are too many pages". Rousskov Expires April 6, 2005 [Page 23] Internet-Draft Draft Submission Toolset: Requirements October 2004 2004/09/20 * Added "E-mail Interface" section to document how key toolset functionality can be accessed via e-mail. Compared e-mail and web interfaces. (Suggested by Pekka Savola) * Split "WG ID" meta-data into "WG ID" and "WG Flag". The former seems to be easy to extract from the draft name. Noted that the latter (i.e., "this is a working group draft" status) cannot be inferred from some WG drafts (Pekka Savola). * Added "List of drafts obsoleted by this draft" external meta-data item (Pekka Savola), but questioned whether we are ready to automate that. * Added more conflicting opinions to XXX15 and proposed a solution. * Added "Preprocessing" subsection to reflect the discussion on how/whether handle include PIs in XML draft sources. Needs more discussion/work. * Further clarified how an author can request the draft revision to be published (i.e., forwarded to IESG or RFC Editor for review and publication as an RFC or BCP). It's just a checkbox on the web interface. Raised doubts we can pull this off (see XXX45). * Suggested in XXX2 that we would inform all authors but not seek their consent (except for the submitter) when posting their draft. 2004/09/09 * Polished high-level page/action summary and replaced text-based steps diagram with something that looks more like a diagram. * Added "Comparison with current procedures" section placeholder for summarizing what this draft improves/changes/violates. * Frequent draft updates is not always a good thing (Henrik Levkowetz) * Added ideas regarding frequent draft updates warnings (Stanislav Shalunov) * Added "State of this draft" section to encourage review. Rousskov Expires April 6, 2005 [Page 24] Internet-Draft Draft Submission Toolset: Requirements October 2004 2004/09/02 * Documented all major toolset pages and corresponding actions. 2004/09/01 * Deleted all primary modes except for what used to be called "Posting Automation". Focus on the latter and mention other modes as exceptions or side-effects. * Changed draft outline and depth to describe specific submission steps and corresponding web pages rather than more general ideas/requirements. * Assume, for now, that Chair authorization of WG draft work must exist for WG draft to be published. This needs to be documented and perhaps relaxed to allow post-submission approvals. 2004/08/30 * Use "toolset" instead of a less accurate "interfaces" in the draft title and throughout the text (Henrik Levkowetz) * Use "post" instead of "publish"" in the draft title and throughout the text (Barbara B. Fuller and Larry Masinter) * Nits, clarifications, datapoints (Harald Tveit Alvestrand, Henrik Levkowetz, Larry Masinter, and Barbara B. Fuller for the Secretariat) 2004/08/25 * Initial revision. Author's Address Alex Rousskov The Measurement Factory EMail: rousskov@measurement-factory.com URI: http://www.measurement-factory.com/ Rousskov Expires April 6, 2005 [Page 25] Internet-Draft Draft Submission Toolset: Requirements October 2004 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society (2004). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Rousskov Expires April 6, 2005 [Page 26]