Proto Team H. Levkowetz
Internet-Draft ipUnplugged
Expires: August 30, 2004 March 2004
Protocol Pilot: Workgroup Chair Followup of AD Evaluation Comments
<draft-ietf-proto-ad-comments-pilot-01.txt>
Status of this Memo
By submitting this Internet-Draft, I certify that any applicable
patent or other IPR claims of which I am aware have been disclosed,
and any of which I become aware will be disclosed, in accordance with
RFC 3668.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress".
The list of current Internet-Drafts can be accessed at http://
www.ietf.org/ietf/1id-abstracts.txt .
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html .
This Internet-Draft will expire on August 30, 2004.
Copyright Notice
Copyright (C) The Internet Society (2004). All Rights Reserved.
Abstract
This document describes a pilot implementation of a protocol change
within the IETF. The essence of the change is to have workgroup
chairs handle the feedback of AD (Area Director) Evaluation comments
on a draft to the authors (and workgroup if necessary) and make sure
that needed draft changes are made, and the AD notified when a new
draft which resolves the comments is available.
1. Introduction
As part of the currently ongoing effort to improve the work flow
Levkowetz Expires August 30, 2004 [Page 1]
Internet-Draft WG Chair followup of AD-Comments March 2004
(particularly speed of approval) of documents, the PROTO team
[PROTO] is defining pilot projects to test possible protocol changes.
This document describes such a pilot.
The purpose of the pilot described here is to test offloading
follow-up work which an Area Director (AD) traditionally has done
after he has read through and evaluated a document submitted to the
IESG for publication. It is hoped that offloading this onto the
chair (or one of the chairs) of the workgroup which submitted the
draft will increase the speed of follow-up and the transparency of
the process, and reduce the workload of the AD to boot. The pilot
does not include offloading of follow-up for drafts which do not
originate in a workgroup.
For a discussion of the reasoning underlying piloting of process
changes, see [JULY14].
2. Pilot description
2.1 Participants
This pilot involves Area Directors of selected areas, and some or all
of the chairs for which the Area Director is Area Advisor.
2.2 Running time and pilot size
This pilot is to be run not less than 4 months, and not more than 8
months, unless early experience shows it to be clearly detrimental.
It is expected that it will be started shortly after the IETF 59
meeting, and completed in time for the results to be reported at the
IETF 60 meeting. The pilot should be run with no less than 2 and not
more than 6 ADs, and between 5 and 20 workgroups.
The running time should be chosen such that the participating ADs and
WG chairs have opportunity to get past the initial learning and
first-time execution barrier, and get some familiarity with the
process before the pilot is closed and evaluated.
2.3 Assumptions
The pilot assumes that the steps an Area Director currently (before
this pilot goes into effect) go through for an AD Evaluation are as
follows:
1. Read and evaluate the document, taking notes of issues found. It
is expected that each AD has his own style and method of
evaluating documents, but roughly the elements given in Section
3.3 of [SIRS] are probably present in the review.
Levkowetz Expires August 30, 2004 [Page 2]
Internet-Draft WG Chair followup of AD-Comments March 2004
2. Depending on the magnitude of the issues found (and other
considerations?), either
2.a) return the document to the chairs with the review,
requesting further workgroup work, and post the review to the
workgroup mailing list
2.b) send the full review to the authors, with copy to the
chairs, and ask for issues to be resolved; post a summary of the
review to the workgroup mailing list
2.c) send the full review to the authors, possibly with copy to
the chairs, and ask for nits to be fixed.
3. Follow-up, nudge and iterate until the authors (and workgroup if
required) has fixed the issues found, and submitted an updated
draft. At this point, the draft is ready for IETF last call if
it is a standards-track document (or BCP), or for placement on
telechat agenda otherwise.
2.4 Pilot Process Description
The pilot process is changed compared with the process described
above in that the responsibility for step 3 above is put squarely on
the Workgroup Chair, rather than on the Area Director. Step 2 should
preferably be modified so that the Area Director sends the AD
Evaluation review comments to the chair(s), who in turn forward them
to the authors and workgroup as appropriate. The steps are then as
follows:
1. The AD reads, evaluates and writes comments pretty much as
before. However, note that since the communication between AD
and authors is not direct, the need for clear and
well-articulated review comments is somewhat larger.
2. Depending on the magnitude of the issues found (and other
considerations?), the AD returns the full review to the chairs,
and requests either:
2.a) that further workgroup work be undertaken to put the
document into shape to be published
2.b) that authors and workgroup are informed of the issues found
and resolve them in a revised draft
Levkowetz Expires August 30, 2004 [Page 3]
Internet-Draft WG Chair followup of AD-Comments March 2004
2.c) that the authors fix nits as needed.
As covered below, the comments will be posted to the workgroup
mailing list. The comments will normally also be posted by the
AD in the ID Tracker [IDTRACKER]. Working groups that use issue
tracking should also record the issues (and eventually their
resolution) in the issue tracker.
3. If there is more than one chair, the chairs decide on which one
should be responsible for ensuring that the needed fixes are
done.
4. The chair responsible reads through the AD Evaluation comments,
making very certain that all comments are understood, so that it
is possible to follow up on them with the authors and workgroup.
If there is some uncertainty as to what is requested, this must
be resolved with the Area Director.
5. The responsible chair sends the comments to the author(s) and to
the workgroup mailing list, in order to have a permanent record
of the comments. It is recommended that the chair solicit from
the author(s) an estimate on when the fixes will be done - i.e.,
when the submission of a revised draft can be expected.
6. When incorporating the fixes in the new version of the draft, it
is strongly recommended that the revising editor keep a summary
list showing how the issues were addressed issue by issue, and
showing what the revised text is. If such a list is forwarded to
the AD with the revised draft, it will make it possible for the
AD to verify the fixes very quickly.
7. The responsible chair follows-up, nudges and iterates until the
authors (and workgroup if required) has fixed the issues found,
and submitted an updated draft. At this point, the AD is
notified of the revised draft, and provided with the summary list
of issues and resulting text changes.
8. The Area Director verifies that the issues he found during AD
Evaluation are resolved by the new version of the draft.
9. (Hopefully, that's it, but in the worst case this starts over at
1 again...)
2.5 Wrap-up
At the end of the pilot lifetime, it is expected that an evaluation
of the experienced benefits is made, using input solicited from the
Levkowetz Expires August 30, 2004 [Page 4]
Internet-Draft WG Chair followup of AD-Comments March 2004
participating Area Directors and Workgroup Chairs by means of an
email questionnaire, web-page form or something similar. The
questions are given below, in Section 2.5.2. A per-review
questionnaire is also provided in Section 2.5.1.
2.5.1 Questionnaire to be done after each individual AD Review
To be done by both WG Chair and AD.
R1. I'm submitting this questionnaire as
1. Area Director
2. Workgroup Chair
R2. Document name:
R3. WG Chair shepherding of the AD evaluation comments for this
draft speeded up the procedure:
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
R4. WG Chair shepherding of the AD evaluation comments for this
draft resulted in the comments being resolved in a satisfactory
manner:
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
R5. WG Chair shepherding of the AD evaluation comments for this
draft resulted in a more transparent process:
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
Levkowetz Expires August 30, 2004 [Page 5]
Internet-Draft WG Chair followup of AD-Comments March 2004
R6. WG Chair shepherding of the AD evaluation comments for this
draft resulted in a more well-documented process:
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
R7. The interaction with the document editors in resolving the
comments worked out well:
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
R8. - Public Comments?
R9. - Comments to IESG and PROTO-Team only?
R10. WG Chair shepherding of the AD evaluation comments for this
draft worked out well, overall:
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
R11. - Public Comments?
R12. - Comments to IESG and PROTO-Team only?
2.5.2 Questionnaire for the Pilot as a Whole
To be done by both WG Chair and AD.
X1. Document name:
X2. I clearly understood what was expected of me in this pilot.
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
Comments?
Levkowetz Expires August 30, 2004 [Page 6]
Internet-Draft WG Chair followup of AD-Comments March 2004
X3. What is your evaluation of the benefit of the procedure you've
tried out in this pilot?
1. Definitely harmful
2. Somewhat harmful
3. Mixed feelings
4. Somewhat beneficial
5. Definitely beneficial
Comments?
X4. What is your evaluation of the added effort required for the
procedure you've tried out in this pilot?
1. Major increased effort
2. Somewhat increased
3. No change
4. Somewhat decreased effort
5. Major decreased effort
Comments?
X5. Considering all factors, this procedure should be made the
normal way of handling AD evaluation comments.
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
Comments?
X6. What do you consider to be the major advantages of this
procedure change?
X7. What do you consider to be the major disadvantages of this
procedure change?
X8. How would you change the procedure to minimise the
disadvantages?
X9. Comments to the IESG and PROTO-Team only:
3. Security Considerations
This document specifies a pilot implementation of a change in IETF
procedures. It does not raise or consider any protocol-specific
security issues. When evaluating the result of the pilot, the IESG
should check if the changes has reduced the quality of security
Levkowetz Expires August 30, 2004 [Page 7]
Internet-Draft WG Chair followup of AD-Comments March 2004
review and consideration for protocols, and take this into
consideration when deciding whether the changes should be made
permanent.
4 Informative References
[RFC2026] Bradner, S., "The Internet Standards Process -- Revision
3", BCP 9, RFC 2026, October 1996.
[RFC2028] Hovey, R. and S. Bradner, "The Organizations Involved in
the IETF Standards Process", BCP 11, RFC 2028, October
1996.
[JULY14] Klensin, J. and S. Dawkins, "A model for IETF Process
Experiments", draft-klensin-process-july14-02 (work in
progress), April 2004.
[SIRS] Carpenter, B. and D. Crocker, "Careful Additional Review
of Documents (CARD)by Senior IETF Reviewers (SIRS)",
draft-carpenter-solution-sirs-01 (work in progress), June
2003.
[IDTRACKER]
"The IETF Draft Tracker", Web Application: https://
datatracker.ietf.org/.
[PROTO] "The IESG Process and Tools (PROTO) Team", Web Page:
http://psg.com/~mrw/PROTO-Team.
Author's Address
Henrik Levkowetz
ipUnplugged AB
Arenavagen 23
Stockholm S-121 28
SWEDEN
Phone: +46 708 32 16 08
EMail: henrik@levkowetz.com
Levkowetz Expires August 30, 2004 [Page 8]
Internet-Draft WG Chair followup of AD-Comments March 2004
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Disclaimer of Validity
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Copyright Statement
Copyright (C) The Internet Society (2004). This document is subject
to the rights, licenses and restrictions contained in BCP 78, and
except as set forth therein, the authors retain all their rights.
Acknowledgment
Funding for the RFC Editor function is currently provided by the
Internet Society.
Levkowetz Expires August 30, 2004 [Page 9]