INTERNET-DRAFT S. Varshavchik Expires Jan 12, 2000 Double Precision, Inc. Jul 12, 1999 Variable Envelope Return Path SMTP Extension draft-varshavchik-verp-smtpext-00.txt Status Of This Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. 1. Abstract This document describes an extension to the SMTP service [1], called Variable Envelope Return Path (VERP). The VERP extension implements a way of automatically identifying undeliverable mail recipients, even when non-delivery reports originate from mail systems that do not implement delivery status notifications as specified in [2] and [3]. 2. Introduction All E-mail software can expect to deal with undeliverable mail. [2] and [3] implement a way for undeliverable mail to be handled in a completely automatic fashion, without requiring manual intervention. For example, mailing list managers can automatically identify addresses that are no longer deliverable, and remove them from the mailing list. Although [2] and [3] are widely implemented, there are still a lot of S. Varshavchik Expires Jan 12, 2000 [Page 1] VERP SMTP Extension S. Varshavchik Jul 12, 2000 systems that do not use them. This translates into a non-trivial amount of manual work, to identify undeliverable addresses and remove them from the mailing list. Even when the percentage of undeliverable addresses starts out rather small, over time they accumulate to the point of requiring manual intervention. VERPs represent an alternative way of handling non-delivery notices. The advantage of VERPs is that they work every time, even when non- delivery notices are not in the form specified by [2]. In a VERP, the recipient address is encoded into a portion of the return path. When undeliverable mail comes back, the mail software decodes the return address and obtains the address responsible for the non- delivery notice. For example, mail sent by a mailing list manager to the address carries a return address of . The mail system for domain.com knows that all mail with the local address starting with "mlist-return-" must go to the mailing list manager. The mailing list manager takes the return address, retrieves the remaining portion of the local address, "john=example.org", and determines that the undeliverable address was . This does not rely on RFC 1894, and will work for all non-delivery notices. Unfortunately, VERPs have a known drawback when used with large mailing lists: an individual copy of each message must be sent to every individual recipient. It is no longer possible to conserve network resources by transmitting only one copy of each message, addressed to every recipient in the same domain (or a couple of messages where the number of recipients in the same domain is very large). A separate message must be sent to every recipient when VERPs are used, because the recipient's address must to be encoded as a part of the return path. This document specifies an SMTP service extension that enables mail systems to exchange message with variable envelope return paths, without transmitting one message per recipient. 3. Framework for the VERP SMTP transport extension This SMTP transport extension [1] is laid out as follows. (1) The name of the SMTP transport extension defined here is Variable Envelope Return Path. (2) The EHLO keyword associated with this extension is VERP. S. Varshavchik Expires Jan 12, 2000 [Page 2] VERP SMTP Extension S. Varshavchik Jul 12, 2000 (3) The VERP EHLO keyword takes no parameters. (4) One optional ESMTP keyword VERP is associated with the MAIL FROM command. This parameter takes no values. (5) No additional ESMTP verbs are defined by this extension. (6) The next section specifies how support for this extension affects the behavior of a server and client SMTP. 4. The VERP SMTP extension When a VERP keyword is present in the MAIL FROM command, [4], additional restrictions are imposed on the RFC 822 address [5], specified by that MAIL FROM command, and on all RFC 822 addresses in the subsequent RCPT TO commands that refer to the same message (that is, until the next DATA, RSET, or QUIT command). The term "VERP message" refers to any E-mail message whose MAIL FROM command includes the VERP keyword. The term "VERP-compliant server" refers to any E-mail server that supports the Variable Envelope Return Path SMTP extension. When a VERP keyword is present in the MAIL FROM command: (1) The address specified by the MAIL FROM verb MUST contain at least one @ character. (2) The address in every RCPT TO verb referring to the same message MUST contain at least one @ character. (3) The domain portion of the address in the MAIL FROM and RCPT TO verbs MUST be compliant with the definition of in [6]. That is, it MUST contain only letters, digits, hyphens, and periods. The domain portion is the portion of the address that follows the last @ character, 4.1 Delivery failures When a VERP-compliant server is unable to deliver a VERP message to one or more of its recipients, the VERP server MUST do one of the following: 1) Return an RFC 1891 delivery status notification to the return address, or: 2) Transmit a separate non-delivery notice for each failed recipient. The return address for each non-delivery notice MUST be the address that's formed by applying the procedure described in section 7 of this document to the return address S. Varshavchik Expires Jan 12, 2000 [Page 3] VERP SMTP Extension S. Varshavchik Jul 12, 2000 of the message and the failed recipient's address. If more than one recipient failed, a separate notice MUST be sent for each undeliverable address. 5. Final delivery A VERP-compliant server may have locally-defined conventions which records the return address in each message, for informational purposes. If this is the case, the recorded return address MUST be formed by applying the procedure described in section 7 of this document to the return address and the recipient's address. 6. Relaying When a VERP-compliant server determines that a recipient of a VERP message is not a local mailbox, and the message must be relayed to another server, the VERP-compliant server MUST: (1) If the VERP-compliant server's local policies require the return and/or recipient addresses to be rewritten, any rewritten addresses MUST have at least one @ character. (2) If the VERP-compliant server determines that the remote server is also a VERP compliant server, the VERP keyword MUST be included in the MAIL FROM command used to relay the VERP message to the remote server. (3) If the remote server is not a VERP compliant server, The VERP compliant server SHOULD send a separate copy of the VERP message for every recipient, and the return address of each message MUST be formed by applying the procedure described in section 7 of this document to the original return address, and the address of each recipient. Alternatively, the message SHOULD NOT be returned as undeliverable. If it is, the rules defined in section 4.1 MUST be applied. This also applies if the SMTP-compliant server determines that the VERP message is to be forwarded via some other protocol to a non-SMTP gateway, unless the non-SMTP protocol has equivalent features that are completely identical in function to Variable Envelope Return Path SMTP service extension (including any translations of E-mail addresses to and from the non-RFC822 format). This SMTP service extensions allows E-mail software to send a single VERP message to all addresses the same mail domain, as long as all S. Varshavchik Expires Jan 12, 2000 [Page 4] VERP SMTP Extension S. Varshavchik Jul 12, 2000 mail servers used to deliver the message support the Variable Envelope Return Path SMTP extension. When a VERP message reaches a non VERP-compliant server, a separate message with a variable envelope return path is generated for each recipient. 7. Variable envelope return path encoding This encoding method starts with one return address and one recipient address. As mentioned previously, both addresses MUST be valid RFC822 addresses, [5], and MUST contain at least one @ character. The portion of each address following the last @ character MUST be compliant with [6]. Let "sdomain" represent the portion of the return address that follows the last @ character. Let "slocal" represent the portion of the return address that precedes the last @ character. Let "rdomain" represent the portion of the recipient address that follows the last @ character. Let "rlocal" represent the portion of the recipient address that precedes the last @ character. To encode the recipient address within the envelope sender address, create an address of the following form: slocal-encodedrlocal=rdomain@sdomain Where "encodedrlocal" is formed by taking rlocal and encoding it as follows: 1) Each @, :, %, !, and + character in rlocal is replaced by a single '+' character followed by two uppercase hexadecimal characters whose value is the ASCII code of the replaced character. 2) All other characters are unchanged. Other characters MAY, but SHOULD NOT be also encoded in the same fashion. This can be represented using BNF as follows: encodedverp: slocal "-" encodedrlocal "=" rdomain "@" sdomain encodedrlocal: * (char-literal / char-encoded ) char-literal: any character valid in an RFC822 address [5], S. Varshavchik Expires Jan 12, 2000 [Page 5] VERP SMTP Extension S. Varshavchik Jul 12, 2000 except @, :, %, !, and + char-encoded: "+" hexdigit hexdigit hexdigit: ("0" / "1" / "2" / "3" / "4" / "5" / "6" / "7" / "8" / "9" / "A" / "B" / "C" / "D" / "E" / "F" ) 8. Variable envelope return path decoding Non-delivery notices for VERP messages will be sent to either the original address, , or to the VERP-encoded address, . Messages sent to will be RFC 1891-compliant delivery status notifications. These messages will be machine-readable, and the mail software will be able to identify failed addresses from the RFC 1891 delivery report. Non-delivery notices will also be sent to the VERP-encoded address, and the mail software will be able to reconstruct the failed address from the VERP-encoded address by simply reversing the steps used in encoding: 1) Extracting encodedrlocal and rdomain from the recipient address. There will be at least one = character in the encoded portion of the return address. encodedrlocal is everything up to the last = character. Everything following the last = character is rdomain. 2) Replacing all occurrences of "+" followed by two hexadecimal digits in encodedrlocal with the equivalent ASCII character. 3) Using the decoded rlocal, @, then rdomain. 9. Examples Suppose that a VERP-compliant server named "example.com" receives a message with the following SMTP conversation (for brevity, non- relevant headers have been omitted): 250 example.com ESMTP EHLO domain.com 250-example.com ESMTP 250-SIZE 250-DSN 250-VERP 250 HELP MAIL FROM: VERP SIZE=100 250 Ok RCPT TO: S. Varshavchik Expires Jan 12, 2000 [Page 6] VERP SMTP Extension S. Varshavchik Jul 12, 2000 250 Ok RCPT TO: 250 Ok RCPT TO: 250 Ok RCPT TO: 250 Ok RCPT TO: 250 Ok DATA 250 Ok From: "John" Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST) Subject: Meeting canceled. Today's 2pm meeting has been rescheduled for tomorrow, 9am, due to a scheduling conflict. . The message is delivered to the local mailbox for . The message looks like this: Return-Path: From: "John" Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST) Subject: Meeting canceled. Today's 2pm meeting has been rescheduled for tomorrow, 9am, due to a scheduling conflict. The VERP-compliant server at example.com connects to the mail server for old.example.com. old.example.com does not support the Variable Envelope Return Path extension. Therefore, old.example.com receives two messages. The SMTP conversation for the first message is as follows: 250 old.example.com ESMTP EHLO example.com 250-old.example.com ESMTP 250-SIZE 250-DSN 250 HELP MAIL FROM: 250 Ok RCPT TO: 250 Ok DATA 250 Ok S. Varshavchik Expires Jan 12, 2000 [Page 7] VERP SMTP Extension S. Varshavchik Jul 12, 2000 From: "John" Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST) Subject: Meeting canceled. Today's 2pm meeting has been rescheduled for tomorrow, 9am, due to a scheduling conflict. . The SMTP conversation for the second message is as follows: MAIL FROM: 250 Ok RCPT TO: 250 Ok DATA 250 Ok From: "John" Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST) Subject: Meeting canceled. Today's 2pm meeting has been rescheduled for tomorrow, 9am, due to a scheduling conflict. . example.com connects to new.example.com and determines that new.example.com runs a modern ESMTP server that supports the VERP keyword. The SMTP conversation then goes like this: 250 new.example.com ESMTP EHLO example.com 250-new.example.com ESMTP 250-SIZE 250-DSN 250-VERP 250 HELP MAIL FROM: VERP SIZE=100 250 Ok RCPT TO: 250 Ok RCPT TO: 250 Ok DATA 250 Ok From: "John" Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST) Subject: Meeting canceled. Today's 2pm meeting has been rescheduled for tomorrow, 9am, due S. Varshavchik Expires Jan 12, 2000 [Page 8] VERP SMTP Extension S. Varshavchik Jul 12, 2000 to a scheduling conflict. . 10. Security concerns All the usual security considerations applicable to SMTP are also applicable to this extension. Relay of VERP messages to non-VERP servers requires a single message with many recipients to be exploded into many messages with one recipient. In all cases, however, there will never be any additional overhead beyond the resources that are required when variable envelope return paths are manually implemented by the mail sender, instead of using the VERP SMTP extension. Mail systems which support the VERP extension SHOULD have adequate security measures, including blocks against unauthorized access and relaying. S. Varshavchik Expires Jan 12, 2000 [Page 9] VERP SMTP Extension S. Varshavchik Jul 12, 2000 11. References [1] Klensin, J., Freed, N., Rose, M., Stefferud, E., Crocker, D. "SMTP Service Extensions", RFC 1425, United Nations University, Innosoft International, Inc., Dover Beach Consulting, Inc., Network Management Associates, Inc., The Branch Office, February 1993 [2] Moore, K., and G. Vaudreuil, "An Extensible Message Format for Delivery Status Notifications", RFC 1894, University of Tennessee, Octel Network Services, January 1996. [3] Moore, K. "SMTP Service Extension for Delivery Status Notifications", RFC 1891, University of Tennessee, January 1996. [4] Postel, J., "Simple Mail Transfer Protocol", STD 10, RFC 821, USC/Information Sciences Institute, August 1982. [5] Crocker, D., "Standard for the Format of ARPA Internet Text Messages", STD 11, RFC 822, UDEL, August 1982. [6] Mockapetris, P., "Domain Names - Implementation and Specification", RFC 1035, ISI, November 1987 12. Author's address Sam Varshavchik Double Precision, Inc. PO Box 668 Greenwood Lake, NY 10925 S. Varshavchik Expires Jan 12, 2000 [Page 10]