Network Working Group John C. Klensin Internet Draft Ned Freed Keith Moore Jeroen Houttuin Requirements for Internet Mail Transport Agents January 24, 1995 Status of this Memo This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months. Internet-Drafts may be updated, replaced, or obsoleted by other documents at any time. It is not appropriate to use Internet-Drafts as reference material or to cite them other than as a "working draft" or "work in progress". To learn the current status of any Internet-Draft, please check the 1id-abstracts.txt listing contained in the Internet-Drafts Shadow Directories on ds.internic.net (US East Coast), nic.nordu.net (Europe), ftp.isi.edu (US West Coast), or munnari.oz.au (Pacific Rim). 1. Abstract This document reflects apparent ambiguities in RFC 821 that remain after the clarifications in RFC 1123. It also includes some material from RFC 1123 that appeared to need amplification. These have been identified in multiple ways, mostly by tracking flaming on the header-people list and problems of unusual readings or interpretations that have turned up as the SMTP extensions have been deployed. It is important to note that everything here is in response to some identified confusion or bad behavior, not just paranoia. Although SMTP was designed as a mail transport and delivery protocol, this specification also contains information that is important to its use as a "mail posting" protocol, as recommended for POP3 [RFC-POP3] and IMAP4 [RFC-IMAP4]. Except when the historical terminology is necessary for clarity, this document uses the current "client" and "server" terminology to identify the sending and receiving SMTP processes, respectively. A companion document discusses mail bodies and formats: RFC-822, MIME, and their relationship. 2. Clarifications, Corrections, and Requirements for SMTP implementations. 2.1. Eight bit transmission. The unextended SMTP service provides seven bit transport only. SMTP clients MUST NOT transmit messages with information in the high-order bit of octets. If such messages are transmitted in violation of this rule, receiving SMTP servers MAY clear the high-order bit or reject the message as invalid. Reference: RFC821, page 3, next-to-last paragraph. Eight-bit transmission MAY be requested of the server by the client using extended SMTP facilities and the "8BITMIME" extension. 2.2. VRFY and EXPN issues 2.2.1 VRFY failure conditions When a name that is the argument to VRFY could identify more than one mailbox, the server MAY either note the ambiguity or identify the alternatives. In other words, either of the following are legitimate response to VRFY: 553 User ambiguous or 553- Ambiguous; Possibilities are 553-Joe Smith 553-Harry Smith 553 Melvin Smith Under normal circumstances a client receiving a 533 reply would be expected to expose the result to the user. Reference: RFC821, page 8 and example on page 9. 2.2.2. VRFY normal response. When normal (2yz or 551) responses are returned from a VRFY or EXPN request, the reply MUST include the mailbox name, e.g., "" (where "bar" is a fully qualified domain name) must appear in the syntax. EXPN and VRFY MUST return only legal domain addresses that are usable in SMTP RCPT commands. 2.2.3 Meaning of VRFY or EXPN success response. A server MUST NOT return a 220 code in response to a VRFY or EXPN command unless it has actually verified the address. In particular, a server MUST NOT return 220 if all it has done is to verify that the syntax given is valid. In that case 502 (Command not implemented) or 500 (Syntax error, command unrecognized) SHOULD be returned (note that implementation of VRFY is required by RFC1123 and EXPN is strongly recommended; this specification does not change that requirement and, hence, implementations that return 500 or 502 for VRFY are not in compliance with the specification). 2.2.4 VRFY, EXPN, and security. As RFC-1123 points out, individual sites may want to disable one or both of VRFY or EXPN for security reasons. As a corollary to the above, implementations that permit this MUST NOT appear to have verified addresses that are not, in fact, verified. Implementations disabling these commands for security reasons SHOULD return a 252 response, rather than a code that could be confused with successful or unsuccessful verification. Returning a 250 reply code with the address listed in the VRFY command after having checked it for syntax only violates this rule. Of course, an implementation that "supports" VRFY by always returning 550 whether or not the address is valid is equally not in conformance. 2.2.5. Forwarding responses and other addresses. RFC-821 permits an SMTP server to return a 551 reply (User not local; please try...) to indicate that the client should send the mail elsewhere. Such a "forwarding address" SHOULD be stable and MUST be usable from the worldwide Internet (e.g., it must represent a mailbox at a fully qualified domain name). Similarly, a 251 reply (User not local; will forward...) MAY be returned to inform the client that the message has been (or will be) forwarded. SMTP client software that is designed to take advantage of 251 or 551 replies should note that, in practice, mail forwarding is used for both "change of address" (both temporary and permanent) and "centralized post office" purposes and that it is impossible to differentiate between these two without additional information. Return of 251 or 551 replies is never mandatory. 2.2.6 Semantics and applications of EXPN. While EXPN is often very useful in debugging and understanding problems with mailing lists and multiple-target-address aliases, some systems have attempted to use source expansion of mailing lists as a means of eliminating duplicates. The propagation of aliasing systems with mail on the Internet--both for hosts (typically with MX and CNAME DNS records) and for mailboxes (various types of local host aliases) has made it nearly impossible for these strategies to work, and mail systems SHOULD NOT attempt them. 2.3. Typographical errors. 2.3.1. The next to last line on page 11 of RFC821 contains a typographical error. The line should end "...mail is delivered to the" 2.3.2. On page 18, third line from the end: "supporting be mail" -> "supporting a mail" 2.4. Termination of connections. A server-SMTP MUST NOT intentionally close the connection except: o After receiving a QUIT connand and responding with a 221 reply. o After detecting the need to shutdown the SMTP service and returning a 451 reply to any command. In particular, a server that closes connections in response to commands that are not understood is in violation of this specification. A server-SMTP which is forcibly shut down via external means SHOULD attempt to send a line containing 451 response code to the client-SMTP before exiting. The client-SMTP will normally read the 451 response code after sending its next command. Reference: RFC821, page 13 and the discussion of QUIT on RFC821, page 26. 2.5. SMTP server greeting message. SMTP Servers SHOULD include identification of their software and version information in the connection greeting reply after the 220 code. This practice permits much more efficient isolation and repair of any problems. While some systems also identify their contact point for mail problems, this is not a substitute for maintaining the required Postmaster address. Reference: RFC821, page 19. 2.6. The Return-path header. RFC1123 requires that the SMTP server system making final delivery create a Return-Path header that contains the reverse path. It is sometimes difficult for an SMTP server to determine whether or not it is making final delivery since forwarding or other operations may occur after the message is delivered. However any further (forwarding, gateway, or relay) systems MAY remove the return path and rebuild the MAIL FROM command as needed to ensure that exactly one such line appears in a delivered message. An SMTP server MUST NOT add a return path to a message header that already contains one without removing the original Return-path header. The primary intent for the Return-path is that it designate the address to which messages indicating non-delivery or other mail system failures at to be sent. For this to be unambigious, exactly one return path should be present when the message is delivered. Systems using RFC822 syntax with non-SMTP transports SHOULD preserve the intent. Text in RFC 822 that appears to contradict the use of Return-path (or the envelope MAIL FROM address) as the destination of error messages is overridden by it. In particular, (i) a gateway from SMTP->elsewhere SHOULD insert a return-path header, unless it is known that the "elsewhere" transport also uses Internet domain addresses and maintains the envelope sender address separately. (ii) a gateway from elsewhere->SMTP SHOULD delete any return-path header present in the message, and either copy that information to the SMTP envelope or combine it with information present in the envelope of the other transport system to construct the MAIL FROM part of the SMTP envelope. 2.7. Order of commands. The text in RFC821 that describes the order in which commands may be issued is misleading and has led to some confusion. After adjustment for the SMTP extension mechanism, the explanation should read: There are restrictions on the order in which these commands may be used. A session that is to contain mail transactions MUST first be initialized by the use of the HELO or EHLO command although an SMTP server SHOULD accept commands for non-mail transactions (e.g., VRFY or EXPN) without this initialization. HELO or EHLO commands MAY also be issued by a client later in the session. If they are issued later, the SMTP server MUST clear all buffers and state, i.e., the sequence of RSET followed immediately by HELO is redundant, but not harmful (except, obviously, to performance). If the HELO or EHLO commands are not acceptable to the server-SMTP, 501, 500, or 502 failure replies MUST be returned as appropriate. The server-SMTP must stay in the same state after transmitting these replies as it was in before the HELO or EHLO were received. RFC 1123 contains a discussion of arguments to HELO and conditions under which the HELO command can be rejected. In particular, HELO MUST NOT be rejected because the client's putative name does not match some criteria established by the server (e.g., verification of reverse DNS mapping). The NOOP, HELP, EXPN, VRFY, and RSET commands can be used at any time during a session, or without previously initializing a session. SMTP servers SHOULD process these normally (i.e., not return a 503 code) even if no HELO or EHLO command has yet been received; clients SHOULD open a session with HELO or EHLO before sending these commands. If the above rules are followed, the example in RFC821 that shows "550 access denied to you" in response to an EXPN command is essentially meaningless unless a HELO or EHLO command preceeds the EXPN. The MAIL, SEND, SOML, or SAML commands begin a mail transaction. Once started a mail transaction consists of one of the transaction beginning commands, one or more RCPT commands, and a DATA command, in that order. A mail transaction may be aborted by the RSET command. There may be zero or more transactions in a session. If the transaction beginning command argument is not acceptable a 501 failure reply MUST be returned and the SMTP server must stay in the same state. If the commands in a transaction are out of order to the degree that they cannot be processed by the server a 503 failure reply MUST be returned and the SMTP server must stay in the same state. The last command in a session must be the QUIT command. The QUIT command can not be used at any other time in a session, but may be used by the client SMTP to request connection-closing even if no session-opening command has been sent and accepted. Reference: RFC821, top of page 27, down to the beginning of 4.1.2. 2.8. Private-use commands Commands starting in "X" may be used by bilateral agreement between the client (sending) and server (receiving) SMTPs. An SMTP server that does not recognize such a command is expected to reply with "500 Command not recognized". An extended SMTP server MAY list the feature names associated with these private commands in the response to the EHLO command. As specified in the SMTP Service Extensions RFC [RFC-1651], commands sent or accepted by SMTP systems that do not start with "X" MUST be documented in published RFCs and be at least candidates for standardization. 2.9. List of commands by category. The first line of page 29 of RFC821 should read: "...are the SMTP and Extended SMTP commands: EHLO". 2.10. Form of mailbox local-parts. For maximum interoperability, a host that expects to receive mail SHOULD avoid defining mailboxes where the contains (or requires) either a quoted-string or backslash quoting characters. Systems MUST NOT define mailboxes in such a way as to require the use of non-ASCII characters (octets with the high order bit set to one). These characters MUST NOT be used in MAIL FROM or RCPT TO commands or other commands that require mailbox names. Reference: RFC821, page 31. 2.11. Format and interpretation of date fields 2.11.1 Four digit years RFC1123 expands the definition of "" in dates to permit and encourage 4-digit years. SMTP servers SHOULD use four-digit years whenever dates are specified. SMTP servers MUST NOT reject mail on the basis of its containing four-digit years. Reference: RFC821, page 32; RFC 1123. 2.11.2 Time zones The use of the term "default" in the RFC 821 definition does not imply that zone information is optional. A time zone designator MUST be supplied. As the Internet grows, comparability of Received fields is important for detecting problems, especially slow relays. SMTP servers that create Received fields SHOULD use explicit offsets (e.g., -0800), rather than names of any type. If it is desired to also use a time zone name, it should be included in a commment. 2.11.3 Time semantics The Date specified in the RFC 822 header is normally presumed to be the date and time at which the user completed composition of the message. As such, it SHOULD be supplied by the message composing system. An SMTP server MUST NOT alter this date if it appears in a message received by it for transmission or relaying. In environments in which mail composition agents queue mail locally before transmitting it to a sending SMTP server (or the other component of a split-MUA arrangement), the RFC 822 date SHOULD reflect the time at which message composition was completed, not the time of transfer to the MTA. In environments in which Mail User Agent/ Mail Transfer Agent distinctions are made, SMTP servers normally SHOULD insert a Received field to reflect the receipt of the message for transmission from the MUA. 2.12. Domains RFC-821 and RFC-822 were developed before the domain name system had fully stabilized. The domain syntax for both (using RFC-822's syntax) is: domain-ref = sub-domain *("." sub-domain) sub-domain = let-dig *(ldh-str) domain-literal = "[" IP-address-literal "]" IP-address-literal = snum 3*("." snum) snum = one, two, or three digits representing a decimal integer value in the range 0 through 255 let-dig = Alpha / Digit ldh-str = *( Alpha / Digit / "-" ) 1*(let-dig) (the '#' indicates a comma separated list.) Alpha = ASCII character in the range A-Z or a-z. As specified in RFC1035, case is not significant in domain strings. Digit = 0 - 9 Note that the IP-address-literal syntax above specifies four or more of these numbers to accomodate IPv6. 2.13. Reply codes. 2.13.1. Reply codes after DATA and the subsequent CRLF.CRLF. When a SMTP server returns a positive completion status (2yz code) after the DATA command is completed with CRLF.CRLF, it accepts responsibity for: + delivering the message (if the recipient mailbox exists), or + if attempts to deliver the message fail due to transient conditions, retrying delivery some reasonable number of times at intervals as specified in RFC 1123, or + if attempts to deliver the message fail due to permanent conditions, or if repeated attempts to deliver the message fail due to transient conditions, returning appropriate notification to the sender of the original message (using the address in the SMTP MAIL FROM command). When a SMTP server returns a transient error completion status (4yz) code after the DATA command is completed with CRLF.CRLF, it MUST NOT make any further attempt to deliver that message. The client-SMTP retains responsibility for delivery of that message. The sending user should be able to interpret the return of a transient or permanent failure status as a non-delivery indication. 2.13.2. Reply code 502 Questions have been raised as to when reply code 502 (Command not implemented) should be returned in preference to other codes. 502 SHOULD be used when the command is actually recognized by the SMTP server, but not implemented. If the command is not recognized, code 500 SHOULD be returned. Extended SMTP systems MUST NOT list capabilities in response to EHLO for which they will return 502 (or 500) replies. 2.14. Generating SMTP commands from RFC822 headers Some systems use RFC-822 headers (only) in a mail submission protocol, or otherwise generate smtp commands from RFC 822 headers when such a message is handed to an MTA from a UA. While the MTA-UA protocol is a private matter, not covered by any Internet Standard, there are problems with this approach. For example, there have been repeated problems with proper handling of "bcc" copies and redistribution lists when information that conceptually belongs to a mail envelopes is not separated early in processing from header information (and kept separate). It is recommended that the UA provide its initial MTA with an envelope separate from the message itself. However, if the envelope is not supplied, SMTP commands should be generated as follows: (i) each recipient addresses from a TO, CC, or BCC header field should be copied to a RCPT command (generating multiple message copies if that is required for queuing or delivery). This includes any addresses listed in a rfc822 "group". Any BCC fields should then be removed from the headers. Once this process is completed, the remaining headers should be checked to verify that at least one To:, Cc:, or Bcc: header remains. If none do, then a bcc: header with no additional information MUST be inserted. (ii) the return address in the MAIL command should be derived from the system's identity for the submitting (local) user. That return address should also be copied to the Sender header field if it is different from the address in the From header field. (Any Sender field that was already there should be removed.) Systems may provide a way for submitters to override the envelope return address, but may want to restrict its use to priviliged users. (This will not prevent mail forgery, but may lessen its incidence.) (iii) If a Return-path header is present in the message to be sent, the MTA MUST reject the message. A submission protocol based on RFC-822 information alone MUST NOT be used to gateway a message from a foreign (non-SMTP) mail system into an SMTP environment. Attempts to do so have repeatedly caused mail loops and other behavior adverse to the proper functioning of the Internet mail environment. In gateway situations, the SMTP recipient addresses (and, in general, the MAIL FROM address) MUST be derived from the foreign system's envelope or external information. [[Note in draft: I changed my mind about Keith's suggestion here and put it back in, although significantly rephrased.]] 2.15 Other gateway issues. In general, gateways between the Internet and other mail systems SHOULD attempt to preserve any layering semantics across the boundaries between the two mail systems involved. Gateway- translation approaches that attempt to take shortcuts by mapping, e.g., envelope information from one system to the message headers or body of another have generally proven to be inadequate in important ways. Systems translating between environments that do not support both envelopes and headers and Internet mail must be written with the understanding that some information loss is almost inevitable. 2.16. Mail forging and "spoofing". As knowledge of Internet mail increases, so does the knowledge that SMTP mail inherently cannot be authenticated, or integrity checks provided, at the transport level. Real security lies only in end-to-end methods involving the message bodies. A corollary to this is that efforts to make it more difficult for users to set envelope MAIL FROM and header "From" fields to point to valid addresses other than their own are largely misguided: they do not prevent any would-be mail spoofer from doing so, and do frustrate legitimate applications in which mail is sent by one user on behalf of another or in which error replies should be directed to a special address. On the other hand, systems that provide convenient ways for users to alter these fields on a per-message basis should attempt to establish a primary and permanent mailbox address for the user so that Sender fields can be generated correctly. 2.17. Relays, forwarding, resending, and header interpretation RFC-822 is not specific about the semantics of "resent-" fields, and additional issues about them are addressed in the companion document to this one on mail bodies. However, there are important issues for MTAs engaged in relays and forwarding as well. 2.17.1. Address rewrites. A mail forwarder that is rewriting a message address that belongs to a single user (e.g., instantiating a single alias) MAY add resent- fields to identify that activity and the new recipients. If it does not, it SHOULD add a Received header to reflect what it has done (in addition to, or as part of, the Received header that it MUST add when it handles the message). In particular, an SMTP server that returns a 251 response code SHOULD add Resent- fields, a Received header that specifically identifies the action taken, or both. 2.17.2. Fields to be added. When Resent- fields are added to a message, at least one Resent-to or Resent-cc field, together with a Resent-date, SHOULD be provided. Resent-From MAY be provided and maybe be useful if the information isn't obvious from context. 2.17.3 Valid messages. In general, SMTP servers SHOULD avoid rejecting (bouncing) messages because they believe the format of the bodies of those messages are incorrect. They MUST NOT reject messages on the basis of trying to match numbers of Resent- fields. 2.18. "Blind" copies. Addresses may appear in the RCPT TO commands to an SMTP server that do not appear in the message headers for a number of reasons. The two most common of these involve the use of a mailing address as a "list exploder" -- a single address that resolves into multiple addresses -- and the appearance of "blind copies". In order to avoid defeating some of the purpose of these mechanisms, SMTP clients and servers SHOULD NOT copy the RCPT TO command arguments into the headers, even as informational or private-extension headers. Since this rule is often violated in practice, and cannot be enforced, sending SMTP systems that are aware of "bcc" use MAY find it helpful to send each blind copy as a separate message transaction containing only a single RCPT TO command. More generally, while there are often similarities, there is no inherent relationship between either "reverse" (MAIL FROM, SAML FROM, etc.) or "forward" (RCPT TO) addresses in the SMTP transaction ("envelope") and the addresses in the headers. Receiving systems SHOULD NOT attempt to deduce such relationships and use them to alter the headers of the message for delivery. The popular "Apparently-to" header is a violation of this principle and SHOULD NOT be used. 3. Security Considerations SMTP mail is inherently insecure in that it is feasible for even fairly casual users to negotiate directly with receiving and relaying SMTP servers and create messages that will trick a naive recipient into believing that they came from somewhere else. Constructing such a message so that the "spoofed" behavior cannot be detected by an expert is somewhat more difficult, but not sufficiently so as to be a deterrent to someone who is determined and knowledgeable. This specification does not address the security issues associated with SMTP other than to advocate that useful functionality not be disabled in the hope of providing some small margin of protection against an ignorant user who is trying to fake mail. 4. References [RFC-821] J. Postel, "Simple Mail Transfer Protocol", 08/01/1982 [RFC-1123] R. Braden, "Requirements for Internet hosts - application and support", 10/01/1989 [RFC-1651] J. Klensin, N. Freed, M. Rose, E. Stefferud, D. Crocker, "SMTP Service Extensions", 07/18/1994. 5. Authors' Addresses John C. Klensin MCI Data and Information Services 2100 Reston Parkway Reston, VA 22091 USA Email: Klensin@mail1.reston.mci.net Phone: +1 703 715 7361 Fax: +1 703 715 7436 Ned Freed Innosoft International, Inc. 1050 East Garvey Avenue South West Covina, CA 91790 USA Email: ned@innosoft.com Phone: +1 818 919 3600 Fax: +1 818 919 3614 Jeroen Houttuin RARE Secretariat Singel 466-468 NL-1017 AW Amsterdam Europe Tel +31 20 6391131 Fax +31 20 6393289 Email: X.400 /S=houttuin/O=rare/PRMD=surf/ADMD=400net/C=nl/ RFC 822 houttuin@rare.nl Keith Moore Computer Science Dept. University of Tennessee 107 Ayres Hall Knoxville, TN 37996-1301 USA EMail: moore@cs.utk.edu 6. Acknowledgements The authors, who are co-initiators of this project, would like to express appreciation to Dave Crocker and Marshall Rose for comments on an intermediate draft, to participants in the header-people mailing list (and the former comp.mail.headers newsgroup) who raised some of the issues discussed in this specification and helped work out a preliminary consensus on others, and to the Bob Braden as Editor and the contributors to RFC 1123 which laid the foundations for the present work.