Network Working Group M. Schmeing Internet-Draft FGAN Expires: February 24, 2007 J. Brendecke K. Carlberg G11 August 23, 2006 SMTP Service Extension for Priority Message Handling draft-schmeing-smtp-priorities-05.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on February 24, 2007. Copyright Notice Copyright (C) The Internet Society (2006). Abstract This memo defines an extension to the SMTP (Simple Mail Transfer Protocol) service whereby messages are sent with a priority to achieve a certain order in which the messages are transferred by an MTA (Message Transfer Agent). This priority or precedence order is used instead of the first-come-first-serve rule. This extension is Schmeing, et al. Expires February 24, 2007 [Page 1] Internet-Draft Priority Extension for SMTP August 2006 not to be confused with "Importance of a Message" which is widely deployed using an email header such as Importance or even Priority or Precedence with common values of HIGH, NORMAL and LOW. Importance of a Message does not affect the priority of the transport itself in any way. Nevertheless, there may be policy defined relations between priorities and importance indicators. This extension uses the term priority in the meaning of expedited treatment of a message by the server according to its priority. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Terminology & background . . . . . . . . . . . . . . . . . . . 5 4. Framework for the Priority Extension . . . . . . . . . . . . . 6 5. The Priority Service Extension . . . . . . . . . . . . . . . . 8 5.1. Expedited Transfer . . . . . . . . . . . . . . . . . . . . 8 5.1.1. Probability . . . . . . . . . . . . . . . . . . . . . 9 5.1.2. Preemption of sessions or transactions . . . . . . . . 9 5.1.3. Handling of emails with multiple recipients . . . . . 10 5.2. Timely Delivery . . . . . . . . . . . . . . . . . . . . . 10 5.3. Resolving conflicts for the sending order of messages with the same priority . . . . . . . . . . . . . . . . . . 10 5.4. Size Limitation . . . . . . . . . . . . . . . . . . . . . 11 5.5. Further Constraints . . . . . . . . . . . . . . . . . . . 12 6. Recipient Dependent Priority . . . . . . . . . . . . . . . . . 13 6.1. Maillists, Aliases and Forwarding . . . . . . . . . . . . 13 7. NameSpaces . . . . . . . . . . . . . . . . . . . . . . . . . . 14 7.1. Conflicts with external constraints . . . . . . . . . . . 15 7.2. Handling Non-priority Servers and Mismatching Policies . . 16 7.3. Recipients without priorities . . . . . . . . . . . . . . 17 7.4. Invalid priority levels . . . . . . . . . . . . . . . . . 17 7.5. Mappings to other NameSpaces . . . . . . . . . . . . . . . 17 8. Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 9. Security Considerations . . . . . . . . . . . . . . . . . . . 20 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 11. List of abbreviations . . . . . . . . . . . . . . . . . . . . 22 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 23 12.1. Informative Reference . . . . . . . . . . . . . . . . . . 23 12.2. Normative References . . . . . . . . . . . . . . . . . . . 23 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 25 Intellectual Property and Copyright Statements . . . . . . . . . . 26 Schmeing, et al. Expires February 24, 2007 [Page 2] Internet-Draft Priority Extension for SMTP August 2006 1. Introduction Prioritization is an issue that gains attention when resources are scarce and sets of users (and their data) are considered more important than others. Military Message Handling Systems (MMHS) such as AUTODIN and the Defense Messaging System (DMS), have requirements to prioritize their traffic to help ensure that important data is given preferential treatment over what would normally be viewed as best effort. Deployments of MMHS typically include the ability to preempt or displace less important data traffic. This displacement can be in the form of dropped packets, removal of reservations or termination of end-to-end sessions. Outside MMHS systems such as GETS rely on increasing the probability of obtaining resources or successfully forwarding downstream packets instead of preempting resources. In either case, the critical feature incumbent in both systems is an ability to prioritize data in anticipation of resource contention. The Simple Mail Transfer Protocol [7] (SMTP) is one of the most popular applications used over the Internet by today's general public. The examinations documented in [1] show that SMTP, including some SMTP extensions, also provides most of the features requested by the military (a user community whose requirements are more stringent than the general public). But one of the most important requirements of MMHS not provided by SMTP is priority or precedence in handling between Message Transfer Agents (MTA). This document defines an SMTP Service Extension offering such a service allowing Mail Transfers Agents to determine which emails require expedited handling. Schmeing, et al. Expires February 24, 2007 [Page 3] Internet-Draft Priority Extension for SMTP August 2006 2. Terminology The key words "REQUIRED", "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" in this document are to be interpreted as described in "Key words for use in RFCs to Indicate Requirement Levels" [4]. The terms SMTP client and SMTP server are used with the same definition as given in section 2.3.2 of RFC 2821 [7]: o The SMTP client is the (computer) process initiating an SMTP session and controlling it issuing commands to the server. o The SMTP server is the process waiting for clients to connect and executing the commands from the client. As this document is concerned with SMTP only, the terms client and server may as well be used without the SMTP prefix. An email is waiting to be sent, if it resides in the queue of an MTA and can be send to the next hop or delivered to its final recipient(s) once available resources at the sending MTA allow this. Emails having their processing delayed for some reason are NOT waiting to be sent during this delay. The most important reason for emails to be delayed is a transient error response from the next MTA to which the email must be transferred. An email is ready to be sent, if it is waiting to be sent and either no emails with higher priority are waiting to be sent or available resources allow to send more emails in parallel than the number of emails with higher priorities that are waiting to be sent. Note that an email may be ready to be sent but the transfer or delivery process can not yet be initiated, because the number of emails ready to be sent exceeds the number of emails that can be processed in parallel. The policy defining the use of priorities within a given NameSpace is called "NameSpace Policy". A NameSpace is a name (or label) for a finite and ordered set of priorities. In some instances, this memo or a NameSpace Policy may allow additions to the definitions of the NameSpace Policy. These additions are called "local policy". Schmeing, et al. Expires February 24, 2007 [Page 4] Internet-Draft Priority Extension for SMTP August 2006 3. Terminology & background RFC 3487 [15] articulates a set of requirements for prioritizing access to services and resources using the Session Initiation Protocol (SIP). As in the case of SMTP, SIP has a priority field used to convey the importance of a session (or message, in the case of SMTP) to the end-user. However, the value within the field was not meant for intermediate nodes. From these requirements, the SIP working group defined a new optional Resource Priority field in RFC 4412 [16] for the SIP protocol. The new field was divided into two distinct parts. The first is Value subfield, which contains the priority associated with the SIP message. The second subfield is the NameSpace, which identifies a set or family of 1 or more priorities. The strength in this approach is that support for the prioritization feature is not bound to a one-size-fits-all design. As an example, the DoD/NATO community has an existing set of 5 priorities used for its messaging system. Other communities of interest may have sets that are smaller or larger than that currently used by DoD/NATO. Underneath the SIP layer, and its Resource Priority extension, on- going work is being done by the NSIS working group to define a QSPEC that includes fields correlating to SIP's NameSpace/Value tuple. In addition, the TSV Working Group is in the process of defining an extension to RSVP that defines a priority policy element correlating to the NameSpace/Value tuple. Both of these efforts are aimed at facilitating underlying signaling of network elements to obtain resources based on the priority set at the application layer. Schmeing, et al. Expires February 24, 2007 [Page 5] Internet-Draft Priority Extension for SMTP August 2006 4. Framework for the Priority Extension o The textual name of this extension is "Priority Message Handling". o The EHLO-Keyword is PRIORITY. o One mandatory parameter is used with this keyword. This parameter is a list of NameSpaces supported by the server. The syntax for this value is described below. The "token-nodot" production is copied from RFC 3265 [10]. namespace-list = namespace * (COMMA namespace) namespace = token-nodot token-nodot = 1*(alphanum / "-" / "!" / "%" / "*" / " " / "+" / "`" / "'" / "~") A NameSpace is a name of an ordered set of priorities with attached policy. o No new SMTP verbs are defined. o One optional parameter using the keyword "PRIORITY" is added to the RCPT TO command. The value associated with this parameter is in alpha-numeric form. The syntax of the value is : priority-arg = "PRIORITY" EQUAL value value = namespace "." priority priority = token-nodot o Two examples of PRIORITY parameters are shown below: PRIORITY=Example.1 PRIORITY=AnotherExample.Flash o The 'value' parameter in the 'PRIORITY' extension indicates the concatenated tuple of the 'namespace', the 'priority' within that NameSpace and the "." character separating the two. An entry in a 'namespace' must be unique from all other NameSpaces. Entries in a 'priority' must be unique within a NameSpace, but they may be the same as those in other NameSpaces. o The maximum length of a RCPT TO command line is increased by 9 characters plus the length of the 'value-list' by the possible Schmeing, et al. Expires February 24, 2007 [Page 6] Internet-Draft Priority Extension for SMTP August 2006 addition of the PRIORITY keyword and value. o As one of the most likely instances to assign transport priorities is the submitting party of an email (i.e. the originator or sender), this extension is valid for message submission according to RFC 2476 [6]. Schmeing, et al. Expires February 24, 2007 [Page 7] Internet-Draft Priority Extension for SMTP August 2006 5. The Priority Service Extension The priority of a message expresses itself in the order they are transfered from the client to the server. This is largely independent from the order in which they where received by the server. The Priority Service Extension increases the possibilities of SMTP service extensions such as Deliver-By [8] and Delivery Status Notification [11][14][13][12] or (non-standard) header fields such as Importance. In military scenarios, priorities usually come with additional associated constraints. Examples are limitations of the message size or the allowed delivery time. 5.1. Expedited Transfer The main service provided by the Priority Message Handling SMTP Service Extension is expedited transfer of emails with a higher priority. Therefore a client that has more than one email to send at a given time MUST send those with a higher priority before those with a lower one. Priorities are assigned on a per-recipient basis. The priority of an email is the highest priority assigned to any of the yet unprocessed recipients. Thus the priority of the email may vary while it is processed by an MTA or UA and may be different at different MTAs. The priority of any given recipient is nevertheless constant. As a default policy, emails with higher priority waiting to be sent by a client MUST NOT initiate transactions for emails with lower priorites. It MAY however send an email with highest priority even to recipients with lower priorities if this can be done within the same transaction as the higher priorities. Policy associated with a specific NameSpace may override this default policy; an example being the use of a weighted round robin initiation of transactions in order to prevent a starving of resources from lower priorities. The handling of messages with the same priority level is described in Section 5.3. Especially in networks with limited available data rate the actual transfer over the network may create a significant portion of the overall delivery time. Besides the actions taken at the application level it can thus be important to deploy priority or precedence mechanisms offered by the network itself to ensure timely delivery of the emails. One example would be the use of RSVP and the work-in- progress effort extension to RSVP that prioritizes reservations. If such services are available, it is a matter of NameSpace Policy to Schmeing, et al. Expires February 24, 2007 [Page 8] Internet-Draft Priority Extension for SMTP August 2006 decide whether to use them. The mapping from the priorities defined at the SMTP level to those offered by the underlying network is as well to be defined in the NameSpace Policy. If the NameSpace Policy does not regulate the use of network priorities, their use can be defined by local policy Most current SMTP MTAs are capable of handling several inbound and outbound connections at once. With this feature it should be possible to start additional outbound connections for emails with higher priorities even if there are already several connections running for other emails. The manner in which expedited transfer can be accomplished is divided into two approaches. One is probability, the other involves preemption, both of which are described in more detail below. 5.1.1. Probability As the name suggests, probabiblity involves increasing the chances of obtaining resources without adversely affecting previously established connections. One example would involve requesting resources set aside for specific priority levels. If these additional resources are exhausted, then the desired connection is denied. Queues, new timers, or combinations thereof can be used to facilitate the higher priority requests, but the key is that mechanisms focus on increasing the probability of message transfer. 5.1.2. Preemption of sessions or transactions Preemption is binary type of action and focusses only on a comparision of priorities to determine if previously established transactions must be displaced in favor of higher priority requests. If no additional connection is possible, a client MUST abort a running session for emails with lower priority no later than directly after the current transaction. The client even MAY interrupt an active transaction and SHOULD do this, if otherwise constraints such as delivery time would be violated for the email with higher priority. This preliminary termination of sessions or transactions is called preemption. If preemption of running transactions occurs, the client MUST preempt the lowest number of transactions sufficient to process the higher priority emails. It must chose the transactions for the emails with the lowest priorities currently processed. NameSpace Policies and local policies MAY add additional criteria to choose the transaction to be preempted in the case that more transactions than required would qualify for preemption due to the priority of the emails. Regulations from local policy MUST NOT be applied before NameSpace Policy and MUST NOT conflict with NameSpace policy settings. If Schmeing, et al. Expires February 24, 2007 [Page 9] Internet-Draft Priority Extension for SMTP August 2006 after applying the priority test and all criteria defined by applicable NameSpace Policy and the local policy still the number of transactions to preempt is greater than needed, the decision is left to the implementation. If the client has an option to interrupt transactions in a way that it can be restarted at the interruption point later, it SHOULD deploy it. An example for a mechanism providing such a service is the "SMTP Service Extension for Checkpoint/Restart" defined in RFC 1845 [2] If a client opts for the preemption of sessions instead of transactions, it MUST preempt the next session that reaches the end of a transaction independent of the priority of the emails being processed. 5.1.3. Handling of emails with multiple recipients If a client has to send an email with multiple recipients bearing different priorities to the same server, it SHOULD do so in one transaction treating the email according to the highest priority set for any of the recipients. The reorganization of the transfer order MUST NOT alter or even corrupt the emails, therefore allowing safe transmission of all emails in dependency of their priority. 5.2. Timely Delivery An important constraint usually associated especially with higher priority levels is, that messages with that priority have to reach its destination within a defined period of time. In some cases, higher priorities mean shorter maximum allowed time of delivery. As SMTP does not natively offer a service for timely delivery, the decision whether to use delivery time constraints for any or all priority levels is left to the NameSpace Policy. The "Deliver By SMTP Service Extension" (Deliver-By Extension) defined in [8] is an example of an SMTP extension providing a service that can be used to implement such constraints. If applicable NameSpace Policy does not specify time constraints for a given priority, no such constraints are applied. 5.3. Resolving conflicts for the sending order of messages with the same priority If two or more emails with the same priority are ready to be sent at Schmeing, et al. Expires February 24, 2007 [Page 10] Internet-Draft Priority Extension for SMTP August 2006 the same time, clients SHOULD send them in parallel if possible. NameSpace Policy and local policy MAY define rules to determine the order in which emails of the same priority are sent if the number of emails ready to be sent exceeds the number of emails that can be processed in parallel. Definitions from NameSpace Policy always take precedence over definitions from local policy. NameSpace Policy MAY disallow local policy to define addtitional criteria to determin sending order. If all applicable rules to determine the sending order from this memo, applicable NameSpace Policy and (if permitted) local policy still leave more emails ready to be sent than can be processed in parallel, the final decision is left to the implementation. 5.4. Size Limitation The larger a message, the longer the time required to send it over a given connection. In constrained networks this can impair the possibility of the MTS to deliver large messages with high priorities in time. Therefore, NameSpace Policy MAY define size constraints for any or all of its priority levels. These constraints can always be enforced independent of the SMTP Service Extensions supported by the server receiving the email. It is always possible for the server to count the number of received octets after the client indicated the end of the email and base the decision to accept or reject the email on this number. Therefor, after completely receiving the content of the email the server MUST check that the actual received size does not exceed the lowest size limitation allowed by the priorities set for all recipients. If the actual size is too big for at least one recipient, the server MUST NOT send a successful reply but MUST reject the email with an error code of "556 Priority defined size limit exceeded". If the client uses an SMTP Service Extension which allows to transmit the content in several chunks, such as [9], the server SHOULD check the current size after every chunk and SHOULD abort the email with the above error code as soon as the size constraint applicable for the email is violated. In addition to just counting the received octets while the email is being transmitted, a client and server may also deploy mechanisms, that allow for an earlier detection of size violations. An example for such a service is the "SMTP Service Extension for Message Size Declaration" (SIZE Extension) from RFC 1870 [3]. The SIZE Extension allows a client to declare the size of the email with an optional parameter to the MAIL FROM: SMTP command. This Schmeing, et al. Expires February 24, 2007 [Page 11] Internet-Draft Priority Extension for SMTP August 2006 parameter can be used for example to decide for each individual recipient whether the email fulfils the size constraints. Unless NameSpace Poicy defines another course of action, if the size provided with the SIZE argument of the MAIL FROM: command is too large for the priority set for any of the recipients, the recipient MUST be rejected with an error code of "556 Priority defined size limit exceeded". Processing for the remaining recipients is to continue normally. If the NameSpace Policy does not address size constraints, the default is that emails may be of arbitrary size. Constraints imposed by other circumstances such as available storage space or general limitations are, of course, unaffected by this. 5.5. Further Constraints Depending on the actual scenario in which this priority extension is deployed, it may be necessary to add further constraints to certain priority levels. Although it is strongly believed that limiting the delivery time and email size should be sufficient for most if not all purposes, NameSpace Policy may add further constraints such as limitations on content types or security classifications. For example, in military applications of priorities, messages without classification or with low classification may only be sent at the lower priority levels. Schmeing, et al. Expires February 24, 2007 [Page 12] Internet-Draft Priority Extension for SMTP August 2006 6. Recipient Dependent Priority Often, not all recipients need to receive the message with the same (high) priority. For example copy (CC, carbon copy) and blind copy (BCC, blind carbon copy) recipients usually can accept a lower priority than the primary recipient(s) because they do not have to act on the message but receive it only for informational purposes. Limiting the high priority messages only to the primary recipient complies with the basic principle of using high priorities only if really necessary. So the number of high priority messages is reduced to a minimum. Assigning every recipient an individual priority could be realized by sending the same message several times. This is unnecessary, non-practical and a waste of resources. Binding the priority to the recipient allows to deploy the multiple recipient per transaction feature of SMTP even when having different priorities for different recipients of the message. To lower the burden on the network, an MTA SHOULD send any message to as many recipients as possible with one transaction, even if the recipients have different priorities bound to them. In this case, the MTA MUST treat the email according to the highest priority bound to any of the recipients and MUST retain the original priority values for each recipient. A recipient address, that has a priority other than the non-priority set, is called a prioritized recipient (address). 6.1. Maillists, Aliases and Forwarding Several options exist to translate the address of an email recipient into one or more other addresses. Examples for this are aliases, maillist or email redirections e.g. via a .forward file. Priorities are assigned for a reason. Therefore, if a prioritized recipient address is to be translated or expanded as explained above, the translating or expanding entity (maillist manager, MTA etc.) SHOULD retain the original priority if possible and SHOULD set it for all expanded or translated addresses. NameSpace Policy however MAY deviate from this behavior for good reason. Schmeing, et al. Expires February 24, 2007 [Page 13] Internet-Draft Priority Extension for SMTP August 2006 7. NameSpaces A NameSpace identifies a set of one or more alpha-numeric priorities. These NameSpaces are registered with IANA and are unique for the registry associated with the SMTP priority extension. Ideally a given NameSpace satisfies the needs for groups of organizations and user sets. As an example, a NameSpace that correlates to the Q.735.3 protocol in support of Multi-Level Precedence and Preemption (MLPP) would be applicable to all messaging systems that support Q.735.3. New NameSpaces MUST be defined as an Informational RFC after Expert Review in accordance with the policy set forth in RFC 2434 [5]. The Expert Reviewer will consult with the Application Area Directors and the IEPREP mailing list. The following elements MUST be addressed when registering a new NameSpace: o Priority level(s) must be enumerated as a finite ordered list o A priority algorithm must be defined and described. This algorithm defines the schema of how messages are handled based on their labeled priority. For example, one algorithm may describe the use of preemption of resources of a lower priority message (i.e., dropped message) to satisfy a higher priority message. Another algorithm may involve queuing of messages based on availability of extra resources set aside for specific priorities. A third algorithm may be a hybrid of preemption and queuing. o Any new new response codes (warning or Error) unique to the NameSpace must be listed and explained. o The document needs to specify a new row for the following table that summarizes the features of the NameSpace. For the tabel below, there is ONE NameSpace label and ONE Algorithm. There N-number of levels, each having a corresponding Warning-Code OR Error-Code. It is expected that the warning/error codes are the same for each NameSpace.Priority tuple. Finally, an RFC is used as the reference for the table of information. NameSpace Algo Priority(s) Warn-Code Error-Code Reference --------- -------- ----------- --------- ---------- ---------