IMAP Extensions Working Group C. Daboo Internet Draft: IMAP VIEW Extension M. Pustilnik M. Crispin Document: draft-daboo-imap-view-00.txt June 1999 IMAP VIEW Extension Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet- Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) The Internet Society (1999). All Rights Reserved. Daboo, Crispin, Pustilnik Expires December 1999 [Page 1] Internet Draft IMAP VIEW Extension June 1999 Table of Contents 1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 Conventions Used in This Document . . . . . . . . . . . . . 2 3 Introduction and Overview . . . . . . . . . . . . . . . . . . 2 4 Commands . . . . . . . . . . . . . . . . . . . . . . . . . . 4 4.1 VIEW SET Command . . . . . . . . . . . . . . . . . . . . 4 4.2 VIEW Command . . . . . . . . . . . . . . . . . . . . . . 5 5 Responses . . . . . . . . . . . . . . . . . . . . . . . . . . 6 5.1 VIEW Untagged Response . . . . . . . . . . . . . . . . . 6 5.1.1 New message arrival . . . . . . . . . . . . . . . . . 6 5.1.2 Messages moving in the view . . . . . . . . . . . . 7 5.2 EXISTS Untagged Response . . . . . . . . . . . . . . . . 8 5.3 EXPUNGE Untagged Response . . . . . . . . . . . . . . . 9 6 Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 10 7 Security Considerations . . . . . . . . . . . . . . . . . . 11 8 References . . . . . . . . . . . . . . . . . . . . . . . . . 11 9 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 11 10 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 11 1 Abstract The VIEW extension to the Internet Message Access Protocol [IMAP4] permits a subset of messages in the mailbox to be processed separately from the entire set of messages in the mailbox. This allows a client to restrict its view to only the messages appearing in this set. The subset of messages also need not be returned in order of their sequence numbers, allowing clients to access messages in a particular sort order. The subsetting and sorting of messages is handled by existing [IMAP4] commands that return a set of messages as their result. This extension takes the set returned from such a command and applies the VIEW methodology to it. 2 Conventions Used in This Document The key words "REQUIRED", "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" in this document are to be interpreted as described in "Key words for use in RFCs to Indicate Requirement Levels" [KEYWORDS]. Formal syntax is defined using ABNF [ABNF] as modified by [IMAP4]. In examples, "C:" and "S:" indicate lines sent by the client and server respectively. 3 Introduction and Overview The VIEW extension is present in any IMAP4 implementation which returns "VIEW" as one of the supported capabilities in the Daboo, Crispin, Pustilnik Expires December 1999 [Page 2] Internet Draft IMAP VIEW Extension June 1999 CAPABILITY command. The VIEW extension specifies two new commands and introduces two new untagged responses, modifies two existing [IMAP4] untagged responses, as well as putting requirements on standard IMAP4 protocol responses when the command is in effect. Each of the features in VIEW are optimizations; clients can provide the same functionality, albeit more slowly, by using existing commands. The design goal of the VIEW extension is to allow clients to reference messages only by their "view position numbers" as opposed to sequence numbers or UIDs, if so desired, and thus remove the need for the client to maintain its own mapping between sequence number, UID and view position. This extension makes the following changes to the IMAP4 protocol: a) adds a new VIEW SET command which takes an optional sub-command When the new VIEW SET command is used by the client, the following changes also take place: b) New FETCH item: VIEW indicates the position of messages in the view, which may be different from message sequence number c) New command: VIEW the VIEW modifier is used with a FETCH, COPY, STORE or SEARCH command, and instructs the server to interpret the sequence numbers provided by the client in the command as VIEW positions, or to return results using view positions rather than sequence numbers d) New untagged responses: VIEW indicates new message arrival, a change in position of a message within the current view or a message moving in or out of the current view. The sequence number, UID and new view position number of the message are supplied, and optionally the old position when the message is not new VIEW SEARCH indicates the set of messages in the current view that match search criteria for a VIEW SEARCH command Daboo, Crispin, Pustilnik Expires December 1999 [Page 3] Internet Draft IMAP VIEW Extension June 1999 e) Changes to untagged responses and response codes EXISTS the standard EXISTS response is enhanced to include information about the total number of messages in the current view EXPUNGE the standard EXPUNGE response is enhanced to include information about message UID and view position when a VIEW SET command is in effect, the server MUST include the VIEW item in all FETCH responses to the client for any unsolicited responses The rest of this document describes these changes more rigorously. The document will use the IMAP4 SEARCH command as an example of a sub-command to VIEW SET. However, other commands can be used with VIEW SET as well. 4 Commands 4.1 VIEW SET Command Arguments: OPTIONAL sub-command and associated arguments Responses: REQUIRED untagged responses: modified EXISTS if sub-command present Result: OK - view completed, now in view state NO - view failure BAD - command unknown or arguments invalid When the VIEW SET command is in effect, it results in the following changes to the IMAP4 protocol: a) The server creates a 'view' that is a subset of messages in the mailbox that match the results of the sub-command. The position of a message in the view, its view position number, is then given by the VIEW fetch item response. Messages outside of the view have a view position of 0 (zero). b) The VIEW command can be used by the client to request information about messages in the current view set, without requiring the client to determine the view positions of every message in the mailbox (c.f. Section 4.2). c) The server MUST return a VIEW fetch item in every unsolicited FETCH response to ensure the client does not need to maintain its own mapping from sequence numbers to view positions. Daboo, Crispin, Pustilnik Expires December 1999 [Page 4] Internet Draft IMAP VIEW Extension June 1999 d) A new VIEW untagged response MUST be sent by the server when new messages arrive or when existing messages move within the current view, or are moved into or out of the current view. e) The server MUST modify the EXISTS untagged response to include additional information to indicate the current number of messages in the view set. f) The server MUST modify the EXPUNGE untagged response to include additional information to indicate the expunged message's UID and view position number. g) The VIEW SET command with no sub-command can be used to turn off the view facility and return the server to standard IMAP4 behaviour. Examples: C: A999 VIEW SET SEACH FROM "Smith" S: * 23 EXISTS 5 S: A999 OK VIEW SET completed 4.2 VIEW Command Arguments: command name command arguments Responses: untagged responses: FETCH, VIEW SEARCH Result: OK - VIEW command completed NO - VIEW command error BAD - command unknown or arguments invalid The VIEW command has two forms. In the first form, it takes as its arguments a COPY, FETCH, or STORE command [IMAP4] with arguments appropriate for the associated command. However, the numbers in the message set argument are view position numbers instead of message sequence numbers. In the second form, the VIEW command takes a SEARCH command with SEARCH command arguments [IMAP4]. The interpretation of the arguments is the same as with SEARCH [IMAP4]; however, the numbers returned in a VIEW SEARCH response for a VIEW SEARCH command are view position numbers instead of message sequence numbers. In addition, the search is carried out only on those messages in the current view set, so a view position of 0 (zero) MUST NOT be returned in the search results. The number after the "*" in an untagged FETCH response is always a message sequence number, not a view position number, even for a VIEW command response. However, server implementations MUST implicitly include the VIEW fetch item as part of any FETCH response caused by a VIEW command, regardless of whether a VIEW was specified as a Daboo, Crispin, Pustilnik Expires December 1999 [Page 5] Internet Draft IMAP VIEW Extension June 1999 message data item to the FETCH. Examples: C: A999 VIEW FETCH 1:3 FLAGS S: * 23 FETCH (FLAGS (\Seen) VIEW 2) S: * 24 FETCH (FLAGS (\Seen) VIEW 3) S: * 25 FETCH (FLAGS (\Seen) VIEW 1) S: A999 OK VIEW FETCH completed C: A999 VIEW SEARCH UNSEEN S: * VIEW SEARCH 1 2 4 6 S: A999 OK VIEW SEARCH completed 5 Responses 5.1 VIEW Untagged Response There are two forms of this response. 5.1.1 New message arrival VIEW indicates that a new message has arrived with the given sequence number, UID and view position number When using the VIEW SET command, the VIEW untagged response augments the EXISTS untagged response as a means of new mail notification. Specifically it is used to tell the client where new messages appear in the current view. The server MUST send an untagged VIEW response for every new message that arrives in the mailbox, after the corresponding untagged EXISTS response is sent to indicate this, when the VIEW SET command is in effect. When the VIEW SET command is in effect, and the view position number of a new message is non-zero (i.e. the new message appears in the current view), the untagged VIEW response causes the view position numbers of all subsequent messages in the mailbox's current view to be incremented. Example: C: NOOP S: * 173 EXISTS 24 S: * 1 RECENT S: * 173 VIEW 5274234 14 S: NOOP complete indicates that a new message with sequence number 173 and UID 5274234 has been inserted into the view with view position Daboo, Crispin, Pustilnik Expires December 1999 [Page 6] Internet Draft IMAP VIEW Extension June 1999 number 14. Any message that used to have a view position number 14 is now at position 15, etc. If a new message arrives outside the current view (i.e. not matching the specified search criteria), the view position number of the new message is given as 0. Example: C: NOOP S: * 174 EXISTS 24 S: * 2 RECENT S: * 174 VIEW 5274235 0 S: NOOP complete indicates that a message has arrived with sequence number 174 and UID 5274235, however, the VIEW SET command in effect prevents it from being in the current view. A VIEW untagged response MUST NOT be sent when no command is in progress; nor while responding to a FETCH, SEARCH or STORE command. This rule is necessary to prevent loss of synchronization of message sequence numbers between client and server. 5.1.2 Messages moving in the view VIEW indicates that a message with the given sequence number and UID has changed its position within the view Some sub-commands of the VIEW SET command may change the sort order of messages in the view. In such a case, changes to message flags or keywords can cause the position of messages in sorted order to change. Also, when search criteria are specified, changes to message flags or keywords can cause messages to move into or outside of the view. The server MUST communicate these changes to the client by sending it the VIEW untagged response. When VIEW SET search criteria are in effect, changes made to a message can cause it to drop out of view or, on the contrary, become visible. When a message drops out of view, the new view position number is sent as 0. Similarly, when a message appears in view, for example, as a result of another agent changing the message attributes, the old view position number is sent as 0. When the old view position number in the VIEW untagged response is non-zero, the effect of VIEW is a message inserted into the view at the position given by . The view position numbers of all subsequent messages inthge current view are incremented. Daboo, Crispin, Pustilnik Expires December 1999 [Page 7] Internet Draft IMAP VIEW Extension June 1999 When the new view position number in the VIEW untagged response is non-zero, the effect of VIEW is a message removed from the view at the position given by . The view position numbers of all subsequent messages inthe current view are decremented. It is meaningless to have a VIEW untagged response where both and view position numbers are 0; the server MUST NOT generate VIEW responses of this form. A VIEW untagged response MUST NOT be sent as the result of messages being expunged (which is handled by the modified untagged EXPUNGE response). Examples: S: * 172 VIEW 5274234 14 2 indicates that the view position of the message with sequence number 172 and UID equal to 5274234 has changed from 14 to 2. The view position numbers of the messages in the range 2:13 are incremented. S: * 172 VIEW 5274234 14 0 indicates that the message with view position number 14 has dropped out of view. The view position numbers of subsequent messages are decremented. The message continues to exist, however, it is outside of the current view and is therefore no longer accessible to the session via the VIEW command. S: * 173 VIEW 5274234 0 2 indicates that the message with UID 5274234 has appeared in the current view and has been assigned view position number 2. The view position numbers of subsequent messages are incremented. A VIEW untagged response MUST NOT be sent when no command is in progress; nor while responding to a FETCH, SEARCH or STORE command. This rule is necessary to prevent loss of synchronization of message sequence numbers between client and server. 5.2 EXISTS Untagged Response <# in mailbox> EXISTS <# in view> indicates the total number of messages in the mailbox and in the current view The modified EXISTS untagged response MUST only be used when the VIEW SET command is in effect. This MUST be sent in response to a VIEW SET command taking a sub-command argument. Otherwise this Daboo, Crispin, Pustilnik Expires December 1999 [Page 8] Internet Draft IMAP VIEW Extension June 1999 response is sent under the same circumstances as the unmodifed EXISTS response (i.e. in when there is a change to the total number of messages in the mailbox). Example: C: A142 SELECT INBOX S: * 172 EXISTS S: * 3 RECENT S: * OK [UNSEEN 12] Message 12 is first unseen S: * OK [UIDVALIDITY 3857429045] UIDs valid S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) S: OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited S: A142 OK [READ-WRITE] SELECT completed C: A143 VIEW SET SEARCH FROM "Smith" S: * 172 EXISTS 23 S: A143 OK VIEW SET completed In this example, there are 172 messages in the mailbox and 23 messages have a from address including the text "Smith". 5.3 EXPUNGE Untagged Response EXPUNGE indicates that a message with given sequence number, UID and view position number has been permanently removed from the mailbox When VIEW SET is in effect, the EXPUNGE untagged response [IMAP4] is modified to include the UID and view position number of the message being expunged. The server MUST use this modified form of the EXPUNGE response whenever a VIEW SET command is in effect. When search criteria are specified in the VIEW SET command, and a message outside the current view is expunged, the view position number of the expunged message is given as 0. When the view position number of the expunged message is non-zero, the modified EXPUNGE untagged response causes the view position numbers of all subsequent messages in the mailbox's current view to be decremented. Examples: S: * 172 EXPUNGE 5274234 14 indicates that the message with sequence number 172, UID 5274234 and view position number 14 has been expunged from its mailbox. Any message previously at position 15 is now at position 14, etc Daboo, Crispin, Pustilnik Expires December 1999 [Page 9] Internet Draft IMAP VIEW Extension June 1999 S: * 173 EXPUNGE 5274234 0 indicates that a message with sequence number 173, UID 5274234 was expunged, however, the message was outside the current view. A modified EXPUNGE untagged response MUST NOT be sent when no command is in progress; nor while responding to a FETCH, SEARCH or STORE command. This rule is necessary to prevent loss of synchronization of message sequence numbers between client and server. 6 Formal Syntax The following syntax specification uses the Augmented Backus-Naur Form (ABNF) notation as specified in [ABNF]. Non-terminals referenced but not defined below are as defined by [IMAP4]. Except as noted otherwise, all alphabetic characters are case- insensitive. The use of upper or lower case characters to define token strings is for editorial clarity only. Implementations MUST accept these strings in a case-insensitive fashion. viewset ::= "VIEW SET" [SP sub_command] sub_command ::= ;; [IMAP4] command that returns a set ;; of messages e.g. SEARCH or SORT new_resp ::= nz-number "VIEW" SP uniqueid SP position [SP position] ;; sequence number of message ;; UID of new message ;; followed by the new view position number ;; optionally followed by the old view position ;; 0 indicates "outside the current view" exists_resp ::= number "EXISTS" SP number ;; total number of messages ;; number of message in current view ;; replaces standard IMAP4 EXISTS response ;; when VIEW SET is in effect expunge_resp ::= nz-number "EXPUNGE" SP uniqueid SP position ;; sequence number of message ;; UID of message being expunged ;; followed by its view position number ;; 0 indicates message being expunged ;; that is "outside the current view" ;; replaces standard IMAP4 EXPUNGE response ;; when VIEW SET is in effect Daboo, Crispin, Pustilnik Expires December 1999 [Page 10] Internet Draft IMAP VIEW Extension June 1999 position ::= number ["." number] ;; position number of messages in the view ;; may be hierarchical if sub-command defines ;; hierarchic ordering of messages 7 Security Considerations There are no known security issues with this extension. 8 References [ABNF] Crocker, Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 2234, Internet Mail Consortium, Demon Internet Ltd, November 1997. [KEYWORDS] Bradner, "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, Harvard University, March 1997. [IMAP4] Crispin, M., "Internet Message Access Protocol - Version 4rev1", RFC 2060, University of Washington, December 1996. [MIME-IMB] Freed, N., and Borenstein, N.., "MIME (Multipurpose Internet Mail Extensions) Part One: Format of Internet Message Bodies", RFC 2045, November 1996. [RFC-822] Crocker, D., "Standard for the Format of ARPA Internet Text Messages", STD 11, RFC822, August 1982. 9 Acknowledgments Steve Hole, Chris Newman and Larry Osterman have made significant contributions to the creation and refinement of the ideas expressed in this document. 10 Authors' Addresses Cyrus Daboo Cyrusoft International, Inc. Suite 780, 5001 Baum Blvd. Pittsburgh, PA 15213 Phone: (412)605-0499 Email: daboo@cyrusoft.com Mark Pustilnik Microsoft 1 Microsoft Way Redmond, WA 98052 Daboo, Crispin, Pustilnik Expires December 1999 [Page 11] Internet Draft IMAP VIEW Extension June 1999 Phone: (425)703-5758 Email: markpu@microsoft.com Mark R. Crispin Networks and Distributed Computing University of Washington 4545 15th Aveneue NE Seattle, WA 98105-4527 Phone: (206) 543-5762 EMail: MRC@CAC.Washington.EDU Daboo, Crispin, Pustilnik Expires December 1999 [Page 12]