Internet Draft: Lemonade Notifications Architecture R. Gellens (Editor) Document: draft-ietf-lemonade-notifications-06.txt Qualcomm Expires: July 2008 January 7, 2008 Intended Status: Informational Lemonade Notifications Architecture Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) The IETF Trust (2008). All Rights Reserved. Abstract This document discusses how to provide notification and filtering mechanisms to IMAP to meet Lemonade goals. This document also discusses the use of server to server notifications, and how server to server notifications fit into an architecture which provides server to client notifications. Gellens [Page 1] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . 2 2. Notifications logical architecture and LEMONADE Profile bis 2 3. Event-based synchronization . . . . . . . . . . . . . . . 4 4. Push Email . . . . . . . . . . . . . . . . . . . . . . . . 5 5. Server to server notifications considerations . . . . . . . 5 5.1. Scope of server to server notifications . . . . . . . 5 5.2. Basic Operation . . . . . . . . . . . . . . . . . . . . 8 5.3. Notification payloads . . . . . . . . . . . . . . . . 9 5.4. Server to server notification protocol details . . . . 9 5.4.1. Generic case . . . . . . . . . . . . . . . . . . . 10 5.5. Event orders . . . . . . . . . . . . . . . . . . . . . 10 5.6. Reliability . . . . . . . . . . . . . . . . . . . . . 11 6. Security Considerations . . . . . . . . . . . . . . . . . . 11 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . 11 8. Normative References . . . . . . . . . . . . . . . . . . . . 11 9. Informative References . . . . . . . . . . . . . . . . . . 11 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 12 11. Editor's Address . . . . . . . . . . . . . . . . . . . . . 13 Appendix A: Changes from Previous Versions . . . . . . . . . . 13 Intellectual Property Statement . . . . . . . . . . . . . . . 15 Full Copyright Statement . . . . . . . . . . . . . . . . . . . 16 1. Introduction The lemonade work ([LEMONADE-PROFILE] and [LEMONADE-PROFILE-BIS]) identified a need to provide notification and filtering mechanisms for use with IMAP [IMAP]. In addition, external groups which make use of IETF work have communicated their requirements, which the lemonade working group has accepted [OMA-LEMONADE-ARCH]. OMA requirements for within-IMAP ("inband") and out-of-IMAP ("outband") server to client notifications are listed in [OMA-ME-RD]. 2. Notifications logical architecture and LEMONADE Profile bis The target logical architectures involving the LEMONADE Profile and notifications are discussed in the revised Lemonade Profile ("profile-bis") document [LEMONADE-PROFILE-BIS]. Gellens [Page 2] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 Figure 1 illustrates how notification and filtering fit in the context of Lemonade profile-bis. +--------------+ | |.... +=========| Notification |.NF. ! | Mechanism |.... ! | |^ ^ ! +--------------+! ! Notif.! ! ! Protocol! !======================! ! ! ! ! ! ! ! ! !=====================\ ! != ====================\.... +---------+ ! ! +-----------.NF.---+ | | V ! | .... | | MTA | +-----+ IMAP |.... | LMTP/ |.... |<== | | <======> |.VF. IMAP ....| SMTP |.AF. | | MUA |\ ME-2a |.... Store .DF.|<=======|.... | | | \ | ....| | | +-----+ \ +------------------+ +---------+ \ ! \ !URLAUTH SUBMIT\ ! \ +----v-----+ \ | | +-----+ \ | LEMONADE | SMTP | |==> ===>| Submit |===============>| MTA | ME-2b | Server | | | | | +-----+ +----------+ Figure 1: Filtering mechanism defined in Lemonade Profile bis architecture. In Figure 1, four categories of filters are defined: 1. AF: Administrative Filters: Created and maintained by mail admin. AF are typically not configured by the user and are used to apply policies, content filtering, virus protection, spam filtering, etc. 2. DF: Deposit Filters: Executed on deposit of new mail. Can be defined as SIEVE filters [SIEVE]. They can include vacation notices. Gellens [Page 3] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 3. VF: View Filters: Define which messages are important to a client. May be implemented as pseudo-virtual mailboxes [CONTEXT]. Clients may use this to restrict which messages they synchronize. 4. NF: Notification Filters: Determines when out-of-IMAP ("outband") notifications are sent to the client. The filters are manageable by the client as follows: * NF and DF: When the NF is internal to the mail store, via a Sieve management protocol. See [IMAP-SIEVE] and [MANAGE-SIEVE] for more information. When the NF is within the notification mechanism, protocols specific to it may be used for management. * VF: via pseudo-virtual mailboxes as defined in [CONTEXT]. In Figure 1, the NF are shown both as part of the mail store (for example, using Sieve) and as an external notification mechanism. Either approach can be used. 3. Event-based synchronization +----------------+ +---------------+ +------------+ | COMPLETE | (VF) | VIEW | (NF) | PUSH | | REPOSITORY | View | REPOSITORY |Notification| REPOSITORY | | |Filters| | Filters | | | all email | | email to be | | important | | in the account |=======|synched by the |=========| email / | | | | mobile client | | | events | | | | (CONTEXT) | | | | +----------------+ +---------------+ | +------------+ | | IDLE / | NOTIFY Out-of-IMAP | Notifications | | V V Figure 2: Filters and Repositories For in-IMAP ("inband") notifications, the MUA (client) issues IDLE [IDLE], or the successor extension command NOTIFY [NOTIFY]; the LEMONADE IMAP server sends notifications as unsolicited responses to the client. Gellens [Page 4] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 Out-of-IMAP ("outband") notifications are messages (notification payloads), and possibly a set of surrounding exchanges, sent in an appropriate format to a particular address (such as an IP address and port). This may be the address of the client. These messages are not IMAP messages, and hence conform to the interface of a notification server and mechanism which is responsible for finalizing the format and sending the notifications to the client using an appropriate protocol. 4. Push Email A good user experience of "push email" requires that when "interesting" events occur in the mail store, the client is informed so that it can connect and resynchronize. [LEMONADE-PROFILE-BIS] contains more information, especially in Section 4.1.2. 5. Server to server notifications considerations The following sections focus on considerations and usage of the Lemonade notifications for server to server notifications With server to server notifications, a messaging system (e.g., a mail server, voice mail system, etc.) generates event notifications. These notifications describe mailbox state change events (e.g., arrival of a new message, mailbox full, etc.) See [MSGEVENT] for a list of such events. These state change notifications are sent to a notification mechanism, which may generate an alert or notifications for delivery to one or more clients or the user. Server to server notifications allow the messaging system to generate end user or client notifications without needing to keep track of notification settings for users or clients. The notification mechanisms are the entities which maintain notification preferences for clients or users. Using server to server notifications the messaging system can provide the end user with a unified notification experience (the same look and feel for all messaging systems' accounts), while allowing smooth integration of additional messaging systems. Gellens [Page 5] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 5.1. Scope of server to server notifications The POP3 and IMAP4 Internet mail protocols allow mail clients to access and manipulate electronic mail messages on messaging systems. By definition and scope, these protocols do not provide off-line methods to notify an end user when the mailbox state changes. Nor does either protocol define a way to aggregate the status within the end user's various mailboxes. The desire for this functionality is obvious. For example, from the very early days of electronic mail, various notifications mechanisms have been used, including login shell checks, and simple hacks such as [BIFF]. To provide an end user with unified notifications and one centralized message-waiting indication (MWI), notification mechanisms are needed which aggregate the information of all the events occurring on the end user's different messaging systems. Server to server notifications allow the messaging system to send state change events to the notification mechanisms when something happens in or to an end user's mailbox. Notification mechanisms can be broadly grouped into three general architectures: external smart clients, intrinsic notification, and separate notification mechanisms. External smart clients are agents independent of the mail system that periodically check mailbox state (or receive notifications, for example via IMAP IDLE) and inform the user or the user's mail client. Many such systems have been used over the years, including login shells that check the user's mail spool, laptop/desktop tiny clients that periodically poll the user's mail servers, etc. Intrinsic notification is any facility within a mail system that generates notifications, for example the server component of [BIFF], or, for more modern systems, the recent Sieve extensions for notifications [SIEVE-NOTIFY]. Separate notification systems are architecturally cleaner in that they separate the state change event notification from the end-user or client notification, allowing a mail system to do former, and specialized systems (such as those which handle presence) to be responsible for the latter. This separation also allows notifications to be generated for any service, not just electronic mail. It is important to emphasize that for the purposes of the Lemonade work, the scope of server to server notifications is limited to communications between messaging systems and notification mechanisms (the third architectural type). Communication between the notification mechanisms and the end user or devices (which might use SMS, WAP Push, instant messaging, etc.) are out of scope. Likewise, the scope generally presumes a security relationship between the Gellens [Page 6] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 messaging system and the notification mechanism; it then becomes the responsibility of the notification mechanism. However, the specifics of security, trust relationships, and related issues, depend on the specifics of both server to server notifications and notification mechanisms. In addition to an external notification mechanism, Sieve can be used for notifications [SIEVE-NOTIFY]. While this is limited to email, because many systems already provide Sieve support, it is often a fairly easy and quick deployment option to provide some form of notifications. Here is an example of server to server notifications within a complete context including the notification mechanism: A new mail message is deposited in a mailbox; the server then sends a "new message" notification to the notification service, which then notifies the end user by a Short Text Message (SMS). This process can include other mailbox events that are important to the end user or needed by the client to support user preferences, such as "mailbox full" and "message rejected" or other mailbox status changes. The notifications might carry additional information for the end user, such as the mailbox status, message attributes, etc. Gellens [Page 7] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 Figure 3 depicts the full context of server to server notifications; only the left side is in scope: +--------+ +--------+ New | | | SMS | Message | Email | \ |Gateway | -------> |Server 1| \ _ | | +--------+ \ /| +--------+ ^ \ / | \ / ^ | \ +--------------+ / | +--------+ +--------+ | _|+-------------|+ / | | MWI | Read | Voice | | || |/ | |Gateway | Message | Mail |-------->| Notification |------->| | -------> | Server | | ^ _ +| Mechanisms |\ ^ | +--------+ +--------+ | | /| |(out of scope)| \ | | | |/ | | \| | | / ^ +--------------+ ^ \ | |/| | \ | |\| +--------+ / | | \ | | \ +--------+ Mailbox | | /| | | \| | |\ | Wap | Full | Email |/ | | | ^ \ | |_|| Push | -------> |Server 2| | | | | |\| | |Gateway | +--------+ | | | | | \ | +--------+ | | | | | |\| | | | | | | \ | | | | | | |\ | | | | | | |_|+--------+ | | | | | | | | IM | | | | | | | | |Gateway | | | | | | | | | | | | | | | | | +--------+ | | | | | | | Server to OTHER Server PROTOCOLS Notifications (out of scope) (in scope) Figure 3: Scope of server to server notifications 5.2. Basic Operation The messaging system sends state change event notifications to the notification mechanism (which in turn might notify a client or end user) for events that occur in the end user's mailboxes. Each such notification, referring to a single mailbox event, is called a state change event. Gellens [Page 8] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 The state change event contains data regarding the mailbox event which has occurred. The state change event describes the change, but normally does not specify how or if the end user or client is notified; this allows the end user and client notification preferences to be maintained only within the notification mechanism. From the Lemonade viewpoint, out-of-IMAP (outband) notifications are usually desired only when the client is not connected to the IMAP server (since inband notifications are used when there is an IMAP connection). Thus, it is helpful for the messaging system to be able to inform the notification mechanism when the user logs in or out, and which client is used (when this information is available). When Sieve is used, the Sieve engine might have access to this information. 5.3. Notification payloads The cases of Figure 1 and Figure 3 are very similar. In both cases a message is generated by the message store as a result of a state change event. This message may be delivered to the end user, a client, or to an external notification mechanism which might deliver an equivalent message to the user or to a client. Within the context of Lemonade profile (Figure 1), the event is filtered by NF. That is, the Notification Filters logically determine which state change events cause notification to the user or client. Notifications allow for a rich end user experience. This might include conveying mailbox status, new message attributes, etc., to the user or client independent of the client's connection to the mail store. Notifications also allow for different Message Waiting Indicator (MWI) behaviors (e.g., turn MWI indication off after all the messages in all the end user's mailboxes have been read, should such an unlikely thing occur in the real world). The payload of a notification might include a URL referring to the message which caused the event, possibly using URLAUTH [URLAUTH]. Gellens [Page 9] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 5.4. Server to server notification protocol details 5.4.1. Generic case Within the more general case of server to server notification, the payload may be an arbitrary text or binary message. In both cases the interaction model is defined as: 1. An event takes place in the message store 2. The event is filtered. As a result it may be hidden or result in a notification. 3. The notification is a message in a particular payload that is prepared for the target notification mechanisms. 4. The payload is augmented with the necessary information to tell the notification mechanism how and where to send the notification. 5. The augmented payload is then formatted as required by the target notification mechanisms (i.e., the right format on the right port to be sent to the right address, possibly with an appropriate protocol binding, e.g., HTTP PUT) plus the information about where and how to send the notification. This last step is imposed by the notification mechanisms and must be known by the notification generating filter. Different interfaces and bindings may be used depending on the notification channel. In the most general case, the mail system sends bulk state change events to an external notification mechanism, and it is the notification mechanism that filters the events by matching against the user's or client's preferences. In the most mail-specific case, the mail system performs the filtering itself, for example using Sieve. 5.5. Event orders For lemonade profile bis, the event order is generally not important. By including information such as the modification sequence identifier (called a modseq or mod-sequence) [CONDSTORE] in notifications, the receiving client can quickly and easily determine if it has already processed the triggering event (for example, if a notification arrives out of order, or if the client has resynchronized). Gellens [Page 10] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 For generic server to server notifications, the order is likely to matter and the messaging system needs to provide the notifications to the notification mechanism in the order that they are generated by mailbox events. 5.6. Reliability For Lemonade profile bis, lost or delayed notifications to the client are tolerated. A client can resynchronize its state (including that reported by any missing events) when it connects to the server. For generic server to server notifications, it is assumed that the data in a state change event is important, and therefore a high level of reliability is needed between the messaging system and any external notification mechanisms. 6. Security Considerations Notification content (payload) needs to be protected against eavesdropping and alteration when it contains specific information from messages, such as the sender. Even when the content is trivial and does not contain privacy-sensitive information, guarding against denial of service attacks may require authentication or verification of the notification sender. 7. IANA Considerations None. 8. Normative References [IMAP] Crispin, M. "IMAP4, Internet Message Access Protocol Version 4 rev1", RFC 3501, March 2003. http://www.ietf.org/rfc/rfc3501 [LEMONADE-PROFILE] Maes, S.H. and Melnikov, A., "Lemonade Profile", RFC 4550, June 2006. [LEMONADE-PROFILE-BIS] D. Cridland, A. Melnikov, S. Maes, "LEMONADE profile bis", draft-ietf-lemonade-profile-bis-07.txt, (work in progress). Gellens [Page 11] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 9. Informative References [BIFF] Gellens, R., "Simple New Mail Notification", RFC 4146, August 2005. [CONTEXT] D. Cridland, C. King, "Contexts for IMAP4", draft-cridland-imap-context-03.txt (work in progress). [CONDSTORE] Melnikov, A., Hole, S., RFC 4551, June 2006. [IMAP-SIEVE] Leiba, B., "Support for Sieve in Internet Message Access Protocol (IMAP4)", draft-ietf-lemonade-imap-sieve-04.txt (work in progress). [MANAGE-SIEVE] Martin, T. and A. Melnikov, "A Protocol for Remotely Managing Sieve Scripts", draft-martin-managesieve-08.txt, (work in progress). [MSGEVENT] Gellens, R., Newman, C., "Internet Message Store Events", draft-ietf-lemonade-msgevent-05.txt, (work in progress). [IDLE] Leiba, B. "IMAP4 IDLE Command", RFC 2177, June 1997. [NOTIFY] C. King, A. Melnikov, A. Gulbrandsen, "The IMAP NOTIFY Extension", draft-ietf-lemonade-imap-notify-02.txt, (work in progress). [OMA-LEMONADE-ARCH], E. Burger, G. Parsons, "LEMONADE Architecture -- Supporting OMA Mobile Email (MEM) using Internet Mail", draft-ietf-lemonade-architecture-01.txt, (work in progress). [OMA-ME-RD] Open Mobile Alliance Mobile Email Requirement Document, (Work in progress). http://www.openmobilealliance.org/ [SIEVE] Showalter, T., "Sieve: A Mail Filtering Language", RFC 3028, January 2001. [SIEVE-NOTIFY] A. Melnikov, B. Leiba, W. Segmuller, T. Martin, "SIEVE Email Filtering: Extension for Notifications", draft-ietf-sieve-notify-12.txt, (work in progress). [URLAUTH] Crispin, M. and Newman, C., "Internet Message Access Protocol (IMAP) - URLAUTH Extension", RFC 4467, May 2006. Gellens [Page 12] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 10. Acknowledgments The original, and significantly longer, version of this document was authored by Stephane H. Maes and Ray Cromwell of Oracle Corporation. The current and original authors want to thank all who have contributed key insight in notifications and filtering and have authored specifications or drafts used in this document. The current and original authors want to thank the authors of the original work on Server To Server Notification Protocol Requirements (draft-ietf-lemonade- notify-s2s-00) some of whose material has been incorporated in the present document, and in particular, Gev Decktor. 11. Editor's Address Editor: Randall Gellens QUALCOMM Incorporated 5775 Morehouse Drive San Diego, CA 92121 rg+ietf@qualcomm.com Gellens [Page 13] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 Appendix A: Changes from Previous Versions THIS SECTION TO BE REMOVED PRIOR TO PUBLICATION. version -06: o Added pointer to profile-bis 5.4.2 ("External Notifications") o Cleaned up references, split into Normative/Informative o Reworded old Section 4 ("Event-based synchronization"), added NOTIFY and IDLE references o Deleted Section 1 ("Conventions Used in this Document") as none of it applied any longer. o Reworded and reworked old Section 5 ("Server to server notifications considerations") and its subsections and added reference to msgevents draft and RFC 4146. o Changed "notification request" to "state change event". o Added "within-IMAP" and "out-of-IMAP" to most occurrences of "inband" and "outband". o Added mention of user login/logout as a state change that can be used to trigger outband notifications. o Redid Figure 1 to make it easier to understand, and also to show that NF might be within the mail store (Sieve) or in an external notification mechanism. o Deleted section 5.3 ("Server to server terminology"). o Deleted reference to draft-ietf-lemonade-notification-protocol-xx. o Major changes to section 5.4 ("Notification payloads") and section 5.5 ("Server to server notification protocol details"). o Deleted section 5.5.2 ("Abstracted notification protocol"), section 5.5.3 ("Exception Handling"), and section 5.6 ("Server to server complementary information"). o Rewrote section 5.7 ("Event orders") and added reference to RFC 4551. o Rewrote section 5.8 ("Reliability"). o Added references to sieve-notify, IMAP NOTIFY, IMAP, and others. o Deleted many, many references. o Updated references. o Corrected references. o Split references into normative and informative. o Added reference to draft-ietf-lemonade-architecture-00.txt. o Rewrote Introduction. o Changed draft name from "... and Filters" to "... Architecture". version -05: o Significant deletion of sections, per www1.ietf.org/mail-archive/web/lemonade/current/msg03936.html Gellens [Page 14] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 version -04: o Update dates, slight reformatting, add editor's note for References version -03: o Updated examples to use new METADATA syntax o Drop CLEARIDLE and reference A. Melnikov's IMAP-EVENTS o XEMN notification format extended to with event and view attributes o View filter is a work in progress. Several proposals are being discussed, so the draft has been revised to try and capture high level requirements (e.g. out of band notifications must be able to identify which view an event occurred for) o Added notification protocol details and reference version -02: o LPROVISION/LGETPREFS/LSETPREFS removed in favor of mailbox annotations o Updated inband notification section to include discussion of CLEARIDLE and MSGEVENTS o EMN payload clarified for both wakeup and extended formats. o Some reference clean-up o Add server to server notifications based on the expired draft draft-ietf-lemonade-notify-s2s-00. version -01: o Move SMS / WAP examples to an informative appendix. o Restrict the exchange of keys via LPROVISION to secure exchanges. o Differentiate ANNOTATE from LPROVISION on that basis. versin -00: o Initial release Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Gellens [Page 15] Expires July 2008 Internet Draft Lemonade Notifications Architecture January 2008 Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Full Copyright Statement Copyright (C) The IETF Trust (2008). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. Gellens [Page 16] Expires July 2008