LEMONADE Internet Draft: IMAP statistics for Lemonade A. Meylan Document: draft-meylan-imap-statistics-lemonade-00.txt QUALCOMM Expires: April 2005 October 2004 IMAP statistics for Lemonade Status of this Memo By submitting this Internet-Draft, I certify that any applicable patent or other IPR claims of which I am aware have been disclosed and any of which I become aware will be disclosed, in accordance with RFC 3668 (BCP 79). By submitting this Internet-Draft, I accept the provisions of Section 3 of RFC 3667 (BCP 78). Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) The Internet Society (2004). All Rights Reserved. Abstract This memo presents statistics on IMAP's network utilization for some common use cases. In addition we evaluate the performance of dictionary based command compression as well as transport layer compression for IMAP traffic. Meylan Expires April 2005 [Page 1] Internet Draft IMAP statistics for Lemonade October 2004 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Traffic for a few IMAP use-cases . . . . . . . . . . . . . . . 2 2.1 Use-cases description . . . . . . . . . . . . . . . . . . 2 2.2 A few comments . . . . . . . . . . . . . . . . . . . . . . 3 3. IMAP command dictionary based compression . . . . . . . . . . 3 4. IMAP with TLS featuring compression . . . . . . . . . . . . . 4 4.1 TLS use-cases . . . . . . . . . . . . . . . . . . . . . . 4 4.2 Observations . . . . . . . . . . . . . . . . . . . . . . . 5 5. Security Considerations . . . . . . . . . . . . . . . . . . . 5 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 5 7. Informative References . . . . . . . . . . . . . . . . . . . 5 8. Author's Address . . . . . . . . . . . . . . . . . . . . . . . 5 Intellectual Property Statement . . . . . . . . . . . . . . . . 6 Full Copyright Statement . . . . . . . . . . . . . . . . . . . . 6 Disclaimer . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1. Introduction The Lemonade WG is looking at ways to improve IMAP to permit better performance when running on platforms with constrained resources, or communications links with high latency or limited bandwidth. This document presents statistics on IMAP network utilization followed by simulation results for dictionary based compression for IMAP commands and transport layer compression. Both techniques have been mentioned earlier in the WG. 2. Traffic for a few IMAP use-cases In order to get a feel of where IMAP can be optimized for the kind of environments mentioned above, we collect some statistics on network usage for a few IMAP transactions. 2.1 Use-cases description We consider the following use-cases: reading a message of about 1KB, searching in cyrus.andrew.cmu.edu for messages containing 'steve' and copying a message. Naturally the numbers presented depend on the size of the messages and the number of matches in the IMAP mailbox; we hope to have picked representative scenarios. For these experiments, we sniffed the communication between a Cyrus 1.5.2 IMAP server and a Mulberry 3.1.6 client using Ethereal. Mulberry is running on Windows XP, Cyrus on HP-UX. The machines are connected with Ethernet 100Mbps. In each direction, we count the a) bytes at Ethernet level b) IMAP commands (client to server) and responses (server toclient). That is all bytes created by the IMAP protocol, like commands, Meylan Expires April 2005 [Page 2] Internet Draft IMAP statistics for Lemonade October 2004 modifiers or tags. c) fraction IMAP bytes over the total number of bytes d) IP packets used for the transaction The values are presented with the format server>/client> Use Case a) b) c) d) Reading 638/6216 178/512 0.28/0.08 8/8 Copying 91/135 37/21 0.40/0.16 1/2 Searching 565/9155 187/72 0.33/0.01 7/8 2.2 A few comments In the client to server direction, the IMAP bytes typically represent a large fraction of the total traffic. In mobile environments, the client to server is referred to as the uplink and usually has a lower bandwidth than the reverse direction (downlink). Therefore, reducing the size of IMAP commands would really help the slower uplink. With this setup, TCP was a bandwidth waster; often shortly after an IMAP request the TCP segment used for the request is Ack'ed, sending 54B on the wire. Then a few milliseconds later the IMAP Response is sent and contains the same TCP Ack information. Using delayed TCP Acks on the IMAP sever would solve this issue. Alternatively a proxy could throw the too frequent TCP Acks from the server away in order to save bandwidth. 3. IMAP command dictionary based compression A proposal in Lemonade consists in reducing the size of the IMAP commands. Let us pick an example scenario where a client sends the following set of commands : FETCH, STORE, SEARCH, STORE, COPY. We assume that every IMAP Command is compressed to 1 byte and all modifiers (FETCH Data items, SEARCH Keys) are reduced to 2 bytes thanks to a dictionary based compression scheme. The following lists IMAP uplink traffic with the (original message size / compressed size) A00007 FETCH 1:11 (FLAGS RFC822.SIZE UID INTERNALDATE ENVELOPE BODYSTRUCTURE) (80 B/ 45B) A00013 STORE 3 +FLAGS (\Deleted) (34B/ 27B) A00015 SEARCH FROM steve TEXT quota\r\n (38B/ 29B) A00013 STORE 3 +FLAGS (\Deleted) (35B/ 28B) A00009 COPY 2 INBOX.single_graphics\r\n (38B / 33B) Meylan Expires April 2005 [Page 3] Internet Draft IMAP statistics for Lemonade October 2004 By applying the described scheme the overall IAMP request traffic would drop from 225 to 162 bytes, which is a 28% reduction on the uplink. We expect that reduction to hold for longer IMAP sessions where the user mostly browses messages because the uplink traffic remains mostly IMAP commands. On the downlink or in case the user sends messages or attachments, the reduction depends on the type of content being transferred. But in general, one should not expect a big gain. Indeed the size of the IMAP message content dominates over the size of the IMAP protocol bytes. Naturally, the gain would come at the cost of additional IMAP functionalities. 4. IMAP with TLS featuring compression It has also been proposed to combat IMAP's chattiness by using compression at the transport layer. This method presents the advantage of being compatible with the current version of IMAP. One considers a transport layer protocol capable of compressing the data end-to-end. For example OpenSSL features compression method that uses zlib[3]. 4.1 TLS use-cases We use the same setup as mentioned in section 2.1 in which the IMAP server does not support Binary Extension. Therefore base 64 encoding is used for attachments. In order to evaluate the gains provided by adding compression to the transport layer, one captures a complete IMAP stream (tcpdump -w ) and compresses it with gzip --fast. Using the --fast option is an attempt to mimic the behavior of a stream compression algorithm: it might run on a small CPU and has to go fast, potentially at the cost of limited compression performance. One considers three IMAP use-cases: 1) Browsing a collection of text messages (read about 100 messages from an IMAP mailing list on cyrus.andrew.cmu.edu) 2) Read a few emails and download an MS Word document that is not compressed (3.87MB, 148-page document with 47 figures) 3) Read a few emails and download three JPEG pictures (3.8Mpixels, 24bits RGB about 2MB per picture) The table below presents a) File size: size of the attached files to download from the IMAP server b) Gzipped file size: size of the files compressed with gzip -fast Meylan Expires April 2005 [Page 4] Internet Draft IMAP statistics for Lemonade October 2004 c) IMAP session: size of the IMAP session required to retrieve the messages and their attached files d) Compressed IMAP session: size of a hypothetical IMAP session that uses TLS with gzip --fast to compress the stream. e) Compression ratio: ratio of a raw IAMP session over a compressed IMAP session. Use case a) b) c) d) e) 1) n/a n/a 0.364 0.111 3.28:1 2) 3.87 1.20 5.68 2.02 2.81:1 3) 5.97 5.95 8.77 6.56 1.34:1 4.2 Observations The compression for text messages is good, comparable to the expected compression ratio for text [1]. When downloading the un-compressed MS word document, the compression ratio remains pretty high even though this particular document contains many figures that might prevent higher compression ratio. It is worth running a stream compression whenever a base64 encoding will be used. Though the encoding consistently adds about 1/3 [2] to the size of the input data, stream compression will return much of this overhead, even for compressed attachments like JPEG images. On the other hand stream compression will likely not provide any gain if IMAP's binary extension is used for compressed attachments. 5. Security Considerations This draft presents experimental results; it does not currently suggest any protocol changes, and hence no alterations to the existing security aspects/risks are proposed. It may be worth noting that one advantage of a TLS-based compression may be higher use of TLS with server certificates. 6. IANA Considerations This draft presents experimental results; it does not currently suggest any protocol changes, and hence no requests are being made on IANA. 7. Informative References [1] http://www.cs.waikato.ac.nz/~singlis/ratios.html [2] http://www.mhonarc.org/~ehood/MIME/2045/rfc2045.html#6.8 [3] http://www.openssl.org/docs/ssl/... SSL_COMP_add_compression_method.html Meylan Expires April 2005 [Page 5] Internet Draft IMAP statistics for Lemonade October 2004 8. Author's Address Arnaud Meylan QUALCOMM Incorporated 5775 Morehouse Dr. San Diego, CA 92121 USA ameylan@qualcomm.com Intellectual Property Statement The IETF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on the IETF's procedures with respect to rights in standards-track and standards-related documentation can be found in BCP-11. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementors or users of this specification can be obtained from the IETF Secretariat. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this standard. Please address the information to the IETF Executive Director. Full Copyright Statement Copyright (C) The Internet Society (2004). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Meylan Expires April 2005 [Page 6] Internet Draft IMAP statistics for Lemonade October 2004 Disclaimer This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Meylan Expires April 2005 [Page 7]