LEMONADE
Internet Draft: IMAP statistics for Lemonade A. Meylan
Document: draft-meylan-imap-statistics-lemonade-00.txt QUALCOMM
Expires: April 2005 October 2004
IMAP statistics for Lemonade
Status of this Memo
By submitting this Internet-Draft, I certify that any applicable
patent or other IPR claims of which I am aware have been disclosed
and any of which I become aware will be disclosed, in accordance
with RFC 3668 (BCP 79).
By submitting this Internet-Draft, I accept the provisions of
Section 3 of RFC 3667 (BCP 78).
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt The list of
Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Copyright Notice
Copyright (C) The Internet Society (2004). All Rights Reserved.
Abstract
This memo presents statistics on IMAP's network utilization for some
common use cases. In addition we evaluate the performance of
dictionary based command compression as well as transport layer
compression for IMAP traffic.
Meylan Expires April 2005 [Page 1]
Internet Draft IMAP statistics for Lemonade October 2004
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Traffic for a few IMAP use-cases . . . . . . . . . . . . . . . 2
2.1 Use-cases description . . . . . . . . . . . . . . . . . . 2
2.2 A few comments . . . . . . . . . . . . . . . . . . . . . . 3
3. IMAP command dictionary based compression . . . . . . . . . . 3
4. IMAP with TLS featuring compression . . . . . . . . . . . . . 4
4.1 TLS use-cases . . . . . . . . . . . . . . . . . . . . . . 4
4.2 Observations . . . . . . . . . . . . . . . . . . . . . . . 5
5. Security Considerations . . . . . . . . . . . . . . . . . . . 5
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 5
7. Informative References . . . . . . . . . . . . . . . . . . . 5
8. Author's Address . . . . . . . . . . . . . . . . . . . . . . . 5
Intellectual Property Statement . . . . . . . . . . . . . . . . 6
Full Copyright Statement . . . . . . . . . . . . . . . . . . . . 6
Disclaimer . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1. Introduction
The Lemonade WG is looking at ways to improve IMAP to permit better
performance when running on platforms with constrained resources, or
communications links with high latency or limited bandwidth. This
document presents statistics on IMAP network utilization followed by
simulation results for dictionary based compression for IMAP
commands and transport layer compression. Both techniques have been
mentioned earlier in the WG.
2. Traffic for a few IMAP use-cases
In order to get a feel of where IMAP can be optimized for the kind
of environments mentioned above, we collect some statistics on
network usage for a few IMAP transactions.
2.1 Use-cases description
We consider the following use-cases: reading a message of about 1KB,
searching in cyrus.andrew.cmu.edu for messages containing 'steve'
and copying a message. Naturally the numbers presented depend on
the size of the messages and the number of matches in the IMAP
mailbox; we hope to have picked representative scenarios.
For these experiments, we sniffed the communication between a Cyrus
1.5.2 IMAP server and a Mulberry 3.1.6 client using Ethereal.
Mulberry is running on Windows XP, Cyrus on HP-UX. The machines are
connected with Ethernet 100Mbps. In each direction, we count the
a) bytes at Ethernet level
b) IMAP commands (client to server) and responses (server toclient).
That is all bytes created by the IMAP protocol, like commands,
Meylan Expires April 2005 [Page 2]
Internet Draft IMAP statistics for Lemonade October 2004
modifiers or tags.
c) fraction IMAP bytes over the total number of bytes
d) IP packets used for the transaction
The values are presented with the format
<direction client->server>/<direction server->client>
Use Case a) b) c) d)
Reading 638/6216 178/512 0.28/0.08 8/8
Copying 91/135 37/21 0.40/0.16 1/2
Searching 565/9155 187/72 0.33/0.01 7/8
2.2 A few comments
In the client to server direction, the IMAP bytes typically
represent a large fraction of the total traffic. In mobile
environments, the client to server is referred to as the uplink and
usually has a lower bandwidth than the reverse direction (downlink).
Therefore, reducing the size of IMAP commands would really help the
slower uplink.
With this setup, TCP was a bandwidth waster; often shortly after an
IMAP request the TCP segment used for the request is Ack'ed, sending
54B on the wire. Then a few milliseconds later the IMAP Response is
sent and contains the same TCP Ack information. Using delayed TCP
Acks on the IMAP sever would solve this issue. Alternatively a
proxy could throw the too frequent TCP Acks from the server away in
order to save bandwidth.
3. IMAP command dictionary based compression
A proposal in Lemonade consists in reducing the size of the IMAP
commands. Let us pick an example scenario where a client sends the
following set of commands : FETCH, STORE, SEARCH, STORE, COPY. We
assume that every IMAP Command is compressed to 1 byte and all
modifiers (FETCH Data items, SEARCH Keys) are reduced to 2 bytes
thanks to a dictionary based compression scheme.
The following lists IMAP uplink traffic with the (original message
size / compressed size)
A00007 FETCH 1:11 (FLAGS RFC822.SIZE UID
INTERNALDATE ENVELOPE BODYSTRUCTURE) (80 B/ 45B)
A00013 STORE 3 +FLAGS (\Deleted) (34B/ 27B)
A00015 SEARCH FROM steve TEXT quota\r\n (38B/ 29B)
A00013 STORE 3 +FLAGS (\Deleted) (35B/ 28B)
A00009 COPY 2 INBOX.single_graphics\r\n (38B / 33B)
Meylan Expires April 2005 [Page 3]
Internet Draft IMAP statistics for Lemonade October 2004
By applying the described scheme the overall IAMP request traffic
would drop from 225 to 162 bytes, which is a 28% reduction on the
uplink. We expect that reduction to hold for longer IMAP sessions
where the user mostly browses messages because the uplink traffic
remains mostly IMAP commands.
On the downlink or in case the user sends messages or attachments,
the reduction depends on the type of content being transferred. But
in general, one should not expect a big gain. Indeed the size of
the IMAP message content dominates over the size of the IMAP
protocol bytes.
Naturally, the gain would come at the cost of additional IMAP
functionalities.
4. IMAP with TLS featuring compression
It has also been proposed to combat IMAP's chattiness by using
compression at the transport layer. This method presents the
advantage of being compatible with the current version of IMAP. One
considers a transport layer protocol capable of compressing the data
end-to-end. For example OpenSSL features compression method that
uses zlib[3].
4.1 TLS use-cases
We use the same setup as mentioned in section 2.1 in which the IMAP
server does not support Binary Extension. Therefore base 64
encoding is used for attachments. In order to evaluate the gains
provided by adding compression to the transport layer, one captures
a complete IMAP stream (tcpdump -w <out_file>) and compresses it
with gzip --fast. Using the --fast option is an attempt to mimic
the behavior of a stream compression algorithm: it might run on a
small CPU and has to go fast, potentially at the cost of limited
compression performance.
One considers three IMAP use-cases:
1) Browsing a collection of text messages (read about 100 messages
from an IMAP mailing list on cyrus.andrew.cmu.edu)
2) Read a few emails and download an MS Word document that is not
compressed (3.87MB, 148-page document with 47 figures)
3) Read a few emails and download three JPEG pictures
(3.8Mpixels, 24bits RGB about 2MB per picture)
The table below presents
a) File size: size of the attached files to download from the IMAP
server
b) Gzipped file size: size of the files compressed with gzip -fast
Meylan Expires April 2005 [Page 4]
Internet Draft IMAP statistics for Lemonade October 2004
c) IMAP session: size of the IMAP session required to retrieve the
messages and their attached files
d) Compressed IMAP session: size of a hypothetical IMAP session that
uses TLS with gzip --fast to compress the stream.
e) Compression ratio: ratio of a raw IAMP session over a
compressed IMAP session.
Use case a) b) c) d) e)
1) n/a n/a 0.364 0.111 3.28:1
2) 3.87 1.20 5.68 2.02 2.81:1
3) 5.97 5.95 8.77 6.56 1.34:1
4.2 Observations
The compression for text messages is good, comparable to the
expected compression ratio for text [1].
When downloading the un-compressed MS word document, the compression
ratio remains pretty high even though this particular document
contains many figures that might prevent higher compression ratio.
It is worth running a stream compression whenever a base64 encoding
will be used. Though the encoding consistently adds about 1/3 [2]
to the size of the input data, stream compression will return much
of this overhead, even for compressed attachments like JPEG images.
On the other hand stream compression will likely not provide any
gain if IMAP's binary extension is used for compressed attachments.
5. Security Considerations
This draft presents experimental results; it does not currently
suggest any protocol changes, and hence no alterations to the
existing security aspects/risks are proposed. It may be worth
noting that one advantage of a TLS-based compression may be higher
use of TLS with server certificates.
6. IANA Considerations
This draft presents experimental results; it does not currently
suggest any protocol changes, and hence no requests are being made
on IANA.
7. Informative References
[1] http://www.cs.waikato.ac.nz/~singlis/ratios.html
[2] http://www.mhonarc.org/~ehood/MIME/2045/rfc2045.html#6.8
[3] http://www.openssl.org/docs/ssl/...
SSL_COMP_add_compression_method.html
Meylan Expires April 2005 [Page 5]
Internet Draft IMAP statistics for Lemonade October 2004
8. Author's Address
Arnaud Meylan
QUALCOMM Incorporated
5775 Morehouse Dr.
San Diego, CA
92121
USA
ameylan@qualcomm.com
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
intellectual property or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; neither does it represent that it
has made any effort to identify any such rights. Information on the
IETF's procedures with respect to rights in standards-track and
standards-related documentation can be found in BCP-11. Copies of
claims of rights made available for publication and any assurances
of licenses to be made available, or the result of an attempt made
to obtain a general license or permission for the use of such
proprietary rights by implementors or users of this specification
can be obtained from the IETF Secretariat.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights which may cover technology that may be required to practice
this standard. Please address the information to the IETF Executive
Director.
Full Copyright Statement
Copyright (C) The Internet Society (2004).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
Meylan Expires April 2005 [Page 6]
Internet Draft IMAP statistics for Lemonade October 2004
Disclaimer
This document and the information contained herein are provided on
an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE
INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Meylan Expires April 2005 [Page 7]