--- /dev/null
+
+
+
+
+
+
+Network Working Group A. Gulbrandsen
+Request for Comments: 4978 Oryx Mail Systems GmbH
+Category: Standards Track August 2007
+
+
+ The IMAP COMPRESS Extension
+
+Status of this Memo
+
+ This document specifies an Internet standards track protocol for the
+ Internet community, and requests discussion and suggestions for
+ improvements. Please refer to the current edition of the "Internet
+ Official Protocol Standards" (STD 1) for the standardization state
+ and status of this protocol. Distribution of this memo is unlimited.
+
+Abstract
+
+ The COMPRESS extension allows an IMAP connection to be effectively
+ and efficiently compressed.
+
+ Table of Contents
+
+ 1. Introduction and Overview .......................................2
+ 2. Conventions Used in This Document ...............................2
+ 3. The COMPRESS Command ............................................3
+ 4. Compression Efficiency ..........................................4
+ 5. Formal Syntax ...................................................6
+ 6. Security Considerations .........................................6
+ 7. IANA Considerations .............................................6
+ 8. Acknowledgements ................................................7
+ 9. References ......................................................7
+ 9.1. Normative References .......................................7
+ 9.2. Informative References .....................................7
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Gulbrandsen Standards Track [Page 1]
+\f
+RFC 4978 The IMAP COMPRESS Extension August 2007
+
+
+1. Introduction and Overview
+
+ A server which supports the COMPRESS extension indicates this with
+ one or more capability names consisting of "COMPRESS=" followed by a
+ supported compression algorithm name as described in this document.
+
+ The goal of COMPRESS is to reduce the bandwidth usage of IMAP.
+
+ Compared to PPP compression (see [RFC1962]) and modem-based
+ compression (see [MNP] and [V42BIS]), COMPRESS offers much better
+ compression efficiency. COMPRESS can be used together with Transport
+ Security Layer (TLS) [RFC4346], Simple Authentication and Security
+ layer (SASL) encryption, Virtual Private Networks (VPNs), etc.
+ Compared to TLS compression [RFC3749], COMPRESS has the following
+ (dis)advantages:
+
+ - COMPRESS can be implemented easily both by IMAP servers and
+ clients.
+
+ - IMAP COMPRESS benefits from an intimate knowledge of the IMAP
+ protocol's state machine, allowing for dynamic and aggressive
+ optimization of the underlying compression algorithm's parameters.
+
+ - When the TLS layer implements compression, any protocol using that
+ layer can transparently benefit from that compression (e.g., SMTP
+ and IMAP). COMPRESS is specific to IMAP.
+
+ In order to increase interoperation, it is desirable to have as few
+ different compression algorithms as possible, so this document
+ specifies only one. The DEFLATE algorithm (defined in [RFC1951]) is
+ standard, widely available and fairly efficient, so it is the only
+ algorithm defined by this document.
+
+ In order to increase interoperation, IMAP servers that advertise this
+ extension SHOULD also advertise the TLS DEFLATE compression mechanism
+ as defined in [RFC3749]. IMAP clients MAY use either COMPRESS or TLS
+ compression, however, if the client and server support both, it is
+ RECOMMENDED that the client choose TLS compression.
+
+ The extension adds one new command (COMPRESS) and no new responses.
+
+2. Conventions Used in This Document
+
+ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
+ "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
+ document are to be interpreted as described in [RFC2119].
+
+ Formal syntax is defined by [RFC4234] as modified by [RFC3501].
+
+
+
+Gulbrandsen Standards Track [Page 2]
+\f
+RFC 4978 The IMAP COMPRESS Extension August 2007
+
+
+ In the examples, "C:" and "S:" indicate lines sent by the client and
+ server respectively. "[...]" denotes elision.
+
+3. The COMPRESS Command
+
+ Arguments: Name of compression mechanism: "DEFLATE".
+
+ Responses: None
+
+ Result: OK The server will compress its responses and expects the
+ client to compress its commands.
+ NO Compression is already active via another layer.
+ BAD Command unknown, invalid or unknown argument, or COMPRESS
+ already active.
+
+ The COMPRESS command instructs the server to use the named
+ compression mechanism ("DEFLATE" is the only one defined) for all
+ commands and/or responses after COMPRESS.
+
+ The client MUST NOT send any further commands until it has seen the
+ result of COMPRESS. If the response was OK, the client MUST compress
+ starting with the first command after COMPRESS. If the server
+ response was BAD or NO, the client MUST NOT turn on compression.
+
+ If the server responds NO because it knows that the same mechanism is
+ active already (e.g., because TLS has negotiated the same mechanism),
+ it MUST send COMPRESSIONACTIVE as resp-text-code (see [RFC3501],
+ Section 7.1), and the resp-text SHOULD say which layer compresses.
+
+ If the server issues an OK response, the server MUST compress
+ starting immediately after the CRLF which ends the tagged OK
+ response. (Responses issued by the server before the OK response
+ will, of course, still be uncompressed.) If the server issues a BAD
+ or NO response, the server MUST NOT turn on compression.
+
+ For DEFLATE (as for many other compression mechanisms), the
+ compressor can trade speed against quality. When decompressing there
+ isn't much of a tradeoff. Consequently, the client and server are
+ both free to pick the best reasonable rate of compression for the
+ data they send.
+
+ When COMPRESS is combined with TLS (see [RFC4346]) or SASL (see
+ [RFC4422]) security layers, the sending order of the three extensions
+ MUST be first COMPRESS, then SASL, and finally TLS. That is, before
+ data is transmitted it is first compressed. Second, if a SASL
+ security layer has been negotiated, the compressed data is then
+ signed and/or encrypted accordingly. Third, if a TLS security layer
+ has been negotiated, the data from the previous step is signed and/or
+
+
+
+Gulbrandsen Standards Track [Page 3]
+\f
+RFC 4978 The IMAP COMPRESS Extension August 2007
+
+
+ encrypted accordingly. When receiving data, the processing order
+ MUST be reversed. This ensures that before sending, data is
+ compressed before it is encrypted, independent of the order in which
+ the client issues COMPRESS, AUTHENTICATE, and STARTTLS.
+
+ The following example illustrates how commands and responses are
+ compressed during a simple login sequence:
+
+ S: * OK [CAPABILITY IMAP4REV1 STARTTLS COMPRESS=DEFLATE]
+ C: a starttls
+ S: a OK TLS active
+
+ From this point on, everything is encrypted.
+
+ C: b login arnt tnra
+ S: b OK Logged in as arnt
+ C: c compress deflate
+ S: d OK DEFLATE active
+
+ From this point on, everything is compressed before being
+ encrypted.
+
+ The following example demonstrates how a server may refuse to
+ compress twice:
+
+ S: * OK [CAPABILITY IMAP4REV1 STARTTLS COMPRESS=DEFLATE]
+ [...]
+ C: c compress deflate
+ S: c NO [COMPRESSIONACTIVE] DEFLATE active via TLS
+
+4. Compression Efficiency
+
+ This section is informative, not normative.
+
+ IMAP poses some unusual problems for a compression layer.
+
+ Upstream is fairly simple. Most IMAP clients send the same few
+ commands again and again, so any compression algorithm that can
+ exploit repetition works efficiently. The APPEND command is an
+ exception; clients that send many APPEND commands may want to
+ surround large literals with flushes in the same way as is
+ recommended for servers later in this section.
+
+ Downstream has the unusual property that several kinds of data are
+ sent, confusing all dictionary-based compression algorithms.
+
+
+
+
+
+
+Gulbrandsen Standards Track [Page 4]
+\f
+RFC 4978 The IMAP COMPRESS Extension August 2007
+
+
+ One type is IMAP responses. These are highly compressible; zlib
+ using its least CPU-intensive setting compresses typical responses to
+ 25-40% of their original size.
+
+ Another type is email headers. These are equally compressible, and
+ benefit from using the same dictionary as the IMAP responses.
+
+ A third type is email body text. Text is usually fairly short and
+ includes much ASCII, so the same compression dictionary will do a
+ good job here, too. When multiple messages in the same thread are
+ read at the same time, quoted lines etc. can often be compressed
+ almost to zero.
+
+ Finally, attachments (non-text email bodies) are transmitted, either
+ in binary form or encoded with base-64.
+
+ When attachments are retrieved in binary form, DEFLATE may be able to
+ compress them, but the format of the attachment is usually not IMAP-
+ like, so the dictionary built while compressing IMAP does not help.
+ The compressor has to adapt its dictionary from IMAP to the
+ attachment's format, and then back. A few file formats aren't
+ compressible at all using deflate, e.g., .gz, .zip, and .jpg files.
+
+ When attachments are retrieved in base-64 form, the same problems
+ apply, but the base-64 encoding adds another problem. 8-bit
+ compression algorithms such as deflate work well on 8-bit file
+ formats, however base-64 turns a file into something resembling 6-bit
+ bytes, hiding most of the 8-bit file format from the compressor.
+
+ When using the zlib library (see [RFC1951]), the functions
+ deflateInit2(), deflate(), inflateInit2(), and inflate() suffice to
+ implement this extension. The windowBits value must be in the range
+ -8 to -15, or else deflateInit2() uses the wrong format.
+ deflateParams() can be used to improve compression rate and resource
+ use. The Z_FULL_FLUSH argument to deflate() can be used to clear the
+ dictionary (the receiving peer does not need to do anything).
+
+ A client can improve downstream compression by implementing BINARY
+ (defined in [RFC3516]) and using FETCH BINARY instead of FETCH BODY.
+ In the author's experience, the improvement ranges from 5% to 40%
+ depending on the attachment being downloaded.
+
+ A server can improve downstream compression if it hints to the
+ compressor that the data type is about to change strongly, e.g., by
+ sending a Z_FULL_FLUSH at the start and end of large non-text
+ literals (before and after '*CHAR8' in the definition of literal in
+ RFC 3501, page 86). Small literals are best left alone. A possible
+ boundary is 5k.
+
+
+
+Gulbrandsen Standards Track [Page 5]
+\f
+RFC 4978 The IMAP COMPRESS Extension August 2007
+
+
+ A server can improve the CPU efficiency both of the server and the
+ client if it adjusts the compression level (e.g., using the
+ deflateParams() function in zlib) at these points, to avoid trying to
+ compress incompressible attachments. A very simple strategy is to
+ change the level to 0 at the start of a literal provided the first
+ two bytes are either 0x1F 0x8B (as in deflate-compressed files) or
+ 0xFF 0xD8 (JPEG), and to keep it at 1-5 the rest of the time. More
+ complex strategies are possible.
+
+5. Formal Syntax
+
+ The following syntax specification uses the Augmented Backus-Naur
+ Form (ABNF) notation as specified in [RFC4234]. This syntax augments
+ the grammar specified in [RFC3501]. [RFC4234] defines SP and
+ [RFC3501] defines command-auth, capability, and resp-text-code.
+
+ Except as noted otherwise, all alphabetic characters are case-
+ insensitive. The use of upper or lower case characters to define
+ token strings is for editorial clarity only. Implementations MUST
+ accept these strings in a case-insensitive fashion.
+
+ command-auth =/ compress
+
+ compress = "COMPRESS" SP algorithm
+
+ capability =/ "COMPRESS=" algorithm
+ ;; multiple COMPRESS capabilities allowed
+
+ algorithm = "DEFLATE"
+
+ resp-text-code =/ "COMPRESSIONACTIVE"
+
+ Note that due the syntax of capability names, future algorithm names
+ must be atoms.
+
+6. Security Considerations
+
+ As for TLS compression [RFC3749].
+
+7. IANA Considerations
+
+ The IANA has added COMPRESS=DEFLATE to the list of IMAP capabilities.
+
+
+
+
+
+
+
+
+
+Gulbrandsen Standards Track [Page 6]
+\f
+RFC 4978 The IMAP COMPRESS Extension August 2007
+
+
+8. Acknowledgements
+
+ Eric Burger, Dave Cridland, Tony Finch, Ned Freed, Philip Guenther,
+ Randall Gellens, Tony Hansen, Cullen Jennings, Stephane Maes, Alexey
+ Melnikov, Lyndon Nerenberg, and Zoltan Ordogh have all helped with
+ this document.
+
+ The author would also like to thank various people in the rooms at
+ meetings, whose help is real, but not reflected in the author's
+ mailbox.
+
+9. References
+
+9.1. Normative References
+
+ [RFC1951] Deutsch, P., "DEFLATE Compressed Data Format Specification
+ version 1.3", RFC 1951, May 1996.
+
+ [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
+ Requirement Levels", BCP 14, RFC 2119, March 1997.
+
+ [RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION
+ 4rev1", RFC 3501, March 2003.
+
+ [RFC4234] Crocker, D. and P. Overell, "Augmented BNF for Syntax
+ Specifications: ABNF", RFC 4234, October 2005.
+
+9.2. Informative References
+
+ [RFC1962] Rand, D., "The PPP Compression Control Protocol (CCP)",
+ RFC 1962, June 1996.
+
+ [RFC3516] Nerenberg, L., "IMAP4 Binary Content Extension", RFC 3516,
+ April 2003.
+
+ [RFC3749] Hollenbeck, S., "Transport Layer Security Protocol
+ Compression Methods", RFC 3749, May 2004.
+
+ [RFC4346] Dierks, T. and E. Rescorla, "The Transport Layer Security
+ (TLS) Protocol Version 1.1", RFC 4346, April 2006.
+
+ [RFC4422] Melnikov, A. and K. Zeilenga, "Simple Authentication and
+ Security Layer (SASL)", RFC 4422, June 2006.
+
+ [V42BIS] ITU, "V.42bis: Data compression procedures for data
+ circuit-terminating equipment (DCE) using error correction
+ procedures", http://www.itu.int/rec/T-REC-V.42bis, January
+ 1990.
+
+
+
+Gulbrandsen Standards Track [Page 7]
+\f
+RFC 4978 The IMAP COMPRESS Extension August 2007
+
+
+ [MNP] Gilbert Held, "The Complete Modem Reference", Second
+ Edition, Wiley Professional Computing, ISBN 0-471-00852-4,
+ May 1994.
+
+Author's Address
+
+ Arnt Gulbrandsen
+ Oryx Mail Systems GmbH
+ Schweppermannstr. 8
+ D-81671 Muenchen
+ Germany
+
+ Fax: +49 89 4502 9758
+ EMail: arnt@oryx.com
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Gulbrandsen Standards Track [Page 8]
+\f
+RFC 4978 The IMAP COMPRESS Extension August 2007
+
+
+Full Copyright Statement
+
+ Copyright (C) The IETF Trust (2007).
+
+ This document is subject to the rights, licenses and restrictions
+ contained in BCP 78, and except as set forth therein, the authors
+ retain all their rights.
+
+ This document and the information contained herein are provided on an
+ "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
+ OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
+ THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
+ OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
+ THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
+ WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
+
+Intellectual Property
+
+ The IETF takes no position regarding the validity or scope of any
+ Intellectual Property Rights or other rights that might be claimed to
+ pertain to the implementation or use of the technology described in
+ this document or the extent to which any license under such rights
+ might or might not be available; nor does it represent that it has
+ made any independent effort to identify any such rights. Information
+ on the procedures with respect to rights in RFC documents can be
+ found in BCP 78 and BCP 79.
+
+ Copies of IPR disclosures made to the IETF Secretariat and any
+ assurances of licenses to be made available, or the result of an
+ attempt made to obtain a general license or permission for the use of
+ such proprietary rights by implementers or users of this
+ specification can be obtained from the IETF on-line IPR repository at
+ http://www.ietf.org/ipr.
+
+ The IETF invites any interested party to bring to its attention any
+ copyrights, patents or patent applications, or other proprietary
+ rights that may cover technology that may be required to implement
+ this standard. Please address the information to the IETF at
+ ietf-ipr@ietf.org.
+
+
+
+
+
+
+
+
+
+
+
+
+Gulbrandsen Standards Track [Page 9]
+\f
projects / uw-imap / commitdiff