Internet Draft: Qpopper MIME Mangling and Macro
Extensions to POP3 R. Gellens
Document: draft-gellens-pop-mangle-02.txt Qualcomm
Expires: May 17, 2009 November 17, 2008
Intended status: Informational
The Qpopper MIME Mangling and Macro Extensions to POP3
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with
the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt The list of
Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Copyright Notice
Copyright (c) 2008 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with
respect to this document.
Abstract
This document describes two extensions to the POP protocol that have
been supported for several years by some client and servers. One
extension, MANGLE, allows clients to request that MIME body parts be
removed or converted to a simpler format, that only selected headers
Gellens Expires January 2008 [Page 1]
Internet Draft Mangle and Macro July 2007
be transmitted, and/or for the message to be transcoded to a
different character set. The other extension, MACRO, allows clients
to define macros which are expanded when used, saving repetitive
transmissions.
The MANGLE extension has been useful in at least two situations: it
allows clients on constrained devices to avoid downloading body
parts which cannot be used on the device, and clients which use the
TOP command to "peek" at messages can have the preview transformed
into more usable content, as well as avoiding the transmission of
undesired headers. The MACRO extension has been especially useful
in conjunction with MANGLE, since a MANGLE request can become
relatively lengthy.
Gellens Expires January 2008 [Page 2]
Internet Draft Mangle and Macro July 2007
Table of Contents
1 Conventions Used in this Document . . . . . . . . . . . . . . 3
2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
3. MIME Mangling (MANGLE) . . . . . . . . . . . . . . . . . . . 4
3.1. MANGLE Capability . . . . . . . . . . . . . . . . . . . 6
4. Macro Definition (MDEF) . . . . . . . . . . . . . . . . . . . 7
4.1. MACRO Capability . . . . . . . . . . . . . . . . . . . . 8
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9
6. Security Considerations . . . . . . . . . . . . . . . . . . 9
7. Normative References . . . . . . . . . . . . . . . . . . . . 9
8. Informative References . . . . . . . . . . . . . . . . . . . 10
9. Author's Address . . . . . . . . . . . . . . . . . . . . . . 10
1 Conventions Used in this Document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [KEYWORDS].
2. Introduction
Since May of 2000, the Qpopper open-source POP server has supported
the MIME mangling (MANGLE) and macro (MACRO) extensions to the
[POP3] protocol.
These extensions have been useful in at least two situations:
o clients on constrained devices can avoid downloading body parts
which cannot be used on the device, can have the content transcoded
into a more desirable character set, and can have undesired headers
removed prior to transmission; and
o clients which use the TOP command to "peek" at messages can
have the preview transformed into more usable content, and can make
the preview more useful by eliminating extraneous headers.
The Eudora for PalmOS client (part of the Eudora Internet Suite for
PalmOS) makes use of both extensions.
The Eudora clients for both MacOS and Windows make use of the MANGLE
extension when using the TOP command.
Gellens Expires January 2008 [Page 3]
Internet Draft Mangle and Macro July 2007
With the prevalence of attachments and formatted mail, these
extensions have been useful in solving the following problems:
o clients on constrained devices or links often download
attachments that waste bandwidth and must be parsed and discarded,
since they cannot be used on the device;
o clients on slow links or memory-limited devices waste bandwidth
receiving undesired headers;
o when the TOP command is used to "peek" at messages, especially
when using a bandwidth-constrained connection (such as a non-3G
mobile link or a slow dialup with a laptop), often the captured
portion of the message is unreadable, being a mess of HTML tags
(since many clients show these previews in their raw form, and,
being truncated, the HTML may not be syntactically valid and hence
difficult for the client to parse).
Obviously, IMAP [IMAP] provides a more feature-packed protocol and
hence allows for a richer user experience. However, IMAP also
imposes significant additional costs on server operators (for
example, since with IMAP the server maintains the authoritative mail
store, the storage requirements are typically much greater than with
POP). In general, the extra costs of IMAP are far outweighed by the
additional benefits, and hence IMAP is the recommended mechanism for
greater functionality, but there are cases where the extra costs
cannot be justified or recovered. In those cases, minor extensions
to POP to provide additional value to mobile devices may be a useful
interim measure until IMAP can be deployed.
3. MIME Mangling (MANGLE)
This extension is announced by including MANGLE in the CAPA
[POP-EXT] response.
The MANGLE extension can be used as a modifier to the TOP, RETR, and
LIST commands. The result is to:
o Remove MIME body parts
o Optionally remove all headers except those requested
o Optionally encode in a specified character set
The syntax is:
mangle = "mangle(" mangling ")"
mangling = attribute "=" value *(";" attribute "=" value )
attribute = "text" / "headers" / "charset" / "lang" / "wrap"
value = text-type / header-list / charset
Gellens Expires January 2008 [Page 4]
Internet Draft Mangle and Macro July 2007
text-type = "plain" / "html" / "enriched"
header-list = header *("," header)
charset = "us-ascii" / iso-8859-r / "utf-8" / "cp1252" /
"iso_2022-jp"
iso-8859-r = "iso-8859-" range
range = "1"-"15"
The following command results in transmitting message 10 as a single
part of content-type text/html with only those headers which were
requested:
retr 10 mangle(text=html;headers=to:,cc:,from:,date:)
The following command asks the server for the size of message 2
after mangling as a single part of content-type text/html with only
those headers which were requested:
list 2 mangle(text=html;headers=to:,cc:,from:,date:)
The message can be sent as one of three text types: text/plain,
text/html, and text/enriched. The source format can be any of these
types or text/plain;format=flowed.
If the headers are not specified, the default is to return the
following ones: "to:", "cc:", "subject:", "from:", "date:",
"reply-to:", "x-Priority:", and "x-UIDL:".
Note that "lang" and "wrap" are currently ignored.
The value supplied with the "headers" attribute is a comma-separated
list of desired headers. Header in the message are matched up to
what is entered, so "headers=sub" will cause both the "Subject:" and
"Subversive:" headers to be returned, if both exist in the message.
In order to be sure to match only the exact header desired, clients
may wish to include the ":". The "Mime-Version:" and
"Content-Type:" headers are always included, and reflect the
mangling that has been applied.
Note that there must not be any spaces within the 'mangle' string
(since spaces are used to separate parameters to POP commands).
Case is not significant with the MANGLE extension.
'mangle(text=plain)' is the same as 'mAnglE(TeXT=PLaiN)',
'headers=to' is the same as 'headers=TO'.
Examples:
top 1 100 mangle(text=plain;headers=to:,subject:)
retr 1 mangle(text=plain;headers=to:,subject:)
Gellens Expires January 2008 [Page 5]
Internet Draft Mangle and Macro July 2007
retr 1 mangle(text=plain;charset=us-ascii;headers=to:,subject:)
list mangle(text=plain;charset=us-ascii;headers=to:,subject:)
list 1 mangle(text=plain;charset=us-ascii;headers=to:,subject:)
Examples of use:
The client retrieves message 1 as it exists in the message store:
C: retr 1
S: +OK 665 octets
S: Received: from foo.example.net (localhost [127.0.0.1])
S: by hotearth.example.net (8.12.10/8.12.3/1.0) with
S: ESMTP id j3L0Lq9B014701
S: for <poptest8@hotearth.example.net>;
S: Wed, 20 Apr 2005 17:21:52 -0700 (PDT)
S: Received: (from randy@localhost)
S: by hotearth.example.net (8.12.10/8.12.1/Submit) id
S: j3L0Lqqj014700 for poptest8;
S: Wed, 20 Apr 2005 17:21:52 -0700 (PDT)
S: Date: Wed, 20 Apr 2005 17:21:52 -0700 (PDT)
S: From: Randall Gellens <randy@example.net>
S: Message-Id: <200504210021.j3L0Lqqj014700@hotearth.example.net>
S: To: poptest8@example.net
S: Subject: test #1
S: X-UIDL: o=6"!2HZ!!LR+!!7C!"!
S: Status: RO
S:
S: This is a test.
S: .
The client retrieves message 1 as text/html and with only those
headers which start with 'foo' (in this case, none):
C: retr 1 mangle(text=html;headers=foo)
S: +OK Message follows
S: Mime-version: 1.0
S: Content-Type: text/html;charset="us-ascii"
S:
S: <html><pre>This is a test.
S: </pre></html>
S: .
Note that some servers (such as Qpopper) also support the "-no-mime"
user name hack. As a way to enable MIME-mangling with clients that
do not implement MANGLE, add "-no-mime" to the user name. For
example, if the userid is "mary", enter it in the client as
"mary-no-mime".
Gellens Expires January 2008 [Page 6]
Internet Draft Mangle and Macro July 2007
3.1. MANGLE Capability
CAPA tag:
MANGLE
Arguments:
none
Added commands:
none
Standard commands affected:
TOP, RETR, LIST
Announced states / possible differences:
both / no
Commands valid in states:
TRANSACTION
Specification reference:
This document
Discussion:
The MANGLE capability indicates the optional MANGLE parameter is
available for use with the TOP, RETR, and LIST commands. See
Section 3 for the details of the MANGLE parameter.
4. Macro Definition (MDEF)
This extension is announced by including MACRO in the CAPA [POP-EXT]
response.
The client uses the MDEF command to define a macro. The server may
impose limits on the length of the macro name and/or value and the
number of macros. In the case of Qpopper, the macro (name and value
plus parenthesis) must be no greater than 500 octets (100 in older
versions of the server), and clients may define up to ten macros.
Redefining a macro of the same name is permitted; the new macro
replaces the old one.
Syntax:
macro = "MDEF" macro-name "(" macro-value ")"
macro-name = 1*printable
macro-value = 1*printable
printable = <any printable ASCII character except for white space>
Gellens Expires January 2008 [Page 7]
Internet Draft Mangle and Macro July 2007
Note: There must not be any spaces within the mdef command (since
spaces are used to separate parameters to POP commands).
Once defined, the name of a macro can be used as a parameter to any
POP command that takes parameters. When used, the macro value
replaces the macro name as the command parameter.
Example:
mdef mang(mangle(text=plain;headers=to:,subject:))
The above example defines a macro with the name 'mang' whose value
is "mangle(text=plain;headers=to:,subject:)".
One situation in which macros are helpful is when clients using
bandwidth-constrained links wish to use the MANGLE extension
multiple times, with identical values in each usage. For example,
rather than a client entering the idential MANGLE extension as a
parameter to every TOP or RETR command, the client first defines the
extension as the value of a macro, then uses the macro name with the
TOP or RETR commands.
Example of use:
C: mdef html(mangle(text=html;headers=foo))
S: +OK Macro "html" accepted
C: retr 1 html
S: +OK Message follows
S: Mime-version: 1.0
S: Content-Type: text/html;charset="us-ascii"
S:
S: <html><pre>This is a test.
S: </pre></html>
S: .
4.1. MACRO Capability
CAPA tag:
MACRO
Arguments:
none
Added commands:
MDEF
Gellens Expires January 2008 [Page 8]
Internet Draft Mangle and Macro July 2007
Standard commands affected:
All commands valid in TRANSACTION state that take one or more
parameters: TOP, RETR, LIST, DELE, UIDL, extension commands
Announced states / possible differences:
both / no
Commands valid in states:
TRANSACTION
Specification reference:
This document
Discussion:
The MACRO capability indicates the MDEF command is available.
See Section 4 for the details of the MDEF command.
Note that the CAPA tag (MACRO) differs from the added command
(MDEF), which violates the SHOULD in Section 5 of [POP-EXT].
This is unfortunate, but because of an installed base of
implementations existing since 2000, fixing it now would be
worse than leaving it.
5. IANA Considerations
None.
6. Security Considerations
Alteration of message text destroys any security wrapper (digital
signature) and is useless within encrypted content. However, since
the alteration is only for the text sent in direct response to the
client, the original unaltered message remains in the message store.
Servers need to be careful that macro expansion not exceed buffer
sizes.
7. Normative References
[KEYWORDS] "Key words for use in RFCs to Indicate Requirement
Levels", S. Bradner, RFC 2119, BCP 14, March 1997.
[POP3] "Post Office Protocol - Version 3", J. Myers, M. Rose, RFC
1939, STD 53, May 1996.
Gellens Expires January 2008 [Page 9]
Internet Draft Mangle and Macro July 2007
[POP-EXT] "POP3 Extension Mechanism", R. Gellens, C. Newman, L.
Lundblade, RFC 2449, November 1998.
8. Informative References
[IMAP] "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1", M.
Crispin, RFC 3501, March 2003.
9. Author's Address
Randall Gellens
QUALCOMM Incorporated
5775 Morehouse Drive
San Diego, CA 92121
rg+ietf@qualcomm.com
Gellens Expires January 2008 [Page 10]