archivemail/archivemail.sgml

<!DOCTYPE RefEntry PUBLIC "-//Davenport//DTD DocBook V3.0//EN" [

<!ENTITY flock "<CiteRefEntry>
<RefEntryTitle><Command/flock/</RefEntryTitle>
<ManVolNum/2/</CiteRefEntry>">

<!ENTITY gzip "<CiteRefEntry>
<RefEntryTitle><Command/gzip/</RefEntryTitle>
<ManVolNum/1/</CiteRefEntry>">

<!ENTITY mutt "<CiteRefEntry>
<RefEntryTitle><Command/mutt/</RefEntryTitle>
<ManVolNum/1/</CiteRefEntry>">

<!ENTITY procmail "<CiteRefEntry>
<RefEntryTitle><Command/procmail/</RefEntryTitle>
<ManVolNum/1/</CiteRefEntry>">

<!ENTITY python "<CiteRefEntry>
<RefEntryTitle><Command/python/</RefEntryTitle>
<ManVolNum/1/</CiteRefEntry>">

<!ENTITY seteuid "<CiteRefEntry>
<RefEntryTitle><Command/seteuid/</RefEntryTitle>
<ManVolNum/2/</CiteRefEntry>">

<!ENTITY crontab "<CiteRefEntry>
<RefEntryTitle><Command/crontab/</RefEntryTitle>
<ManVolNum/5/</CiteRefEntry>">

<!ENTITY mbox "<CiteRefEntry>
<RefEntryTitle><Command/mbox/</RefEntryTitle>
<ManVolNum/5/</CiteRefEntry>">
]>

<RefEntry>

<DocInfo><Date>8 April 2008</Date></DocInfo>

<RefMeta>
<RefEntryTitle>archivemail</RefEntryTitle>
<ManVolNum>1</ManVolNum>
<RefMiscInfo>SP</RefMiscInfo>
</RefMeta>

<RefNameDiv>
<RefName>archivemail</RefName>
<RefPurpose>archive and compress your old email</RefPurpose>
</RefNameDiv>

<RefSynopsisDiv>
<CmdSynopsis>

<Command/archivemail/
<Arg><Option>options</Option></Arg>
<Arg choice=req rep=repeat><Replaceable/MAILBOX/</Arg>

</CmdSynopsis>
</RefSynopsisDiv>

<RefSect1>
<Title>Description</Title>

<Para>
archivemail is a tool for archiving and compressing old email in mailboxes.
By default it will read the mailbox <Replaceable/MAILBOX/, moving messages
that are older that the specified number of days (180 by default) to a
&mbox;-format mailbox in the same directory that is compressed
with &gzip;.
It can also just delete old email rather than archive it.
</Para>

<Para>
<Command/archivemail/ supports reading  <application/IMAP/,
<application/Maildir/, <application/MH/ and <application/mbox/-format
mailboxes, but always writes <application/mbox/-format archives.
</Para>

<Para>
    Messages that are flagged important are not archived or deleted unless
    explicitely requested with the <Option>--include-flagged</Option> option.
    Also, <command/archivemail/ can be configured not to archive unread mail, or
    to only archive messages larger than a specified size. 
</Para>

<Para>
To archive an <application/IMAP/-format mailbox, use the format
<replaceable>imap://username:password@server/mailbox
</replaceable> to specify the mailbox. 
You can omit the password from the <acronym/URL/; use the
<option>--pwfile</option> option to make <command/archivemail/ read the 
password from a file, or alternatively just enter it upon request. 
If the <option>--pwfile</option> option is set, <command/archivemail/ does not
look for a password in the <acronym/URL/, and the colon is not considered a 
delimiter.
Substitute '<userinput/imap/' with '<userinput/imaps/', and 
<command/archivemail/ will establish a secure <acronym/SSL/ connection.  
See below for more <application/IMAP/ peculiarities. 
</Para>


<Para>
<Command/archivemail/ has some support for being run as the root user on
user mailboxes. When running as root, it will &seteuid; to the owner of the
mailbox it is reading, creating any archive files as that user. 
<emphasis>Warning:</emphasis>
<!-- I would like to have a <warning> here, but the dsssl stylesheet thinks a
    warning need be decorated with a graphic.  *sigh*  I won't start hacking
    dsssl because of this. -->
    this automatic seteuid feature is insecure and deprecated. 
    It will be removed from later versions of <command/archivemail/.
</Para>
</RefSect1>

<RefSect1>
<Title>Options</Title>

<VariableList>

<VarListEntry>
<Term>
    <Option>-d <Replaceable/NUM/, --days=<Replaceable/NUM/</Option>
</Term>
<ListItem><Para>Archive messages older than <Replaceable/NUM/ days.  
The default is 180. This option is incompatible with the
<Option/--date/ option below.
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>-D <Replaceable/DATE/, --date=<Replaceable/DATE/</Option>
</Term>
<ListItem><Para>Archive messages older than <Replaceable/DATE/.
<Replaceable/DATE/ can be a date string in ISO format (eg '2002-04-23'), 
Internet format (eg '23 Apr 2002') or Internet format with full month names 
(eg '23 April 2002'). Two-digit years are not supported.
This option is incompatible with the <Option/--days/ option above.
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>-o <Replaceable/PATH/, --output-dir=<Replaceable/PATH/</Option>
</Term>
<ListItem><Para>Use the directory name <Replaceable/PATH/ to store the
mailbox archives. The default is the same directory as the mailbox to be
read.
</Para></ListItem>

<VarListEntry>
<Term>
    <Option>-P <Replaceable/FILE/, --pwfile=<Replaceable/FILE/</Option>
</Term>
<ListItem><Para>Read <acronym/IMAP/ password from file <Replaceable/FILE/
instead of from the command line.  Note that this will probably not work if you
are archiving folders from more than one IMAP account. 
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>-F <Replaceable/STRING/,
    --filter-append=<Replaceable/STRING/</Option>
</Term>
<ListItem><Para>Append <Replaceable/STRING/ to the <acronym/IMAP/ filter string. 
For <acronym/IMAP/ wizards. 
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>-s <Replaceable/NAME/, --suffix=<Replaceable/NAME/</Option>
</Term>
<ListItem><Para>
Use the suffix <Replaceable/NAME/ to create the filename used for archives.  
The default is <filename>_archive</filename>. For example, if you
run <Command/archivemail/ on a mailbox called
<filename>exsouthrock</filename>, the archive will be created
with the filename <filename>exsouthrock_archive.gz</filename>.
</Para>

<Para>
<Replaceable/NAME/ is run through the &python; <application/time.strftime()/
function, which means that you can specify any of the following special
directives in <Replaceable/NAME/ to make archives named after the archive
cut-off date:

<itemizedlist mark='none'>
    <listitem><para><option>%a</option> 
    Locale's abbreviated weekday name.</para></listitem>
    <listitem><para><option>%A</option> 
    Locale's full weekday name.</para></listitem>
    <listitem><para><option>%b</option> 
    Locale's abbreviated month name.</para></listitem>
    <listitem><para><option>%B</option> 
    Locale's full month name.</para></listitem>
    <listitem><para><option>%c</option> 
    Locale's appropriate date and time representation.</para></listitem>
    <listitem><para><option>%d</option> 
    Day of the month as a decimal number [01,31].</para></listitem>
    <listitem><para><option>%H</option> 
    Hour (24-hour clock) as a decimal number [00,23].</para></listitem>
    <listitem><para><option>%I</option> 
    Hour (12-hour clock) as a decimal number [01,12].</para></listitem>
    <listitem><para><option>%j</option> 
    Day of the year as a decimal number [001,366].</para></listitem>
    <listitem><para><option>%m</option> 
    Month as a decimal number [01,12].</para></listitem>
    <listitem><para><option>%M</option> 
    Minute as a decimal number [00,59].</para></listitem>
    <listitem><para><option>%p</option> 
    Locale's equivalent of either AM or PM.</para></listitem>
    <listitem><para><option>%S</option> 
    Second as a decimal number [00,61]. (1)</para></listitem>
    <listitem><para><option>%U</option> 
    Week number of the year (Sunday as the first day of the week) as a decimal number [00,53]. All days in a new year preceding the first Sunday are considered to be in week 0.</para></listitem>
    <listitem><para><option>%w</option> 
    Weekday as a decimal number [0(Sunday),6].</para></listitem>
    <listitem><para><option>%W</option> 
    Week number of the year (Monday as the first day of the week) as a decimal number [00,53]. All days in a new year preceding the first Sunday are considered to be in week 0.</para></listitem>
    <listitem><para><option>%x</option> 
    Locale's appropriate date representation.</para></listitem>
    <listitem><para><option>%X</option> 
    Locale's appropriate time representation.</para></listitem>
    <listitem><para><option>%y</option> 
    Year without century as a decimal number [00,99].</para></listitem>
    <listitem><para><option>%Y</option> 
    Year with century as a decimal number.</para></listitem>
    <listitem><para><option>%Z</option> 
    Time zone name (or by no characters if no time zone exists).</para></listitem>
    <listitem><para><option>%%</option> 
    A literal "%" character.</para></listitem>
</itemizedlist>

</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>-S <Replaceable/NUM/, --size=<Replaceable/NUM/</Option>
</Term>
<ListItem><Para>Only archive messages that are <Replaceable/NUM/ bytes or
greater.
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>-n, --dry-run</Option>
</Term>
<ListItem><Para>
Don't write to any files -- just show what would have been done. This is
useful for testing to see how many messages would have been archived.
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>-u, --preserve-unread</Option>
</Term>
<ListItem><Para>
Do not archive any messages that have not yet been read. <command/archivemail/
determines if a message in a <application/mbox/-format or
<application/MH/-format mailbox has been read by looking at the
<application/Status/ header (if it exists). If the status
header is equal to 'RO' or 'OR' then <application/archivemail/ assumes the
message has been read. <command/archivemail/ determines if a
<application/maildir/ message has
been read by looking at the filename. If the filename contains an 'S' after 
<filename>:2,</filename> then it assumes the message has been read.
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>--dont-mangle</Option>
</Term>
<ListItem><Para>
Do not mangle lines in message bodies beginning with "From ".  When archiving
a message from a mailbox not in <application/mbox/ format, by default
<command/archivemail/ mangles such lines by prepending a '>' to them, since mail
user agents might otherwise interpret these lines as message separators.
Messages from <application/mbox/ folders are never mangled.  See &mbox; for more
information.
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>--delete</Option>
</Term>
<ListItem><Para>
Delete rather than archive old mail. Use this option with caution!
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>--copy</Option>
</Term>
<ListItem><Para>
Copy rather than archive old mail. 
Creates an archive, but the archived messages are not deleted from the
originating mailbox, which is left unchanged.
This is a complement to the <option/--delete/ option, and mainly useful for
testing purposes. 
Note that multiple passes will create duplicates, since messages are blindly
appended to an existing archive.
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>--all</Option>
</Term>
<ListItem><Para>
Archive all messages, without distinction.
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>--include-flagged</Option>
</Term>
<ListItem><Para>
Normally messages that are flagged important are not archived or deleted. If
you specify this option, these messages can be archived or deleted just like
any other message.
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>--no-compress</Option>
</Term>
<ListItem><Para>
Do not compress any archives.
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>--warn-duplicate</Option>
</Term>
<ListItem><Para>
Warn about duplicate <application/Message-ID/s that appear in the input
mailbox.</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>-v, --verbose</Option>
</Term>
<ListItem><Para>
Reports lots of extra debugging information about what is going on.
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>--debug-imap=<Replaceable/NUM/</Option>
</Term>
<ListItem><Para>
Set <acronym/IMAP/ debugging level. This makes <Command/archivemail/ dump its
conversation with the <acronym/IMAP/ server and some internal <acronym/IMAP/
processing to <Literal/stdout/.  Higher values for <Replaceable/NUM/ give more
elaborate output.  Set <Replaceable/NUM/ to 4 to see all exchanged 
<acronym/IMAP/ commands.  (Actually, <Replaceable/NUM/ is just passed 
literally to <Literal/imaplib.Debug/.)
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>-q, --quiet</Option>
</Term>
<ListItem><Para>
Turns on quiet mode. Do not print any statistics about how many messages were
archived. This should be used if you are running <Command/archivemail/ from
cron.
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>-V, --version</Option>
</Term>
<ListItem><Para>
Display the version of <Command/archivemail/ and exit.
</Para></ListItem>
</VarListEntry>

<VarListEntry>
<Term>
    <Option>-h, --help</Option>
</Term>
<ListItem><Para>
Display brief summary information about how to run <Command/archivemail/.
</Para></ListItem>
</VarListEntry>
</VariableList>

</RefSect1>

<RefSect1>
<Title>Notes</Title>
<Para>
<Command/archivemail/ requires &python; version 2.3 or later.
When reading an <application/mbox/-format mailbox, <command/archivemail/ will
create a lockfile with the extension <filename>.lock</filename> so that
procmail will not deliver to the mailbox while it is being processed. It will
also create an advisory lock on the mailbox using &flock;.
<command/archivemail/ will also complain and abort if a 3rd-party modifies the
mailbox while it is being read.
</Para>

<Para>
<Command/archivemail/ will always attempt to preserve the mode,
last-access and last-modify times of the input mailbox. However, archive
mailboxes are always created with a mode of <application/0600/.
If <Command/archivemail/ finds a pre-existing archive mailbox it
will append rather than overwrite that archive.
<Command/archivemail/ will refuse to operate on mailboxes that are symbolic 
links or create tempfiles or archives in world-writable directories.
</Para>

<Para>
<Command/archivemail/ attempts to find the delivery date of a message by
looking for valid dates in the following headers, in order of precedence: 
<application/Delivery-date/,
<application/Received/, 
<application/Resent-Date/ and
<application/Date/.
If it cannot find any valid date in these headers, it
will use the last-modified file timestamp on <application/MH/ and
<application/Maildir/ format mailboxes, or the date on the 
<application/From/ line on <application/mbox/-format mailboxes.
</Para>

<Para>
    A conversion from other formats to &mbox; will silently overwrite existing
    <application/Status/ and <application/X-Status/ message headers. 
</Para>

<RefSect2>
<Title><acronym/IMAP/</Title>
<Para>
When <command/archivemail/ processes an <application/IMAP/ folder, all messages 
in that folder will have their <literal/\Recent/ flag unset, and they will 
probably not show up as 'new' in your user agent later on. 
There is no way around this, it's just how <application/IMAP/ works. 
This does not apply, however, if you run <command/archivemail/ with the options
<option/--dry-run/ or <option/--copy/.
</Para>
<Para>
<command/archivemail/ relies on server-side searches to determine the messages
that should be archived. 
When matching message dates, <application/IMAP/ servers refer to server internal
message dates, and these may differ from both delivery time of a message and 
its <application/Date/ header. 
Also, there exist broken servers which do not implement server side searches. 
</Para>
<RefSect3><Title><acronym/IMAP/ <acronym/URLs/</Title>
<Para>
<command/archivemail/'s <application/IMAP/ <acronym/URL/ parser was written
with the <acronym>RFC</acronym> 2882 (<citetitle>Internet Message
Format</citetitle>) rules for the <token>local-part</token> of email addresses
in mind.
So, rather than enforcing an <acronym/URL/-style encoding of non-<acronym/ascii/
and reserved characters, it allows to double-quote the username and password.
If your username or password contains the delimiter characters '@' or ':', just
quote it like this:
<replaceable>imap://"username@bogus.com":"password"@imap.bogus.com/mailbox
</replaceable>.
You can use a backslash to escape double-quotes that are part of a quoted
username or password.
Note that quoting only a substring will not work, and be aware that your shell
will probably remove unprotected quotes or backslashes. 
</Para>
<Para>
<command/archivemail/ tries to be smart when handling mailbox paths. 
In particular, it will automatically add an <acronym/IMAP/ <literal/NAMESPACE/ 
prefix to the mailbox path if necessary; and if you are archiving a subfolder,
you can use the slash as a path separator instead of the <acronym/IMAP/ server's
internal representation.
</Para>
</RefSect3>
</RefSect2>

</RefSect1>

<RefSect1>
<Title>Examples</Title>

<InformalExample>
<Para>
To archive all messages in the mailbox <filename>debian-user</filename> that
are older than 180 days to a compressed mailbox called
<filename>debian-user_archive.gz</filename> in the current directory: 
<screen>
<prompt>bash$ </prompt><userinput>archivemail debian-user</userinput>
</screen>
</Para>
</InformalExample>

<InformalExample>
<Para>
To archive all messages in the mailbox <filename>debian-user</filename> that
are older than 180 days to a compressed mailbox called
<filename>debian-user_October_2001.gz</filename> (where the current month and
year is April, 2002) in the current directory: 
<screen>
<prompt>bash$ </prompt><userinput>archivemail --suffix '_%B_%Y' debian-user</userinput>
</screen>
</Para>
</InformalExample>

<InformalExample>
<Para>
To archive all messages in the mailbox <filename>cm-melb</filename> that
are older than the first of January 2002 to a compressed mailbox called
<filename>cm-melb_archive.gz</filename> in the current directory: 
<screen>
<prompt>bash$ </prompt><userinput>archivemail --date'1 Jan 2002' cm-melb</userinput>
</screen>
</Para>
</InformalExample>

<InformalExample>
<Para>
Exactly the same as the above example, using an ISO date format instead:
<screen>
<prompt>bash$ </prompt><userinput>archivemail --date=2002-01-01 cm-melb</userinput>
</screen>
</Para>
</InformalExample>

<InformalExample>
<Para>
To delete all messages in the mailbox <filename>spam</filename> that
are older than 30 days:
<screen>
<prompt>bash$ </prompt><userinput>archivemail --delete --days=30 spam</userinput>
</screen>
</Para>
</InformalExample>

<InformalExample>
<Para>
To archive all read messages in the mailbox <filename>incoming</filename> that
are older than 180 days to a compressed mailbox called
<filename>incoming_archive.gz</filename> in the current directory: 
<screen>
<prompt>bash$ </prompt><userinput>archivemail --preserve-unread incoming</userinput>
</screen>
</Para>
</InformalExample>

<InformalExample>
<Para>
To archive all messages in the mailbox <filename>received</filename> that
are older than 180 days to an uncompressed mailbox called 
<filename>received_archive</filename> in the current directory:
<screen>
<prompt>bash$ </prompt><userinput>archivemail --no-compress received</userinput>
</screen>
</Para>
</InformalExample>

<InformalExample>
<Para>
To archive all mailboxes in the directory <filename>$HOME/Mail</filename> 
that are older than 90 days to compressed mailboxes in the
<filename>$HOME/Mail/Archive</filename> directory:
<screen>
<prompt>bash$ </prompt><userinput>archivemail -d90 -o $HOME/Mail/Archive $HOME/Mail/*</userinput>
</screen>
</Para>
</InformalExample>

<InformalExample>
<Para>
    To archive all mails older than 180 days from the given <application/IMAP/
    INBOX to a compressed mailbox INBOX_archive.gz in the
    <filename>$HOME/Mail/Archive</filename> directory, quoting the password and
    reading it from the environment variable <envar>PASSWORD</envar>:
</Para>
<!-- I'm open to suggestions how to avoid making such a super-long line here.  -->
<screen>
<prompt>bash$ </prompt><userinput>archivemail -o $HOME/Mail/Archive imaps://user:'"'$PASSWORD'"'@example.org/INBOX</userinput> 
</screen>
<Para>
    Note the protected quotes. 
</Para>
</InformalExample>
</RefSect1>

<RefSect1>
<Title>Tips</Title>
<Para>
Probably the best way to run <Command/archivemail/ is from your &crontab;
file, using the <Option>--quiet</Option> option.
Don't forget to try the <Option>--dry-run</Option> and perhaps the
<Option>--copy</Option> option for non-destructive testing.
</Para>
</RefSect1>

<RefSect1>
<Title>Exit Status</Title>
<SimPara>Normally the exit status is 0. Nonzero indicates an unexpected error.
</SimPara>
</RefSect1>

<RefSect1>
<Title>Bugs</Title>
<SimPara>
If an <application/IMAP/ mailbox path contains slashes, the archive filename
will be derived from the basename of the mailbox. 
If the server's folder separator differs from the Unix slash and is used in the
<application/IMAP/ <acronym/URL/, however, the whole path will be considered
the basename of the mailbox. 
E.g. the two <acronym/URL/s
<userinput>imap://user@example.com/folder/subfolder</userinput> and
<userinput>imap://user@example.com/folder.subfolder</userinput> will be
archived in <filename>subfolder_archive.gz</filename> and
<filename>folder.subfolder_archive.gz</filename>, respectively, although they
might refer to the same <application/IMAP/ mailbox. 
</SimPara>
<SimPara>
<command/archivemail/ does not support reading <application/MMDF/ or
<application/Babyl/-format mailboxes. In fact, it will probably think it is 
reading an <application/mbox/-format mailbox and cause all sorts of problems.
</SimPara>

<SimPara>
<Command/archivemail/ is still too slow, but if you are running from &crontab;
you won't care. Archiving <application/maildir/-format mailboxes should be
a lot quicker than <application/mbox/-format mailboxes since it is less
painful for the original mailbox to be reconstructed after selective message 
removal.
</SimPara>
</RefSect1>

<RefSect1>
<Title>See Also</Title>
<SimpleList>
<Member> &python;, &gzip;, &mutt;, &procmail;</Member>
</SimpleList>
</RefSect1>

<RefSect1>
<Title>Url</Title>
<SimPara>The <Command/archivemail/ home page is currently hosted at 
<ulink type="http" url="http://archivemail.sourceforge.net">sourceforge</ulink>
</SimPara>
</RefSect1>

<RefSect1>
<Title>Author</Title>
<SimPara> This manual page was written by Paul Rodger &lt;paul at paulrodger dot
com&gt;. Updated and supplemented by Nikolaus Schulz
<email>microschulz@web.de</email></SimPara>
</RefSect1>

</RefEntry>