Cyrus IMAP for Debian --------------------- "All systems administrators have their horror stories. For me, it was setting up a HP Color Bubblejet under Linux using ghostscript before linuxprinting.org was alive. Well that was a piece of cake compared to what I am about to describe in this document." -- "Hosting email for virtual domains using Postfix and Cyrus" Haim Dimermanas, 2001-08-01 "I warned you to read all the documentation first, didn't I?" -- Henrique M. Holschuh, 2002-10-01 IMPORTANT: Cyrus is a closed-box email system. Your system will access your email through LMTP, IMAP and POP3 *only*. No direct file access to the email store is supposed to take place. For more information, please consult http://cyrusimap.org/. There is also Cyrus-HOWTO (Cyrus-IMAP.txt) available as part of the LDP HOWTO collection. Outdated documentation will cause you much grief, so beware of that when hunting anywhere else than the Cyrus mailinglist for information. Backports of the latest packages for Debian Stable are available from http://www.backports.org WARNING: For one to get Cyrus IMAPd to work correctly, one must first get the SASL layer to work correctly. This is far from trivial, so if you don't manage at first, don't go around filling bugs against Cyrus IMAPd before you make damn sure it is not a SASL configuration error. Read the hint list later on this file as well. Start by reading README.Debian.simpleinstall. The Debian packaging of Cyrus has a few quirks which are important to know about: 1. Relocation of many Cyrus IMAP files The default Cyrus install scatters files all over the place. The Debian package installs only a few files in /usr/bin (cyradm). All other programs are installed into /usr/lib/cyrus/bin with convenience tool called /usr/sbin/cyrus which can be used to call all cyrus utilities. Invoke /usr/sbin/cyrus --help without any argument to learn more. The imapd.conf and cyrus.conf configuration files are in /etc. The PAM policy files are in /etc/pam.d. The permissions on /run/cyrus and /run/cyrus/socket are now managed by tmpfiles.d mechanism from systemd (with a help of small shell wrapper to also support sysvinit as init). To change the permissions on socket directory, you'll nee to edit /usr/lib/tmpfiles.d/cyrus-imapd.conf file. In fact, you will most likely have to do so for postfix to deliver to Cyrus, for example. 2. Cyrus Murder, the Cyrus IMAPd/POP3 aggregator is available. However, you will have to configure it yourself. No pre-packaged configuration of Murder is available at this time... The documentation is all there, and the Cyrus packages will happily preserve your Cyrus Murder configuration. You do not have to install the cyrus-imapd or cyrus-pop3d packages in hosts that only need the proxy daemons running, but do note that the /etc/pam.d/imap and /etc/pam.d/pop files are in those packages (and they are needed by the proxies), so you will have to create the files manually. One important note: MUPDATE doesn't support TLS, so you won't be able to use plaintext authentication methods. The easiest thing to do is to put an entry for your mupdate user in sasldb2 and use DIGEST-MD5. 3. Configurable idled support. Cyrus IMAPd supports three options of using IDLE in IMAP sessions. The first option is not to support IDLE at all. The second is to use internal polling in the IMAP daemon. The third option is to use an external daemon, idled. Upstream only supports configuration of this during compilation, Debian however includes a patch which makes this runtime-configurable. Please set the 'idlemethod' imapd.conf option according to your needs and enable idled in cyrus.conf if you want to use it. General notes and hints: ------------------------ o *** ALWAYS READ /usr/share/doc/cyrus-common/NEWS.Debian *** after you upgrade the package. This, and every other NEWS.Debian can automatically be shown to you before the upgrade, see the apt-listchanges package for more information. o QUOTAS ARE LIMITED TO 2GB on some platforms. Be careful to not set quotas over that amount if your platform doesn't support the C datatype "long long". Things will break in very bad ways. Yes, it is a big glitch, and no, there are no easy workarounds. see https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=1212 This has been fixed for the upcoming Cyrus 2.4. o Either turn off logging of the DEBUG level, or don't complain about cyrus verbosity on the logs. Don't ever ask in the mailing lists about messages logged in the DEBUG level before reading the source code. o Watch out for your /dev/random bitbucket! SASL may use it, and if it empties, it will hang the processes wrapped up by SASL. This means just about every Cyrus service (lmtp, imap, pop3, sieve)... Disable APOP in /etc/imapd.conf if you don't need it, as it is a serious draw on randomness resources. o One extremely important point to notice is that saslauthd works ONLY with plaintext. APOP, CRAM-MD5, OTP, DIGEST-MD5 and any other "auxprop" SASL mech will *not* work through saslauthd. This can and will cause serious issues in Cyrus murder environments. o When using ext3, Cyrus really wants data=journal. However, up to kernel 2.4.20 there are dangerous bugs in that option, so you're better off not using that. xfs is faster and better for Cyrus, anyway. Please note that sarge was shipped with 2.4.27, and etch will not ship any 2.4 kernels anymore." 2.4 kernels are NOT shipped with Debian Etch. o nscd users: nscd is highly incompatible with ldap, and somewhat buggy otherwise. If you use nscd and Cyrus segfaults on you, try restarting nscd, or disabling it. o "The Debian libldap2 and cyrus-imapd packages are both compiled using the SASL library. If you use cyrus-imapd together with libnss-ldap, or saslauthd together with libpam-ldap, the resulting double calls to SASL library functions can trigger a double-free bug which may cause the calling process to crash. To avoid such a crash, you must recompile the libldap2 package --without-cyrus-sasl." -- http://bugs.debian.org/145766 [!@#$%!!! I didn't expect SASL 2.1 to still have this annoying problem] o The lmtp service (allocated in Debian Woody to port 2003, and non-existent on Debian Sarge and Etch) is non-standard. It has no port officially allocated anywhere; it is usually run bound to the localhost interface, unless one needs it for clustering and high-availability scenarios. If you need it elsewhere, by all means move it -- you only need to edit /etc/services, or change the port for the lmtp service in /etc/cyrus.conf. o The lmtp service will only allow Cyrus lmtp administrators to authenticate. Set them in /etc/imapd.conf. o Cyrus can now use two different namespaces (the standard one, where all subfolders are children of INBOX, and one where they are all in the same hierarchical level). See /usr/share/doc/cyrus-doc/html/altnamespace.html for details. If you deal with a large population of winboze users, this option can save you some headaches. o One can also chose between netnews-style notation for folders (INBOX.subfolder), where the "." character is reserved to separate folders; or UNIX-style notation (INBOX/subfolder), where dots are allowed in names, and the slash separate folders (the "^" character is reserved in this mode). See /usr/share/doc/cyrus-common/html/altnamespace.html for details. o When using SASL, do keep in mind that cyrus runs under user cyrus, and not root. It cannot read shadow files (unless you add the user cyrus to group shadow), or perform any root-only operations directly. You need to use the saslauthd (or, if available, auxpropd) mechanism to authenticate against root-only data. And that also means user cyrus must be able to talk to the unix socket saslauthd uses (which is controlled by SASL, not Cyrus IMAPd). o Any of the SASL configure options can be inserted in imapd.conf, just prefix it with "sasl_" (e.g.: sasl_mech_list: PLAIN). The list of SASL options is in /usr/share/doc/libsasl2/options.html. o The services are tcp-wrapped. Their hosts.allow/hosts.deny id is the service name in /etc/cyrus.conf. See hosts_access(5). o The PAM service names for use with SASL (via saslauthd) are: "imap", "sieve", "lmtp", "pop", "mupdate". o You need to specify your admin users in /etc/imapd.conf before you can add mailboxes, or deliver through authenticated lmtp. Do NOT use root. We suggest user cyrus, which is already used by the system for all things Cyrus IMAPd... but it need not be an existing user. As long as SASL will authenticate against it, it will work. o Do NOT read your admin user's email via IMAP (see the FAQ for details). o Don't export your mail store over NFS or AFS (read the FAQ for more info). You have been warned. You really want a journaled (as in journaling for the metadata), local filesystem for the store. Failing that, you need something with very strict and correct lock semantics, and full mmap support. o Ext2 is slow on very large directories (right now), and sync metadata writes enabled are a huge performance hit. If you need high IO throughput from Cyrus, you will need to use ext3, reiserfs, xfs or something similar. o You may want to enable/disable synchronous metadata writes to your mail store dirs (check /usr/share/doc/cyrus-doc/html/install.html for more info, in package cyrus-docs). The cyrus-makedirs script tries to do the right thing for ext2 and ext3 filesystems. Failure to correctly update the metadata in the right order can completely screw up your Cyrus store on a power-loss or another disk failure. o Try mounting the store and cyrus database filesystems with noatime for performance gains. Load-balance the store using multiple partitions on different physical devices for even better performance gains. o Cyrus IMAPd should be fed mail through LMTP. If at all possible, use the Unix socket for that -- it automatically authenticates as user postman and that will help wonders. cyrdeliver can also be used to inject mail, but it will simply open an LMTP socket to cyrus and deliver through that -- this is much slower than using LMTP directly. The UNIX socket is in /run/cyrus/socket/lmtp. Use tmpfiles.d if you need to change the permissions of the socket directory. o You can use /usr/sbin/cyrus-makedirs to generate the needed directories for cyrus partitions. It is run automatically by the package postinst, and it knows to parse the /etc/imapd.conf file to verify if hash subdirectories are needed or not. It cannot detect what kind of hashing should be used yet. If you recompile the package with full hashing, change it. o Refer to cyrus-utils.sourceforge.net and the info-cyrus mailinglist for mailbox/imap to cyrus conversion scripts. o If you don't have pop3 or something else enabled by default in cyrus.conf, installed, disable it. Otherwise, Cyrus master will log warnings that the service could not be started. o If you want to run something that is not in /usr/lib/cyrus/bin in cyrus.conf, just use the full path in cyrus.conf (e.g.: cmd="/usr/sbin/squatter"). o Sieveshell is really lacking on auth capabilities, and timsieved is quite strict on what auth capabilities it offers. So, pay attention to sasl_minimum_layer, and see bug #151295 for more details (http://bugs.debian.org/151925). Also, make sure you have the correct set of SASL2 modules installed in in your system. o uw-mailutils has some nice utilities to migrate mail stores from/to imap servers. You might find it quite useful to migrate a site to Cyrus. SNMP logging ------------ cyrmaster is an agentx SNMP subagent, and it can interface to a agentx SNMP master. It will export data at OID .1.3.6.1.4.1.3.6.1 (cyrusMasterMIB). The ucd-snmp daemon (package snmpd) is NOT configured to work as agentx master agent by default -- you have to do that manually, by adding "master agentx" to the /etc/snmp/snmpd.conf file. cyrmaster will register with the snmp agentx master when it is started, so if the snmp master is restarted after cyrmaster, it will not forward the snmp requests to cyrmaster anymore. Check your system for any cron scripts that might be restarting the snmp process if that happens. See /usr/share/snmp/mib/CYRUS-MASTER-MIB.txt for more details. Backing up for rainy days ------------------------ Cyrus automatically checkpoints and backups some of its databases, using the ctl_cyrusdb(8) utility (EVENTS in /etc/cyrus.conf). It is supposed to be also capable of recovering automatically from these backups, and to attempt to do so at startup. However, ctl_cyrusdb -r is NOT FULLY IMPLEMENTED YET... you are on your own to recover from corrupt databases. This recovery can be done using the db3 utilities, and even by smart usage of cvt_cyrusdb(8) and ctl_mboxlist(8). The automatic backups are useful, too, even if they are not restored automatically. The database backups are stored at /var/lib/cyrus/db.backup*, you may want to copy the files there to backup media in a cronjob, or something like that. You can kill the TLS cache database, as long as Cyrus is stopped when you do it. Loss of the delivery database is not very bad, it just means some users might get duplicated messages. Cyrus does NOT backup the mail store automatically. To backup the mail store partitions, you must stop Cyrus and dump the entire partition to your backup media. The MH-like structure of the Cyrus store do make them suitable for incremental backups. Hot-backups of the store can be made, but you risk losing some non-critical metadata when the restore is done. You can backup all Cyrus non-text databases to a flat text file format using the cvt_cyrusdb utility (and recover back from the flat text file format), but you should stop Cyrus first. If you ever need to recover the mail store from backup, you should run cyrreconstruct(8) to rebuild the mailbox indexes. A daily maintenance cronjob uses ctl_mboxlist(8) to dump the mailboxes database to /var/backup. That backup copy can be used as a last-resort copy if the hot backups become corrupted somehow. -- Ondřej Surý , Fri, 22 Jul 2016 09:25:48 +0200