Create mailbox for cyrus-imap

If you follow my guide on postfix/cyrus-imapd set up, you may encounter a problem when login the mail server to retrieve emails.

starttls: TLSv1 with cipher RC4-MD5 (112/128 bits reused) no authentication
badlogin: . [xx.xx.xx.xx] plaintext admin SASL(-13): user not found: checkpass failed

The message is a little confusing. It says the user “admin” is not found. But the email client (outlook express)is configured to use “admin@domainhostseotool.xyz” as the user name. Is that a bug of outlook express that strips the domain name out of the user name? Note that we use a rare tld “.xyz” domain here. If we change admin@domainhostseotool.xyz to admin@domainhostseotool.com and configure it in outlook express, we will see the following error log:

badlogin: . [xx.xx.xx.xx] plaintext admin@domainhostseotool.com SASL(-13): user not found: checkpass failed

Note that this time, the domain name appears in the user name. This almost makes me certain that this is a bug of outlook express. But after I add the user admin@domainhostseotool.com in /etc/sasldb2 using the saslpasswd2 command, the problem persists. The problem is hard to debug unless you know there is other log information pertaining to the mail system besides /var/log/maillog. It is in /var/log/messages:

pop3s[327]: unable to open Berkeley db /etc/sasldb2: Permission denied

ls -l /etc/sasldb2
-rw-r—– 1 root root 155218 Mar 15 03:01 /etc/sasldb2

You can see the the file containing the user account information can not be read by user other than root and users in root group. The pop3s process, however, is run by cyrus

ps -ef|grep “pop3d -s”|grep -v grep
cyrus     2221  2280  0 Mar14 ?        00:00:00 pop3d -s

The user cyrus is not in the root group:

id cyrus
uid=71(cyrus) gid=15(mail) groups=15(mail),79(saslauth)

More accurately, you can use the following command to check the effective user and effective group of pop3s:

ps  -o “euser,egroup,command” -e | grep “pop3d -s”|grep -v grep
cyrus    mail     pop3d -s

So pop3s can not read /etc/sasldb2 and generates the log in /var/log/messages. The solution is now clear: change the owner/group of /etc/sasldb2 to make it readable for pop3:

chown postfix /etc/sasldb2

chgrp mail /etc/sasldb2

ls -l /etc/sasldb2
-rw-r—– 1 postfix mail 155218 Mar 15 03:01 /etc/sasldb2

But the trouble is not over. Remember the user account ending with “.xyz” tld domain? If you try to login pop3 with that account, you will still fail with the error:

login: . [xx.xx.xx.xx] admin plaintext+TLS User logged in
pop3s[5858]: Unable to locate maildrop user.admin: Mailbox does not exist

Yes, we have not created a mailbox for that user manually. Sure, we can create new user account using cyradm:

cm user.newuser

This will create a folder n/user/newuser/ under /var/spool/imap/ and files : cyrus.cache  cyrus.header  cyrus.index under /var/spool/imap/n/user/newuser/. The layout of mailbox is a little weird. The name of the “n” directory is took from the first character of the user name.

But wait a moment, we do not need to create mailbox manually. We have configured cyrus-imap in /etc/imapd.conf to create mailbox automatically.

autocreatequota:        -1
createonpost:            yes
autocreateinboxfolders:        spam
autosubscribeinboxfolders:    spam

So ideally, the mailbox for admin@domainhostseotool.xyz should be created after login. And from the log, we did login successfully. Why does not it create the mailbox and complain the mailbox does not exist? Do we have some error in the configuration file /etc/imapd.conf? I tried different versions/combinations of configuration parameters but none of them worked.

#autocreate_users: 1
autocreatequota:       -1

#autocreate_quota: -1

#autocreate_quota: 100

#autocreate_quota: yes

#autocreate_quota: 1
#autocreate_post: 1

#autocreatepost: 1
#createonpost:            yes
autocreateinboxfolders:        spam

#autocreate_inbox_folders:        spam
#autosubscribeinboxfolders:    spam
#autocreate_subscribe_folders:    spam

Note that some forms of parameter are obsoleted in the most updated version of cyrus-imap(3.0.5). To see the correct parameter names for your version of cyrus-imap, use “man imapd.conf” on your system.  (BTW, you can use “man cyrus-master” to see the help of the core process of cyrus-master.) But interestingly, whatever I change the names of the parameters, cyrus-imapd can be restarted successfully and no error is reported. I do not know how to check errors in /etc/imapd.conf. I suppose if there are configuration errors in /etc/imapd.conf, cyrus-imapd should log it somewhere when running. The official document says cyrus-imap uses syslog with local6 facility to record logs. I even configured rsyslog to try to catch the log information. In the rsyslog configuration file /etc/rsyslog.d/cyrus.log, I wrote the following lines:

local6.*        /var/log/imapd.log
auth.debug      /var/log/auth.log

The above configuration basically says writing all levels of information from local6 facility to /var/log/imapd.log, and writing debug level log from auth facility to /var/log/auth.log. Unfortunately, I can not see anything recorded in /var/log/imapd.log.

The mailbox automatic creation problem confused me for several days until I switched outlook express to from POP3 to IMAP. After login the IMAP server, the mailbox and its corresponding folders are successfully created automatically. It turns out to be a cyrus-imap bug in the pop3 component, which can not properly handle the user name with rare tld domain names such as an .xyz domain.  The imap component, however, can handle that kind of domains correctly. For common tld domains such as a .com domain, both pop3d and imapd can handle it without a problem.

The above words go through the process we set up cyrus-imap to create mailbox automatically on successful login. Without a mailbox, postfix will fail to deliver the email destined to a local address. The log would be as follows:

Mar 15 06:55:38 domainhostseotool  postfix/smtpd[4070]: EC5CD8178A: client=unknown[xx.xx.xx.xx], sasl_method=LOGIN, sasl_username=admin@domainhostseotool.com
Mar 15 06:55:43 domainhostseotool postfix/cleanup[4074]: EC5CD8178A: message-id=<179B8E1DC0D74EC48B5ADAB77056F11F@ABCD49A888261D>
Mar 15 06:55:43 domainhostseotool postfix/qmgr[17997]: EC5CD8178A: from=<admin@domainhostseotool.com>, size=1525, nrcpt=1 (queue active)
Mar 15 06:55:43 domainhostseotool lmtpunix[4057]: accepted connection
Mar 15 06:55:43 domainhostseotool lmtpunix[4057]: lmtp connection preauth’d as postman
Mar 15 06:55:43 domainhostseotool lmtpunix[4057]: verify_user(user.admin) failed: Mailbox does not exist
Mar 15 06:55:43 domainhostseotool master[4077]: about to exec /usr/lib/cyrus-imapd/lmtpd
Mar 15 06:55:43 domainhostseotool lmtpunix[4077]: executed
Mar 15 06:55:43 domainhostseotool postfix/lmtp[4076]: EC5CD8178A: to=<admin@domainhostseotool.com>, relay=domainhostseotool.com[/var/lib/imap/socket/lmtp], delay=5, delays=4.9/0.01/0.01/0.03, dsn=5.1.1, status=bounced (host domainhostseotool.com[/var/lib/imap/socket/lmtp] said: 550-Mailbox unknown.  Either there is no mailbox associated with this 550-name or you do not have authorization to see it. 550 5.1.1 User unknown (in reply to RCPT TO command))
Mar 15 06:55:43 domainhostseotool postfix/cleanup[4074]: 9766281596: message-id=<20180315105543.9766282396@domainhostseotool.com>
Mar 15 06:55:43 domainhostseotool postfix/bounce[4078]: EC5CD8178A: sender non-delivery notification: 9766281596
Mar 15 06:55:43 domainhostseotool postfix/qmgr[17997]: 9766281596: from=<>, size=3759, nrcpt=1 (queue active)
Mar 15 06:55:43 domainhostseotool postfix/qmgr[17997]: EC5CD8178A: removed
Mar 15 06:55:43 domainhostseotool lmtpunix[4077]: accepted connection
Mar 15 06:55:43 domainhostseotool lmtpunix[4077]: lmtp connection preauth’d as postman
Mar 15 06:55:43 domainhostseotool lmtpunix[4077]: verify_user(user.admin) failed: Mailbox does not exist
Mar 15 06:55:43 domainhostseotool postfix/lmtp[4076]: 9766281596: to=<admin@domainhostseotool.com>, relay=domainhostseotool.com[/var/lib/imap/socket/lmtp], delay=0.03, delays=0.01/0/0/0.01, dsn=5.1.1, status=bounced (host domainhostseotool.com[/var/lib/imap/socket/lmtp] said: 550-Mailbox unknown.  Either there is no mailbox associated with this 550-name or you do not have authorization to see it. 550 5.1.1 User unknown (in reply to RCPT TO command))
Mar 15 06:55:43 domainhostseotool postfix/qmgr[17997]: 9766281596: removed
Mar 15 06:55:44 domainhostseotool postfix/smtpd[4070]: disconnect from unknown[xx.xx.xx.xx]

Analyzing the log is an interesting thing to do. You can observe that after the email is bounced due to not found mailbox, the bounce component generates  a non-delivery notification email (9766281596) to the sender. Because the sender and the recipient are the same in this case(admin@domainhostseotool.com), the notification email also gets bounced. This time, no notification email is produced to avoid endless loop.

We can also instruct cyrus-imap to create mailbox automatically for us when it receives emails from postfix. Just need to add “createonpost:            yes” in /etc/imapd.conf. Note that  there is also a bug related to rare tld (such as .xyz) here. If the recipient email address ends with such tld, the mailbox can not be created automatically, and the email delivery will fail. In such case, you need to create the mailbox manually.  We did the following experiment to prove the automatic creation of email box is possible for common tld. First we add the domainhostseotool.com to the mydestination parameter in /etc/postfix/main.cf, which tells postfix to deliver the email whose recipient is  xxx@domainhostseotool.com locally. Second, we add the following line to main.cf:

local_recipient_maps =

The empty value of local_recipient_maps tells postfix not to reject email to unknown local address such as admin@domainhostseotool.com. Otherwise, you will see the following error:

Recipient address rejected: User unknown in local recipient table;

After restarting postfix, we find we can successfully send emails to admin@domainhostseotool.com, and mailbox for admin@domainhostseotool.com is also created successfully.  The automatically created mailbox uses the directories like: domain/d/domainhostseotool.com/a/user/admin/

 

 

Posted in tips of hosting