Dovecot upgrade to v2.4 caused it to stop showing Inbox

Hi,

I’ve been banging my head against the wall for few hours but no solution yet. I upgraded the mail server VM and dovecot got upgraded to 2.4.1. Of course it did not start anymore because the config was not compatible anymore with 2.4.1. Managed to fix most of the stuff but Inbox remains out.

This is the config (doveconf -n):

doveconf -n
# 2.4.1-4 (7d8c0e5759): /etc/dovecot/dovecot.conf
# Pigeonhole version 2.4.1-4 (0a86619f)
# OS: Linux 6.16.1-1-default x86_64  
# Hostname: localhost
# 2 default setting changes since version 2.4.1
dovecot_config_version = 2.4.1
auth_cache_negative_ttl = 0
auth_cache_size = 1024
auth_cache_ttl = 1 hour
auth_debug = yes
auth_mechanisms = plain login
auth_verbose = yes
auth_verbose_passwords = sha1
dovecot_storage_version = 2.4.1
mail_debug = yes
mail_driver = mbox
mail_gid = mail
mail_home = /home/%(user)/
mail_inbox_path = /var/mail
mail_path = ~/mail
mail_privileged_group = mail
protocols = imap
ssl_cipher_list = ALL:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!RC4:!ADH:!LOW@STRENGTH
passdb pam {
  driver = pam
  failure_show_msg = yes
  session = yes
}
userdb passwd {
  driver = passwd
}
service director {
  unix_listener login/director {
  }
  fifo_listener login/proxy-notify {
  }
  unix_listener director-userdb {
  }
}
service imap-login {
  inet_listener imap {
  }
  inet_listener imaps {
  }
  inet_listener https {
    port = 443
    ssl = yes
  }
}
service pop3-login {
  inet_listener pop3 {
  }
  inet_listener pop3s {
  }
}
service submission-login {
  inet_listener submission {
  }
}
namespace inbox {
  inbox = yes
  type = private
  mailbox Drafts {
    special_use = "\\Drafts"
  }
  mailbox Junk {
    special_use = "\\Junk"
  }
  mailbox Trash {
    special_use = "\\Trash"
  }
  mailbox Sent {
    special_use = "\\Sent"
  }
  mailbox "Sent Messages" {
    special_use = "\\Sent"
  }
}
mbox {
  read_locks = fcntl
}
service lmtp {
  unix_listener lmtp {
  }
}
service imap {
}
service pop3 {
}
service submission {
}
service auth {
  unix_listener auth-userdb {
  }
}
service auth-worker {
}
service dict {
  unix_listener dict {
  }
}
ssl_server {
  cert_file = /etc/ssl/private/dovecot.crt
  dh_file = /etc/ssl/private/dh2048.pem
  key_file = /etc/ssl/private/dovecot.pem
}

Inbox (mbox) is located at /var/mail/. One file per user.

Mails are otherwise located at /home//mail.

What is wrong with the config - can anyone help out?

Additional information:

journalctl -u dovecot -f gives following error:

imap(user)<4337><K6AoUg09WpsKAQoK>: 
Debug: Mailbox INBOX: Mailbox opened
Debug: Namespace inbox: Using permissions from /home/user/mail: mode=0755 gid=default
Debug: Mailbox INBOX: Couldn't open mailbox in list index: Refresh-flag set
Error: Mailbox INBOX: Opening INBOX failed: Mailbox isn't selectable
Debug: Mailbox INBOX: Couldn't open mailbox in list index: Refresh-flag set

Managed to “solve” the problem by symlinkin /var/mail/ file to /home//mail/INBOX. But to my understanding, Dovecot will remove symlink support at some point so not a good solution either. But at least gives some time to figure out better solution for system emails and fetchmail (this one probably can be easily configured to fetch emails directly to /home//mail/INBOX).

Major issue. When symlinking /var/mail/ to /home//mail/INBOX resulted bad failure. Edison Android email client can now access incorrect person’s INBOX. Thunderbird works fine but not Edison. Irrelevant of the Android app, this should not be even remotely possible.

Only one user needs to get system messages thus those emails can be located at /var/mail/. But rest can be placed to /home//mail/INBOX. I tried to get this to work with fetchmail with addition:

“/usr/lib/dovecot/deliver -d %T -c /home/fetchmail”

Full poll config at fetchmailrc:

poll imap.gmx.com protocol IMAP:
    user 'user' with password 'pw' is 'user' here mda "/usr/lib/dovecot/deliver -d %T -c /home/fetchmail"
    folder 'INBOX'                     # Fetch INBOX
    folder 'Spam'                      # Fetch Spam folder
    folder 'Sent'                      # Fetch Sent folder

With or without -c /home/fetchmail it does not work. I always end up having same error in fetchmail log:

doveconf: Fatal: open(/home/fetchmail) failed: Permission denied

How to solve this problem?

I’m running fetchmail as a service to fetch emails for all users. Tried to disable apparmor as well but no effect.

Correction… any email client results reading from users INBOX so Dovecot messes up with symlink.

That cannot be right. If your mail is really delivered into /var/mail, this probably should be /var/mail/%{user}. Otherwise it is up to you to know where INBOX for your user(s) is located.

Disclaimer - I am not familiar with Dovecot, I am familiar with UNIX and mail management in UNIX in general.

Nope. This results a file (%user) to be created in /var/mail/ folder. Tried this many many times when sorting out correct dovecot config.

Now this should be back in operation…

auth_cache_size was a problem. By setting this to 0 (was 1024) the problem with mixing user accounts was solved.

Symlinking (even support for this will be removed later by dovecot) /var/mail/ to /home//mail/inbox solved the inbox problem. Struggled a lot with this because I was linking to INBOX and not to inbox. With detailed debug information from doveamd -Dv fetch -u text mailbox INBOX all revealed that dovecot was trying to access lower case inbox file and not capital cased.

Some new issues with dovecot crashing needs to be sorted next… pgp keys are obsolete thus zypper doesn’t work… all at the same time. Nice…

After several days mail server is back up and running. This was not a nice exercise to say the least.

Ended up having original VM disc full and not very easily resizable (due to bad location of swap originally). I’ve done resizing once before on different VM and that was not nice. Additional struggles. So, ended up creating a new VM for this with bigger disc size and swap before main partition thus if needed, it’ll be quite easy to expand later.

Tumbleweed distro with server approach i.e. no GUI.

Major problems with dovecot config - but also errors in postfix config. Interestingly these were not problems in the old installation but in new one caused big issues. One being for example automatic bounce message that failed due lack of qualified domain name etc. This did not happen earlier.

Once after many changes I finally got the dovecot to show mailboxes correctly. Inboxes were symlinked to /var/mail/ which would not be supported later by dovecot. So, I decided to change that fetchmail transports emails directly to /home//mail/inbox. My goodness what a problem that became to solve. Permission issues after permission issues. Also, issues with fetching (not seeing emails i.e. indexing issues) - one user worked fine but the other one did not. And never got the mda to work with server running single fetchmail and single fetchmailrc config and transport mails to /home//mail/inbox (mbox format). Step back, and run fetchmail per user. Quite some struggle still but finally got server per user up and running and zero permission issues. This should be now on a level that won’t be an issue any time soon with dovecot.

Things that worked before but no more:

  • ssl encrypted email transmission locally (not needed but this caused problems in new config and once understood, ssl was removed and problem gone)
  • dovecot delivery (mailbox_command in postfix) caused bounce messages (mails were transmitted outside localhost)

And even more strange things happened:

Email accounts got mixed. This was solved by setting auth cache size to zero in dovecot config. BUT, most likely the original problem was duplicate IP addresses. Two VMs had same IP address. How? Well, not 100% sure but I’ll just explain what I saw. The new VM had originally DHCP for IP address setting. Changed it to static: 10.1.10.15. On a purpose to different than mail-server: 10.1.10.11. Target was to change the new one to 10.1.10.11 once ready and old one to 10.1.10.15. This way no need to make changes to router config.

I used nmcli to change (not add, nmcli con mod) the ipv4.address and saw that it changed. Then got some troubles but fixed those and network was up and running. Now it was running with fixed IP. Fine. Didn’t check further.

Later when new server was ready, I decided to change the old server IP to 10.1.10.14 and new to 10.1.10.11. Again modification with nmcli to new one and with yast to the old one. What really happened was that both nmcli and yast actually added new IP for the server. So, I had new server with IPs: 10.1.10.11 and 10.1.10.15 and old one with 10.1.10.11 and 10.1.10.14. Why there was no conflict with this setting? I don’t know. But most likely this caused issues with clients messing up the communication to mail server which IP is 10.1.10.11. With messing up I mean mixed user accounts etc. when data was fetched by the client from IMAP server - which was both?

Another stupid thing was that I had both old and new server up and running with proper different IPs and dovecot was running find but fetchmail was causing problems. This was during the permission denied era. I sent several test emails to external mailbox to fetch them back. Noticed that sometimes the email had gone already but usually there was a problem with mda. After several hours I realized that heck, I have fetchmail running on both servers and every now and then the old server fetched the test emails. Well, this is what you get when trying to solve problems at very late hour and being tired.

Maybe one day I’ll ditch fetchmail and replace it something modern that works well with systemd but now is not the time. I still have one issue remaining that needs to be sorted out… Thunderbird shows Sent and Sent messages. Cannot get rid of Sent messages folder…