Dovecot maildir sync not working after upgrade to 6.9

[William Orr]

Hey,

I have two mailservers running OpenBSD 6.9, and I use bidirectional
syncing of my maildirs through doveadm, part of dovecot. After the
upgrade, I noticed that the sync process was failing.

Here's a sample run:

kefka|~|02:57:05|0$ doas doveadm sync -u ***@worrbase.com remote:sabin.worrbase.com
doveadm(VERSION dsync 3 5): Error: User doesn't exist
dsync-local(worr)<FNQIMvI9omB4VwEADTvxNg>: Error: read(sabin.worrbase.com) failed: EOF (version not received)
dsync-local(worr)<FNQIMvI9omB4VwEADTvxNg>: Error: Remote command returned error 67: /usr/bin/ssh -i /root/.ssh/id_ed25519.dsync sabin.worrbase.com /usr/local/bin/dsync-in-wrapper.sh

kefka|~|02:57:08|75$ cat /usr/local/bin/dsync-in-wrapper.sh
#!/bin/ksh
read username
/usr/local/bin/doveadm dsync-server -u "$username"

I ktraced the process, and noticed that in the communication with the
remote mail server, that a bunch of doveadm plugins fail to load. It's
worth noting that these are plugins that are dlopen(3)ed in response to
certain commands sent over the wire, so they don't show up in ldd(1) output.

kefka|~|03:00:08|0$ doas kdump | grep symbol
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_user_module'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_mailbox_get_aclobj'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_mailbox_list_get_backend'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_object_list_init'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_object_list_next'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_backend_rights_match_me'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_rights_get_id'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_object_list_deinit'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_object_get_my_rights'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_rights_update_import'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_mailbox_update_acl'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_lookup_dict_rebuild'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_backend_nonowner_lookups_iter_init'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_backend_nonowner_lookups_iter_next'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_backend_nonowner_lookups_iter_deinit'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_backend_nonowner_lookups_rebuild'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_lookup_dict_is_enabled'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_lookup_dict_iterate_visible_init'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_lookup_dict_iterate_visible_next'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol 'acl_lookup_dict_iterate_visible_deinit'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol 'quota_user_module'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol 'quota_root_get_resources'
"doveadm:/usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol 'quota_get_resource'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_list_backend'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_backend_lookup'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_backend_lookup_done'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_search_args_expand'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_language_find'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_user_get_language_list'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_language_detect'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_language_list_get_first'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_user_language_find'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_tokenizer_reset'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_tokenizer_final'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_tokenizer_next'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_filter_filter'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_backend_optimize'
"doveadm:/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol 'fts_backend_rescan'
"doveadm:/usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol 'mail_crypt_user_get_public_key'
"doveadm:/usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol 'mail_crypt_user_generate_keypair'
"doveadm:/usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol 'mail_crypt_box_get_public_key'
"doveadm:/usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol 'mail_crypt_box_generate_keypair'
"doveadm:/usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol 'mail_crypt_box_get_pvt_digests'
"doveadm:/usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol 'mail_crypt_acl_secure_sharing_enabled'"
"doveadm:/usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol 'mail_crypt_box_share_private_keys'
"doveadm:/usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol 'mail_crypt_box_set_shared_key'
"doveadm:/usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol 'mail_crypt_get_private_key'

I checked on both machines, and these symbols are all undefined in the
associated libraries. I do notice though, that these symbols seem to be
present in similar dovecot, non-doveadm plugins:

kefka|~|03:04:56|130$ nm -A /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so | grep acl_user_module
/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: U acl_user_module
kefka|~|03:05:14|0$ nm -A /usr/local/lib/dovecot/lib01_acl_plugin.so | grep acl_user_module
/usr/local/lib/dovecot/lib01_acl_plugin.so:0001b008 D acl_user_module

However, the doveadm plugins don't link against them?

kefka|~|03:04:24|1$ ldd /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so
/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so:
Start End Type Open Ref GrpRef Name
000001d89e043000 000001d89e04d000 dlib 1 0 0 /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so

Is anyone else seeing issues like this? Is there perhaps a
misconfiguration on my end?

Both are running dovecot-2.3.14p0v0 from packages with
dovecot-pigeonhole-0.5.14v1.
pkg_check -F showed nothing unusual that was dovecot-related in /usr/local

On both machines, pkg_check -F found the following in /var:
/var/dovecot/anvil
/var/dovecot/anvil-auth-penalty
/var/dovecot/auth-client
/var/dovecot/auth-login
/var/dovecot/auth-master
/var/dovecot/auth-token-secret.dat
/var/dovecot/auth-userdb
/var/dovecot/auth-worker
/var/dovecot/config
/var/dovecot/dict
/var/dovecot/dict-async
/var/dovecot/director-admin
/var/dovecot/director-userdb
/var/dovecot/dns-client
/var/dovecot/doveadm-server
/var/dovecot/dovecot.conf
/var/dovecot/empty
/var/dovecot/imap-hibernate
/var/dovecot/imap-master
/var/dovecot/imap-urlauth
/var/dovecot/imap-urlauth-worker
/var/dovecot/indexer
/var/dovecot/indexer-worker
/var/dovecot/instances
/var/dovecot/ipc
/var/dovecot/lmtp
/var/dovecot/log-errors
/var/dovecot/login/dns-client
/var/dovecot/login/imap
/var/dovecot/login/ipc-proxy
/var/dovecot/login/login
/var/dovecot/master
/var/dovecot/master.pid
/var/dovecot/old-stats
/var/dovecot/old-stats-mail
/var/dovecot/old-stats-user
/var/dovecot/replication-notify
/var/dovecot/replication-notify-fifo
/var/dovecot/replicator
/var/dovecot/replicator-doveadm
/var/dovecot/replicator.core
/var/dovecot/replicator.db
/var/dovecot/ssl-parameters.dat
/var/dovecot/ssl-params
/var/dovecot/stats
/var/dovecot/stats-mail
/var/dovecot/stats-reader
/var/dovecot/stats-user
/var/dovecot/stats-writer
/var/dovecot/token-login

Let me know if I need to provide more info.

Thanks!

[Brad Smith]

Hi William,

I would try posting what you have to the Dovecot mailing list and see what
Aki says.

Post by William Orr
Hey,
I have two mailservers running OpenBSD 6.9, and I use bidirectional
syncing of my maildirs through doveadm, part of dovecot. After the
upgrade, I noticed that the sync process was failing.
remote:sabin.worrbase.com
doveadm(VERSION dsync 3 5): Error: User doesn't exist
dsync-local(worr)<FNQIMvI9omB4VwEADTvxNg>: Error: read(sabin.worrbase.com)
failed: EOF (version not received)
dsync-local(worr)<FNQIMvI9omB4VwEADTvxNg>: Error: Remote command returned
error 67: /usr/bin/ssh -i /root/.ssh/id_ed25519.dsync sabin.worrbase.com
/usr/local/bin/dsync-in-wrapper.sh
kefka|~|02:57:08|75$ cat /usr/local/bin/dsync-in-wrapper.sh
#!/bin/ksh
read username
/usr/local/bin/doveadm dsync-server -u "$username"
I ktraced the process, and noticed that in the communication with the
remote mail server, that a bunch of doveadm plugins fail to load. It's
worth noting that these are plugins that are dlopen(3)ed in response to
certain commands sent over the wire, so they don't show up in ldd(1) output.
kefka|~|03:00:08|0$ doas kdump | grep symbol
undefined symbol 'acl_user_module'
undefined symbol 'acl_mailbox_get_aclobj'
undefined symbol 'acl_mailbox_list_get_backend'
undefined symbol 'acl_object_list_init'
undefined symbol 'acl_object_list_next'
undefined symbol 'acl_backend_rights_match_me'
undefined symbol 'acl_rights_get_id'
undefined symbol 'acl_object_list_deinit'
undefined symbol 'acl_object_get_my_rights'
undefined symbol 'acl_rights_update_import'
undefined symbol 'acl_mailbox_update_acl'
undefined symbol 'acl_lookup_dict_rebuild'
undefined symbol 'acl_backend_nonowner_lookups_iter_init'
undefined symbol 'acl_backend_nonowner_lookups_iter_next'
undefined symbol 'acl_backend_nonowner_lookups_iter_deinit'
undefined symbol 'acl_backend_nonowner_lookups_rebuild'
undefined symbol 'acl_lookup_dict_is_enabled'
undefined symbol 'acl_lookup_dict_iterate_visible_init'
undefined symbol 'acl_lookup_dict_iterate_visible_next'
undefined symbol 'acl_lookup_dict_iterate_visible_deinit'
undefined symbol 'quota_user_module'
undefined symbol 'quota_root_get_resources'
undefined symbol 'quota_get_resource'
undefined symbol 'fts_list_backend'
undefined symbol 'fts_backend_lookup'
undefined symbol 'fts_backend_lookup_done'
undefined symbol 'fts_search_args_expand'
undefined symbol 'fts_language_find'
undefined symbol 'fts_user_get_language_list'
undefined symbol 'fts_language_detect'
undefined symbol 'fts_language_list_get_first'
undefined symbol 'fts_user_language_find'
undefined symbol 'fts_tokenizer_reset'
undefined symbol 'fts_tokenizer_final'
undefined symbol 'fts_tokenizer_next'
undefined symbol 'fts_filter_filter'
undefined symbol 'fts_backend_optimize'
undefined symbol 'fts_backend_rescan'
undefined symbol 'mail_crypt_user_get_public_key'
undefined symbol 'mail_crypt_user_generate_keypair'
undefined symbol 'mail_crypt_box_get_public_key'
undefined symbol 'mail_crypt_box_generate_keypair'
undefined symbol 'mail_crypt_box_get_pvt_digests'
undefined symbol 'mail_crypt_acl_secure_sharing_enabled'"
undefined symbol 'mail_crypt_box_share_private_keys'
undefined symbol 'mail_crypt_box_set_shared_key'
undefined symbol 'mail_crypt_get_private_key'
I checked on both machines, and these symbols are all undefined in the
associated libraries. I do notice though, that these symbols seem to be
kefka|~|03:04:56|130$ nm -A
/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so | grep
acl_user_module
/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: U acl_user_module
kefka|~|03:05:14|0$ nm -A /usr/local/lib/dovecot/lib01_acl_plugin.so |
grep acl_user_module
/usr/local/lib/dovecot/lib01_acl_plugin.so:0001b008 D acl_user_module
However, the doveadm plugins don't link against them?
kefka|~|03:04:24|1$ ldd
/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so
Start End Type Open Ref GrpRef Name
000001d89e043000 000001d89e04d000 dlib 1 0 0
/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so
Is anyone else seeing issues like this? Is there perhaps a
misconfiguration on my end?
Both are running dovecot-2.3.14p0v0 from packages with
dovecot-pigeonhole-0.5.14v1.
pkg_check -F showed nothing unusual that was dovecot-related in /usr/local
/var/dovecot/anvil
/var/dovecot/anvil-auth-penalty
/var/dovecot/auth-client
/var/dovecot/auth-login
/var/dovecot/auth-master
/var/dovecot/auth-token-secret.dat
/var/dovecot/auth-userdb
/var/dovecot/auth-worker
/var/dovecot/config
/var/dovecot/dict
/var/dovecot/dict-async
/var/dovecot/director-admin
/var/dovecot/director-userdb
/var/dovecot/dns-client
/var/dovecot/doveadm-server
/var/dovecot/dovecot.conf
/var/dovecot/empty
/var/dovecot/imap-hibernate
/var/dovecot/imap-master
/var/dovecot/imap-urlauth
/var/dovecot/imap-urlauth-worker
/var/dovecot/indexer
/var/dovecot/indexer-worker
/var/dovecot/instances
/var/dovecot/ipc
/var/dovecot/lmtp
/var/dovecot/log-errors
/var/dovecot/login/dns-client
/var/dovecot/login/imap
/var/dovecot/login/ipc-proxy
/var/dovecot/login/login
/var/dovecot/master
/var/dovecot/master.pid
/var/dovecot/old-stats
/var/dovecot/old-stats-mail
/var/dovecot/old-stats-user
/var/dovecot/replication-notify
/var/dovecot/replication-notify-fifo
/var/dovecot/replicator
/var/dovecot/replicator-doveadm
/var/dovecot/replicator.core
/var/dovecot/replicator.db
/var/dovecot/ssl-parameters.dat
/var/dovecot/ssl-params
/var/dovecot/stats
/var/dovecot/stats-mail
/var/dovecot/stats-reader
/var/dovecot/stats-user
/var/dovecot/stats-writer
/var/dovecot/token-login
Let me know if I need to provide more info.
Thanks!

Sent with Aqua Mail for Android
https://www.mobisystems.com/aqua-mail