Akonadi server trouble: Kontact crashing on startup

When I start Kontact on opensuse tumbleweed it tells me it can’t start the Akonadi Server. More information:

me@PC:~> akonadictl start
org.kde.pim.akonadictl: Starting Akonadi Server...
org.kde.pim.akonadictl: done.
Connecting to deprecated signal QDBusConnectionInterface::serviceOwnerChanged(QString,QString,QString)
me@PC:~> org.kde.pim.akonadiserver: Starting up the Akonadi Server...
org.kde.pim.akonadiserver: database server stopped unexpectedly
org.kde.pim.akonadiserver: Database process exited unexpectedly during initial connection!
org.kde.pim.akonadiserver: executable: "/usr/sbin/mysqld"
org.kde.pim.akonadiserver: arguments:  ("--defaults-file=/home/matthias/.local/share/akonadi/mysql.conf",  "--datadir=/home/matthias/.local/share/akonadi/db_data/",  "--socket=/run/user/1000/akonadi/mysql.socket",  "--pid-file=/run/user/1000/akonadi/mysql.pid")
org.kde.pim.akonadiserver: stdout: ""
org.kde.pim.akonadiserver: stderr: "2021-05-03 18:06:12 0 [Note]  /usr/sbin/mysqld (mysqld 10.5.9-MariaDB) starting as process 16523  ...
"
org.kde.pim.akonadiserver: exit code: 6
org.kde.pim.akonadiserver: process error: "Process crashed"
org.kde.pim.akonadiserver: Shutting down AkonadiServer...
org.kde.pim.akonadicontrol: Application '/usr/bin/akonadiserver' exited normally...

Until a few days ago everything was fine. A bunch of searches in the net brought no results. Any ideas?

On https://dev.mysql.com/doc/mysql-enterprise-backup/4.1/en/meb-exitcodes.html you can see that error 6 means “Mismatch in config and the value obtained” but that does not tell me much more.

What you can try is to follow the tips on https://fedoraproject.org/wiki/How_to_debug_MySQL_and_MariaDB

Your dump shows exactly how the systemd is trying to start the server.

Same here…

Well thanks, but I didn’t find the specific kind of mismatch mentioned above in this either. I can start mariadb by hand via “systemctl start mariadb”, but akonadi still crashes when I start kontact thereafter.

More ideas very welcome ;).

I forgot: ~/.local/share/akonadi/mysql.conf only has “notes” and no errors.

And then there is

me@PC:~> cat /usr/lib/tmpfiles.d/mariadb.conf 
d /run/mysql 0755 mysql mysql - 
x /var/tmp/mysql.*

Is that erroneous?

Found something I can’t interpret or rather I don’t know how to fix in ~/.local/share/akonadi/mysql.err (btw. is there any chance to append a file to a post?):

2021-05-04 17:24:02 0 [ERROR] 
most similar warnings and errors omitted. Please note if you need them.
...
2021-05-04 17:24:00 0 [Warning] InnoDB: Ignoring a doublewrite copy of  page [page id: space=41, page number=485] with future log sequence  number 68762837647
2021-05-04 17:24:00 0 [Warning] InnoDB: Ignoring a doublewrite copy of  page [page id: space=41, page number=577] with future log sequence  number 68777531536
...
InnoDB: Page [page id: space=0, page number=53] log sequence number 68781953928 is in the future! Current system log sequence number 67799688273.
2021-05-04 17:24:02 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
2021-05-04 17:24:02 0x7f16e8873800  InnoDB: Assertion failure in file /home/abuild/rpmbuild/BUILD/mariadb-10.5.9/storage/innobase/include/fut0lst.h line 127
InnoDB: Failing assertion: addr.page == FIL_NULL || addr.boffset >= FIL_PAGE_DATA
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
InnoDB: about forcing recovery.
210504 17:24:02 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.

Server version: 10.5.9-MariaDB
key_buffer_size=16384
read_buffer_size=131072
max_used_connections=0
max_threads=258
thread_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 567957 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x49000
??:0(my_print_stacktrace)[0x56083a89592b]
??:0(handle_fatal_signal)[0x56083a4080b5]
??:0(__restore_rt)[0x7f16e903fa30]
??:0(__GI_raise)[0x7f16e8b184a5]
??:0(__GI_abort)[0x7f16e8b01864]
??:0(wsrep_write_dummy_event_low(THD*, char const*))[0x56083a034612]
??:0(wsrep_write_dummy_event_low(THD*, char const*))[0x56083a00e78c]
??:0(wsrep_write_dummy_event_low(THD*, char const*))[0x56083a00d645]
??:0(wsrep_write_dummy_event_low(THD*, char const*))[0x56083a030308]
??:0(wsrep_write_dummy_event_low(THD*, char const*))[0x56083a030a0e]
??:0(wsrep_write_dummy_event_low(THD*, char const*))[0x56083a033914]
??:0(wsrep_write_dummy_event_low(THD*, char const*))[0x56083a02eafb]
??:0(Wsrep_server_service::log_state_change(wsrep::server_state::state, wsrep::server_state::state))[0x56083a69e281]
??:0(ha_initialize_handlerton(st_plugin_int*))[0x56083a40cdde]
??:0(sys_var_pluginvar::do_value_ptr(THD*, enum_var_type, st_mysql_const_lex_string const*))[0x56083a1dbefe]
??:0(plugin_init(int*, char**, int))[0x56083a1dfa16]
??:0(unireg_abort)[0x56083a0a116b]
??:0(mysqld_main(int, char**))[0x56083a0a65ff]
??:0(__libc_start_main)[0x7f16e8b02b25]
??:0(_start)[0x56083a091dae]
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
information that should help you find out what is causing the crash.
Writing a core file...
Working directory at /home/matthias/.local/share/akonadi/db_data
Resource Limits:
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        unlimited            unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             47426                47426                processes 
Max open files            3493                 3493                 files     
Max locked memory         65536                65536                bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       47426                47426                signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us        
Core pattern: |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h

Reading what you posted under code I see a good suggestion:

Please refer to InnoDB Recovery Modes - MariaDB Knowledge Base for information about forcing recovery.

Yes, you can post a log file using susepaste, see “man susepaste” on how to operate it.

Yeah, read it but didn’t quite dare. I have none experience whatsoever with data bases. What is more, the data base shouldn’t be corrupted, the problem occurred after an update of the prorgrams (akonadi, etc.). I don’t want to repair what is whole, ignoring the true problem…

I see why you think the data base shouldn’t be corrupted, but it is:

2021-05-04 17:24:00 0 [Warning] InnoDB: Ignoring a doublewrite copy of  page [page id: space=41, page number=485] with future log sequence  number 68762837647
2021-05-04 17:24:00 0 [Warning] InnoDB: Ignoring a doublewrite copy of  page [page id: space=41, page number=577] with future log sequence  number 68777531536
...
InnoDB: Page [page id: space=0, page number=53] log sequence number 68781953928 is in the future! Current system log sequence number 67799688273.

That is way above my head. I still don’t know what command to issue with which option(s) nor do i know name and path to the data base to act on :(. Also, this seems for the case when mariadb can’t start. I can start it though per systemctl…

Oh dear. Well. Command line … wasn’t there a button somewhere here?

Yes, unfortunately, for debugging things the command line is essential, but for susepaste, just have a look at https://susepaste.org/

On the “command to issue” that is indeed no longer relevant now we know the database is corrupt.

On this page I read:

since akonadi is mostly temporary data, I think the easiest is to kill the db_data directory and start over.

Not sure if you want to go that way but I think it is worth a try:

$ cd ~/.local/share/akonadi
$ mv db_data db_databkp 
$ mkdir db_data
$ /usr/sbin/mysqld-akonadi --defaults-file=/home/matthias/.local/share/akonadi/mysql.conf --datadir=/home/matthias/.local/share/akonadi/db_data/ --socket=/run/user/1000/akonadi/default/mysql.socket --pid-file=/run/user/1000/akonadi/default/mysql.pid --initialize --console

Then check if user’s emails, contacts, calendars, events, journals, alarms, notes, etc. are fine in KDE (that is what Akondi is managing) and if so you can delete db_databkp.

If you are missing things you can set back the (corrupted) database using:

$ cd ~/.local/share/akonadi
$ rm db_data
$ mv db_databkp db_data 

I do not know of way to do this using the GUI.

Okay, I followed your advice and Korganizer doesn’t crash anymore :). Hint: the red-colored part gives an error. Actually /usr/sbin/mysqld is a softlink to /usr/sbin/mariadbd so I just omitted the red part and sonething happened, i.e. hard disk activity. My calendar is not back so I will log out and in again and have a look then.

No problem, I am not adverse to the command line, earlier I just overlooked that pastebin will provide a link to the file.

My Calendar is not back :(. Nasty, because I have all my appointments in it…

Btw. The calendar is not shown in the left side panel where added calendars appear. I had a name of a specific date (I don’t remember exactly which one).

I restored a db_data file from a backup in March, where KOrganizer definitely worked. No joy either.

Well, I found a way to help myself. To make a long story short, I restored ~/.local/share/akonadi (about 1.4 GB) from a backup. Regrettably it took about 10 days to understand the situation an to come to this solution / workaround. The density of my backups thins with time, so I lost about 12 days of calendar entries and had to reorganize my sent mail folder. But I’m back in business, thank god. I wish I understood the true cause, though.

Thanks everyone for your help :).

Akonadi is crashing again. Seems to corrupt my mail and/or calendar data and then stumbles during startup. So I’m afraid I have to change over to Thunderbird (from which I came about 5 years ago when TB’s calendar crashed…).

Oh, and I tried mysqlcheck and at last the innodb-force-recovery – to no avail:

me@PC:~> mysqlcheck --all-databases 
mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/run/mysql/mysql.sock' (2) when trying to connect 
me@PC:~> sudo mysqlcheck --all-databases 
mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/run/mysql/mysql.sock' (2) when trying to connect 


me@PC:~> mariadb --innodb-force-recovery=1 
mariadb: unknown variable 'innodb-force-recovery=1' 
me@PC:~> mysqld --innodb-force-recovery=1      
Der absolute Pfad für 'mysqld' ist '/usr/sbin/mysqld', daher kann das Programm möglicherweise nur von Benutzern mit Superuser-Rechten gestartet werden
 (z. B. root). 
me@PC:~> sudo mysqld --innodb-force-recovery=1 
2021-07-06 14:53:50 0 [Note] mysqld (mysqld 10.5.10-MariaDB) starting as process 11546 ... 
mysqld: Please consult the Knowledge Base to find out how to run mysqld as root! 
2021-07-06 14:53:50 0 [ERROR] Aborting 
me@PC:~> systemctl start mysqld --innodb-force-recovery=1      
systemctl: unrecognized option '--innodb-force-recovery=1' 
me@PC:~> systemctl start 'mysqld --innodb-force-recovery=1' 
Invalid unit name "mysqld --innodb-force-recovery=1" escaped as "mysqld\x20--innodb-force-recovery\x3d1" (maybe you should use systemd-escape?). 
Failed to start mysqld\x20--innodb-force-recovery\x3d1.service: Unit mysqld\x20--innodb-force-recovery\x3d1.service not found. 
me@PC:~> systemctl start "mysqld --innodb-force-recovery=1"  
Invalid unit name "mysqld --innodb-force-recovery=1" escaped as "mysqld\x20--innodb-force-recovery\x3d1" (maybe you should use systemd-escape?). 
Failed to start mysqld\x20--innodb-force-recovery\x3d1.service: Unit mysqld\x20--innodb-force-recovery\x3d1.service not found. 
me@PC:~> systemctl start mysqld\ --innodb-force-recovery=1  
Invalid unit name "mysqld --innodb-force-recovery=1" escaped as "mysqld\x20--innodb-force-recovery\x3d1" (maybe you should use systemd-escape?). 
Failed to start mysqld\x20--innodb-force-recovery\x3d1.service: Unit mysqld\x20--innodb-force-recovery\x3d1.service not found.

I’m just no database guy, wrong syntax and all…

Hello,

After a Tumbleweed Update I had the same problem. The first three or five boots kontact and akonadi worked correctly and then this error. I found a post, which helped me verry much:

https://forums.gentoo.org/viewtopic-p-8585883.html?sid=35bd991c0273e155453f7779b3fa6ae5

From this:

…]
In the folder .local/share/akonadi/db_data/ I had two files:
ib_logfile0
and
ib_logfile1
renamed both to *.old and kmail/akonadi works again

I had only one such file, but it worked for me (-:

Tobias

I tried that, Akonadi is still crashing. Thanks anyway, Tobias :).