akonadi does not start after data restore

Hi,

year 2017 started as 2016 ended - with a hardware problem. This time the SSD died one month after warranty period is over.
On an new HD I did a clean installation, all patches applied, and restored the whole user directory from a backup.
Trying to start akonadi:


docb@linux-r65g:~> akonadictl start
Connecting to deprecated signal QDBusConnectionInterface::serviceOwnerChanged(QString,QString,QString)
docb@linux-r65g:~> log_akonadiserver: Database process exited unexpectedly during initial connection!
log_akonadiserver: executable: "/usr/sbin/mysqld"
log_akonadiserver: arguments: ("--defaults-file=/home/docb/.local/share/akonadi/mysql.conf", "--datadir=/home/docb/.local/share/akonadi/db_data/", "--socket=/tmp/akonadi-docb.Pqv1Br/mysql.socket")
log_akonadiserver: stdout: ""
log_akonadiserver: stderr: "170106 11:44:24 [Note] /usr/sbin/mysqld (mysqld 10.0.28-MariaDB) starting as process 2173 ...
"
log_akonadiserver: exit code: 1
log_akonadiserver: process error: "Unknown error"
/usr/bin/mysqladmin: connect to server at 'localhost' failed
error: 'Can't connect to local MySQL server through socket '/tmp/akonadi-docb.Pqv1Br/mysql.socket' (2 "No such file or directory")'
Check that mysqld is running and that the socket: '/tmp/akonadi-docb.Pqv1Br/mysql.socket' exists!
log_akonadiserver: Failed to remove Unix socket
log_akonadiserver: Failed to remove runtime connection config file
log_akonadicontrol: Application 'akonadiserver' exited normally...

of course the /tmp/akonadi… directory is empty.
mysql|mariadb seems not to run…I’m no db-expert, can one explain why it is not coming up?

Thanks!

You probably need to recreate the tmp folder. Then akonadi can put it’s socket files there.

Hi Gertjan,
temp folder is there…thats not the point

Hey Axel, If mysql.socket is in there, remove it. Then start akonadi. I’ve seen this a couple of times, will have a look if this has been reported already.

I had tried this as well. If mysql is started manually it comes up, but akonadi does not connect.
I have filed https://bugs.kde.org/show_bug.cgi?id=374636 in between, lets see…

I do see a weird thing: the hash in the tmp folder is different. Sure you removed the socket file from the right folder?

Yes, all /tmp/akonadi* folders are emtpy!

I found a mysql log file that contains some information:

...
InnoDB: Trx id counter is 514211072
170106 19:44:20 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
InnoDB: Apply batch completed
InnoDB: Cleaning up trx with id 514211238
2017-01-06 19:44:21 7f033cae6740  InnoDB: Assertion failure in thread 139651879692096 in file trx0trx.cc line 462
InnoDB: Failing assertion: trx->update_undo == NULL
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
170106 19:44:21 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

I tried the mentioned innodb-recovery in ~/.local/share/akonadi/mysql.conf , and in level it came up again:

innodb_force_recovery=3

Thats the good news. Bad news is that still some is not working:

log_akonadiserver: Database “akonadi” opened using driver “QMYSQL”
log_akonadiserver: DATABASE ERROR:
log_akonadiserver: Error code: 1036
log_akonadiserver: DB error: “Table ‘pimitemtable’ is read only”
log_akonadiserver: Error text: “Table ‘pimitemtable’ is read only QMYSQL3: Unable to execute statement”
log_akonadiserver: Query: “UPDATE PimItemTable SET atime = :0 WHERE ( ( PimItemTable.id = :1 ) )”
log_akonadiserver: Unable to update item access time

I continue investigating…

not sure if this is of any use: https://docs.kde.org/trunk5/en/pim/kmail/clean-start-after-a-failed-migration.html

Any backups from your mysql install ?

In this case I would lose all local accounts/information. Thats what I try to avoid. But thanks anyway!

Finally I did not find a solution to get akonadi up and running again, but a workaround: I took an older backup and inserted all akonadi-related parts into the current restore. As this older backup was taken from the root account, akonadi was not running in the user-session. So I could get back most of my mail.

That leads to some learnings and question…

  1. Never store mails , contacts or appointsments locally
  2. have a separate mail account the can serve as backup (for those stuff you would normally store locally)
  3. When doing a backup of your user-data, dont do it from a running session. Do it e.g. from root-account
  4. Is anyone running a permanent backup, similar to time-machine on OS-X? How is transactional integrity ensured if akonadi is still running?

.local/share/akonadi is a cache. You always can delete it without losing mail. Your mail is stored by default in .local/share/local-mail. Backup this directory.