Migrating akonadi database from Mageia 4 fails

kleag · August 26, 2015, 4:03pm

Hello,

I have installed OpenSUSE in replacement of Mageia 4. I use the Plasma5 desktop. I installed on the same machine after wiping out the disk and recovered the backup of my .local and .kde4 folders.

When starting akonadictl (from akonadi 4.14.9-24.2), it tries to convert the database and fails. It’s a big problem because I really need to be able to recover years of my work mails.

Thanks in advance for any help.

Gaël

akonadictl:

Starting Akonadi Server... 
   done. 
Connecting to deprecated signal QDBusConnectionInterface::serviceOwnerChanged(QString,QString,QString)
search paths:  (…) 
Found mysql_install_db:  "/usr/bin/mysql_install_db" 
Found mysqlcheck:  "/usr/bin/mysqlcheck" 
akonadi.collectionattributetable                   OK
akonadi.collectionmimetyperelation                 OK
akonadi.collectionpimitemrelation                  OK
akonadi.collectiontable                            OK
akonadi.flagtable                                  OK
akonadi.mimetypetable                              OK
akonadi.parttable                                  OK
akonadi.pimitemflagrelation                        OK
akonadi.pimitemtable                               OK
akonadi.resourcetable                              OK
akonadi.schemaversiontable                         OK
mysql.columns_priv                                 OK
mysql.db                                           OK
mysql.event                                        OK
mysql.func                                         OK
mysql.help_category                                OK
mysql.help_keyword                                 OK
mysql.help_relation                                OK
mysql.help_topic                                   OK
mysql.host                                         OK
mysql.ndb_binlog_index                             OK
mysql.plugin                                       OK
mysql.proc                                         OK
mysql.procs_priv                                   OK
mysql.proxies_priv                                 OK
mysql.servers                                      OK
mysql.tables_priv                                  OK
mysql.time_zone                                    OK
mysql.time_zone_leap_second                        OK
mysql.time_zone_name                               OK
mysql.time_zone_transition                         OK
mysql.time_zone_transition_type                    OK
mysql.user                                         OK
MySQL version OK (required "5.1" , available "10.0" ) 
Database "akonadi" opened using driver "QMYSQL" 
DbInitializer::run() 
checking table  "SchemaVersionTable" 
checking table  "ResourceTable" 
checking table  "CollectionTable" 
"ALTER TABLE CollectionTable ADD COLUMN enabled BOOL NOT NULL DEFAULT true" 
"ALTER TABLE CollectionTable ADD COLUMN syncPref TINYINT DEFAULT 2" 
"ALTER TABLE CollectionTable ADD COLUMN displayPref TINYINT DEFAULT 2" 
"ALTER TABLE CollectionTable ADD COLUMN indexPref TINYINT DEFAULT 2" 
"ALTER TABLE CollectionTable ADD COLUMN referenced BOOL NOT NULL DEFAULT false" 
"ALTER TABLE CollectionTable ADD COLUMN queryAttributes VARBINARY(255)" 
"ALTER TABLE CollectionTable ADD COLUMN queryCollections VARBINARY(255)" 
checking table  "MimeTypeTable" 
checking table  "PimItemTable" 
checking table  "FlagTable" 
checking table  "PartTypeTable" 
"CREATE TABLE PartTypeTable (id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, name VARBINARY(255) NOT NULL, ns VARBINARY(255) NOT NULL)  COLLATE=utf8_general_ci DEFAULT CHARSET=utf8" 
checking table  "PartTable" 
checking table  "CollectionAttributeTable" 
checking table  "TagTypeTable" 
"CREATE TABLE TagTypeTable (id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, name VARBINARY(255) NOT NULL UNIQUE)  COLLATE=utf8_general_ci DEFAULT CHARSET=utf8" 
"INSERT INTO TagTypeTable (id,name) VALUES (1,'PLAIN')" 
checking table  "TagTable" 
"CREATE TABLE TagTable (id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, gid VARBINARY(255) NOT NULL, parentId BIGINT, typeId BIGINT DEFAULT 1, FOREIGN KEY (parentId) REFERENCES TagTable(id) ON UPDATE CASCADE ON DELETE CASCADE, FOREIGN KEY (typeId) REFERENCES TagTypeTable(id) ON UPDATE CASCADE ON DELETE RESTRICT)  COLLATE=utf8_general_ci DEFAULT CHARSET=utf8" 
checking table  "TagAttributeTable" 
"CREATE TABLE TagAttributeTable (id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, tagId BIGINT NOT NULL, type LONGBLOB NOT NULL, value LONGBLOB, FOREIGN KEY (tagId) REFERENCES TagTable(id) ON UPDATE CASCADE ON DELETE CASCADE)  COLLATE=utf8_general_ci DEFAULT CHARSET=utf8" 
checking table  "TagRemoteIdResourceRelationTable" 
"CREATE TABLE TagRemoteIdResourceRelationTable (tagId BIGINT NOT NULL, resourceId BIGINT NOT NULL, remoteId VARBINARY(255) NOT NULL, FOREIGN KEY (tagId) REFERENCES TagTable(id) ON UPDATE CASCADE ON DELETE CASCADE, FOREIGN KEY (resourceId) REFERENCES ResourceTable(id) ON UPDATE CASCADE ON DELETE CASCADE)  COLLATE=utf8_general_ci DEFAULT CHARSET=utf8" 
checking table  "PimItemFlagRelation" 
checking table  "PimItemTagRelation" 
"CREATE TABLE PimItemTagRelation (PimItem_id BIGINT NOT NULL, Tag_id BIGINT NOT NULL, PRIMARY KEY (PimItem_id, Tag_id), FOREIGN KEY (PimItem_id) REFERENCES PimItemTable(id) ON UPDATE CASCADE ON DELETE CASCADE, FOREIGN KEY (Tag_id) REFERENCES TagTable(id) ON UPDATE CASCADE ON DELETE CASCADE)  COLLATE=utf8_general_ci DEFAULT CHARSET=utf8" 
checking table  "CollectionMimeTypeRelation" 
checking table  "CollectionPimItemRelation" 
DbInitializer::run() done 
skipping update 2 
(same thing for 3 to 23)
skipping update 24 
DbUpdater: update to version: 25  mandatory: true 
Starting database update to version 25 
Creating a PartTable_new 
Migrating part types 
     Moved part type "ATR:AddressAttribute" to PartTypeTable 
     Moved part type "ATR:DispatchModeAttribute" to PartTypeTable 
     Moved part type "ATR:MDNStateAttribute" to PartTypeTable 
     Moved part type "ATR:ScamAttribute" to PartTypeTable 
     Moved part type "ATR:SentActionAttribute" to PartTypeTable 
     Moved part type "ATR:SentBehaviourAttribute" to PartTypeTable 
     Moved part type "ATR:TransportAttribute" to PartTypeTable 
     Moved part type "ATR:contactmetadata" to PartTypeTable 
     Moved part type "PLD:ENVELOPE" to PartTypeTable 
     Moved part type "PLD:HEAD" to PartTypeTable 
     Moved part type "PLD:RFC822" to PartTypeTable 
Migrating data from PartTable to PartTable_new 
Swapping PartTable_new for PartTable 
Removing PartTable_old 
Final tuning of new PartTable 
Update done in 15820 ms 
DbUpdater: update to version: 26  mandatory: false 
DbUpdater: update to version: 28  mandatory: false 
Updating indexes 
"CREATE  INDEX CollectionTable_enabledIndex ON CollectionTable (enabled)" 
"CREATE  INDEX CollectionTable_syncPrefIndex ON CollectionTable (syncPref)" 
"CREATE  INDEX CollectionTable_displayPrefIndex ON CollectionTable (displayPref)" 
"CREATE  INDEX CollectionTable_indexPrefIndex ON CollectionTable (indexPref)" 
"CREATE  INDEX PimItemTable_ridIndex ON PimItemTable (remoteId)" 
"CREATE UNIQUE INDEX PartTypeTable_partTypeNameIndex ON PartTypeTable (ns,name)" 
"CREATE UNIQUE INDEX PartTable_pimItemIdTypeIndex ON PartTable (pimItemId,partTypeId)" 
"CREATE  INDEX TagAttributeTable_tagIndex ON TagAttributeTable (tagId)" 
"CREATE UNIQUE INDEX TagRemoteIdResourceRelationTable_TagAndResourceIndex ON TagRemoteIdResourceRelationTable (tagId,resourceId)" 
Updating index failed:  
Sql error: Lost connection to MySQL server during query QMYSQL: Unable to execute query
Query: CREATE UNIQUE INDEX TagRemoteIdResourceRelationTable_TagAndResourceIndex ON TagRemoteIdResourceRelationTable (tagId,resourceId) 
""
Unable to initialize database.
"
0: akonadiserver(_Z11akBacktracev+0x37) [0x46d747]
1: akonadiserver() [0x46d9a2]
2: /lib64/libc.so.6(+0x35200) [0x7fa4839a2200]
3: /lib64/libc.so.6(gsignal+0x37) [0x7fa4839a2187]
4: /lib64/libc.so.6(abort+0x118) [0x7fa4839a3538]
5: /usr/lib64/libQtCore.so.4(_Z17qt_message_output9QtMsgTypePKc+0x64) [0x7fa4851a12b4]
6: akonadiserver(_ZN15FileDebugStream9writeDataEPKcx+0x9d) [0x46fb5d]
(snip)
]
"
ProcessControl: Application 'akonadiserver' returned with exit code 255 (Unknown error)

(snip subsequent tries)

"akonadiserver" crashed too often and will not be restarted!

mysql.err:

2015-08-26 15:38:25 7ff68eeeb740 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.
150826 15:38:25 [Note] InnoDB: Using mutexes to ref count buffer pool pages
150826 15:38:25 [Note] InnoDB: The InnoDB memory heap is disabled
150826 15:38:25 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
150826 15:38:25 [Note] InnoDB: Memory barrier is not used
150826 15:38:25 [Note] InnoDB: Compressed tables use zlib 1.2.8
150826 15:38:25 [Note] InnoDB: Using Linux native AIO
150826 15:38:25 [Note] InnoDB: Using CPU crc32 instructions
150826 15:38:25 [Note] InnoDB: Initializing buffer pool, size = 80.0M
150826 15:38:25 [Note] InnoDB: Completed initialization of buffer pool
150826 15:38:25 [Note] InnoDB: Highest supported file format is Barracuda.
150826 15:38:25 [Note] InnoDB: Log scan progressed past the checkpoint lsn 39834001509
150826 15:38:25 [Note] InnoDB: Database was not shutdown normally!
150826 15:38:25 [Note] InnoDB: Starting crash recovery.
150826 15:38:25 [Note] InnoDB: Reading tablespace information from the .ibd files...
150826 15:38:25 [Note] InnoDB: Restoring possible half-written data pages 
150826 15:38:25 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Doing recovery: scanned up to log sequence number 39839244288
InnoDB: Doing recovery: scanned up to log sequence number 39844487168
InnoDB: Doing recovery: scanned up to log sequence number 39849730048
InnoDB: Doing recovery: scanned up to log sequence number 39854972928
InnoDB: Doing recovery: scanned up to log sequence number 39860215808
InnoDB: Doing recovery: scanned up to log sequence number 39865458688
InnoDB: Doing recovery: scanned up to log sequence number 39870701568
InnoDB: Doing recovery: scanned up to log sequence number 39875944448
InnoDB: Doing recovery: scanned up to log sequence number 39881187328
InnoDB: Doing recovery: scanned up to log sequence number 39886430208
150826 15:38:25 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
InnoDB: Apply batch completed
InnoDB: Doing recovery: scanned up to log sequence number 39891673088
InnoDB: Doing recovery: scanned up to log sequence number 39896915968
InnoDB: Doing recovery: scanned up to log sequence number 39902158848
InnoDB: Doing recovery: scanned up to log sequence number 39907401728
InnoDB: Doing recovery: scanned up to log sequence number 39912644608
InnoDB: Doing recovery: scanned up to log sequence number 39917887488
InnoDB: Doing recovery: scanned up to log sequence number 39923130368
InnoDB: Doing recovery: scanned up to log sequence number 39927375587
150826 15:38:36 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
InnoDB: Apply batch completed
150826 15:38:42 [Note] InnoDB: 128 rollback segment(s) are active.
150826 15:38:42 [Note] InnoDB: Waiting for purge to start
2015-08-26 15:38:42 7ff653fff700  InnoDB: Assertion failure in thread 140695947966208 in file trx0purge.cc line 703
InnoDB: Failing assertion: purge_sys->iter.trx_no <= purge_sys->rseg->last_trx_no
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
150826 15:38:42 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see http://kb.askmonty.org/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.

Server version: 10.0.20-MariaDB
key_buffer_size=134217728
read_buffer_size=131072
max_used_connections=0
max_threads=258
thread_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 697561 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x48000
/usr/sbin/mysqld(my_print_stacktrace+0x29)[0x7ff68f833769]
…

wolfi323 · August 26, 2015, 4:10pm

You don’t need to “migrate” the Akonadi database, you can just delete it (~/.local/share/akonadi/) to solve your problem.
This is a cache only, Akonadi doesn’t store EMails in there (unless it cannot synchronize them back to the actual store).

For the local maildir resource, the EMails should be in ~/.local/share/local-mail by default.
But if you copied the config files, this should be configured correctly anyway.
Btw, you might need to copy over ~/.config/akonadi/ too.

kleag · August 26, 2015, 5:14pm

Thank you for your answer. This is what I was thinking but an attempt to not use the akonadi backup gave me access to a part of my saved mails only. I then when to the wrong direction trying to migrate the database…

Now, I tried again to start with no akonadi database. This time, it seems to start but when I launch kmail, it stops saying

 The Email program encountered a fatal error and will terminate now.
 The error was:

 Failed to fetch the resource collection.

If I look at /home/gael/.local/share/akonadi/akonadiserver.error, I get:


DATABASE ERROR: 
  Error code: 1062 
  DB error:  "Duplicate entry '45-T\xEF\xBF\xBDches' for key 'CollectionTable_parentAndNameIndex'" 
  Error text: "Duplicate entry '45-T\xEF\xBF\xBDches' for key 'CollectionTable_parentAndNameIndex' QMYSQL3: Unable to execute statement" 
  Query: "INSERT INTO CollectionTable (remoteId, remoteRevision, name, parentId, resourceId, enabled, syncPref, displayPref, indexPref, cachePolicyInherit, isVirtual) VALUES (:0, :1, :2, :3, :4, :5, :6, :7, :8, :9, :10)"

So it seems that the import of my old mails fails. But I don’t know where to look at to solve this new problem…

wolfi323 · August 26, 2015, 5:47pm

Well, as I wrote, mails that were not synchromized back might only exist in the cache. But that should not happen normally.

So it seems that the import of my old mails fails. But I don’t know where to look at to solve this new problem…

So this is about a local maildir?
Try to start from scratch with removing ~/.config/akonadi/ and ~/.local/share/akonadi/.

And try to refresh all folders manually in akonadiconsole (right-click on them on the “Browse” tab)

kleag · August 26, 2015, 11:54pm

I’m done ! I got my mails back, thanks.

I finally destroyed akonadi config as you proposed and imported my mails folders as maildir. I completely forgot this part since the last time a started from a fresh install (from Mandriva to Mageia2 probably).

I don’t know why but I just had to remove the (empty) cur, new and tmp folders in the maildir folder for the import to work.

Thanks again

wolfi323 · August 27, 2015, 11:31am

Good to hear!

I don’t know why but I just had to remove the (empty) cur, new and tmp folders in the maildir folder for the import to work.

Hm, I would be surprised if that was necessary, but well. If it helped, fine…