I use openSUSE-Tumbleweed on my personal dev. computer. I am facing this
weird issue with zypper, where trying to do any operation using zypper
(update, search, install, etc.) becomes painfully slow. In particular,
zypper gets stuck for a while in “Reading installed packages…”.
Note that the operation does not get stuck indefinitely (as suggested here);
it does come out of the “loop” but only after a while. CPU utilization is
high (80-100%) while it’s stuck in the “loop”. For example, see the output
below:
$ sudo time zypper update libboost*
Loading repository data...
Reading installed packages...
'libboost_thread1_62_0' providing 'libboost*' is already installed.
Package 'libboost_thread1_62_0' is not available in your repositories. Cannot reinstall, upgrade, or downgrade.
Resolving package dependencies...
The following 2688 items are locked and will not be changed by any action:
Available:
...
Run 'zypper locks -s' to see the complete list of locked items.
No additional space will be used or freed after the operation.
Nothing to do.
41.98user 0.25system 0:49.05elapsed 86%CPU (0avgtext+0avgdata 183632maxresident)k
0inputs+13120outputs (0major+33101minor)pagefaults 0swap
The output from zypper log shows the operations that take up the most time (setHardLockQueries
and reapplyHardLocks). All other operations finish in at most a second.
18:37:14 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 repo memory used: 492 K incore, 622 K idarray
18:37:14 <1> host(11989) [zypp] Repository.cc(addSolv):336 sat::repo(@System){prio -99.0, size 4531} after adding /var/cache/zypp/solv/@System/solv
18:37:14 <1> host(11989) [zypp] RequestedLocalesFile.cc(load):51 Read /var/lib/zypp/RequestedLocales{- 0644 0/0 size 70}
18:37:14 <1> host(11989) [zypp] SolvIdentFile.cc(load):51 Read /var/lib/zypp/AutoInstalled{- 0644 0/0 size 59810}
18:37:14 <1> host(11989) [zypp] HardLocksFile.cc(load):46 Read /etc/zypp/locks{- 0644 0/0 size 241079}
18:37:14 <1> host(11989) [zypp] PoolImpl.h(setHardLockQueries):246 Apply 2688 HardLockQueries
18:37:34 <1> host(11989) [zypp] PoolImpl.h(setHardLockQueries):254 HardLockQueries match 5387 Solvables.
18:37:34 <1> host(11989) [zypp::satpool] PoolImpl.cc(depSetDirty):261 /etc/sysconfig/storage change
18:37:34 <1> host(11989) [zypp::satpool] PoolImpl.cc(prepare):283 pool_createwhatprovides...
18:37:34 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 found 289 file dependencies, 555 installed file dependencies
18:37:35 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 addfileprovides took 110 ms
18:37:35 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 number of solvables: 155111, memory used: 8482 K
18:37:35 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 number of ids: 447886 + 309037
18:37:35 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 string memory used: 1749 K array + 12949 K data, rel memory used: 3621 K array
18:37:35 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 string hash memory: 4096 K, rel hash memory : 4096 K
18:37:35 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 provide ids: 290359
18:37:35 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 provide space needed: 1237191 + 618074
18:37:35 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 shrunk whatprovidesdata from 1237191 to 461143
18:37:35 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 shrunk whatprovidesauxdata from 1237191 to 841545
18:37:35 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 whatprovides memory used: 2960 K id array, 4215 K data
18:37:35 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 whatprovidesaux memory used: 1749 K id array, 3287 K data
18:37:35 <1> host(11989) [libsolv++] PoolImpl.cc(logSat):119 createwhatprovides took 51 ms
18:37:35 <1> host(11989) [zypp::satpool] PoolImpl.cc(setTextLocale):429 pool_set_languages: (en_US, en)
18:37:35 <1> host(11989) [zypp++] PoolItem.cc(setBuddy):168 I__s_(154961)product:openSUSE-20170320-0.x86_64(@System) has buddy I__s_(153045)openSUSE-release-20170320-1.2.x86_64(@System)
18:37:35 <1> host(11989) [zypp++] PoolItem.cc(setBuddy):168 U__s_(138956)product:openSUSE-Addon-NonOss-2014-0.x86_64(repo-non-oss) has buddy U__s_(138941)openSUSE-Addon-NonOss-release-2014-1.24.x86_64(2017-0
18:37:35 <1> host(11989) [zypp++] PoolItem.cc(setBuddy):168 U__s_(108843)product:openSUSE-20170424-0.x86_64(openSUSE-20151128-0) has buddy U__s_(101728)openSUSE-release-20170424-1.2.x86_64(openSUSE-2012017-0
18:37:35 <1> host(11989) [zypp++] PoolItem.cc(setBuddy):168 U__s_(54042)product:openSUSE-20170424-0.x86_64(http-download.opensuse.org-45284912) has buddy U__s_(47361)openSUSE-release-20170424-1.2.x86_62017-0
18:37:35 <1> host(11989) [zypp] PoolImpl.h(reapplyHardLocks):231 Re-apply 2688 HardLockQueries
18:37:55 <1> host(11989) [zypp] PoolImpl.h(reapplyHardLocks):237 HardLockQueries match 5387 Solvables.
18:37:55 <1> host(11989) [zypp] TargetImpl.cc(updateFileContent):724 updating '/var/lib/zypp/LastDistributionFlavor' content.
18:37:55 <1> host(11989) [zypp] TargetImpl.cc(load):1086 Target loaded: 4531 resolvables
I use zypper locks to disable installation/updatation of texlive-doc* packages. I’m
not sure what’s going on here. I’d appreciate any help.
I see that slowness. But it does not last long for me.
It has to read all existing packages. It is probably building a table in memory. It will take longer with a slower CPU. It will probably take longer if you are short on memory (I’m guessing here). And it is probably faster if you have an SSD (again, guessing).
I’m not sure that there’s much you can do to speed it up.
I’m not sure about the lock queries. But it is possible that zypper set temporary locks as it goes along. So they might not all be related to your locks.
The weird thing is that I have a reasonable configuration – 3.1 GHz Xeon,
16 GB RAM, and the operating system runs off of an SSD.
I had suspected some blocking I/O and so I strace-d the zypper process. There’s
nothing in the strace that suggests a blocked (filesystem/network) I/O request. (Also,
the CPU utilization is high when it’s doing its processing, which is further evidence that
it’s probably not I/O.)
It seems like the bottleneck is the dependency-computing algorithm that runs
completely in memory. Specifically, the two functions, [setHardLockQueries](https://doc.opensuse.org/projects/libzypp/SLE12SP1/pool_2PoolImpl_8h_source.html#l00244)
and [reapplyHardLocks](https://doc.opensuse.org/projects/libzypp/SLE12SP1/pool_2PoolImpl_8h_source.html#l00225), that iterate over some list of repositories/packages (?)
end up taking at least 20 seconds each.
You are right though, I’m not sure if there’s anything that can be done. Perhaps, my specific
configuration (with the set of installed packages, locked packages, and their dependencies)
is exposing some worst-case scenario of the algorithm.
well, while waiting for zypper to “Reading installed packages” i managed to find this threat and write this reply … i would agree to “wait & letting zypper finish” … if my TW would not already reside on SSD …
