System Sluggishness on Boot and During Usage

linuxvinh · April 24, 2016, 4:24am

openSUSE 13.2 (Harlequin) (x86_64)
Linux 3.16.7-35-desktop
KDE 4.14.9

I need help trying to track down general system sluggishness during boot and general usage.

When openSUSE 13.2 was newly installed, January 2015, it ran pretty quick. I’ve been keeping up with system updates as well, so have the latest packages. What I have noticed is that, perhaps starting five months ago(?), the system starting booting much slower and general GUI interactions are delayed/stalled.

During boot: The system would get to the splash screen and then hang for a while. I would hear the hard drive quietly click at regular intervals and then finally finish loading to the full GUI. What things in dmesg should I look for to see what is hanging things up? Any other analysis tools?

During usage: After the full GUI has come up, when I click on Firefox, there is a long delay, like minutes, before it starts up. I’ve noticed that perhaps Apper is completely monopolizing the internet connection? As soon as Apper finishes and issues a notification for updates, Firefox immediately starts up. Is there something hanging the system that is waiting on internet services to be up? At other times, clicking on an icon on the desktop or trying to use the Start Launcher menu will hang as well. How do I go about tracking down what is causing this?

I’ve tried monitoring what runs by using “top” after full GUI startup. I see these executables:

rcu_preempt
haveged
btrfs-transaction
plasma-desktop
…

Thanks for any insights!

gogalthorp · April 24, 2016, 4:50am

Clicking drive may indicate bad sector on the drive. Sectors that have to be reread. run smartctl and see what it says.

linuxvinh · April 24, 2016, 9:58am

It didn’t run clean, so I started doing this…

#smartctl --test=short /dev/sda

smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.16.7-35-desktop] (SUSE RPM)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===
Sending command: “Execute SMART Short self-test routine immediately in off-line mode”.
Drive command “Execute SMART Short self-test routine immediately in off-line mode” successful.
Testing has begun.
Please wait 2 minutes for test to complete.
Test will complete after Sun Apr 24 00:46:05 2016

Use smartctl -X to abort test.

#smartctl -l selftest /dev/sda

smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.16.7-35-desktop] (SUSE RPM)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error

1 Short offline Completed: read failure 90% 7686 1049062336

2 Short offline Completed: read failure 90% 7686 1049062251

3 Short offline Completed: read failure 90% 7686 1049061989

4 Short offline Completed: read failure 90% 7686 1049061915

5 Short offline Completed: read failure 90% 7686 1049061915

6 Short offline Completed: read failure 90% 7686 1049061791

7 Short offline Completed: read failure 90% 7685 1049061779

8 Short offline Completed: read failure 90% 7685 1049061774

9 Short offline Completed: read failure 90% 7685 1049061705

#10 Short offline Completed: read failure 90% 7685 1049061648
#11 Short offline Completed: read failure 90% 7685 1049061620
#12 Short offline Completed: read failure 90% 7685 1049061584
#13 Short offline Completed: read failure 90% 7685 1049061683
#14 Short offline Completed: read failure 90% 7684 1049061678
#15 Short offline Completed: read failure 90% 7684 1049061571
#16 Short offline Completed: read failure 90% 7684 1049061563
#17 Short offline Completed: read failure 90% 7684 1049061558
#18 Short offline Completed: read failure 90% 7684 1049061548
#19 Short offline Completed: read failure 90% 7684 1049061540
#20 Extended offline Completed: read failure 90% 7684 1049061534
#21 Short offline Completed: read failure 90% 7683 1049061534

… calculate seek position 1049062336 / 8 …

#dd if=/dev/zero of=/dev/sda conv=sync bs=4096 count=1 seek=131132781

1+0 records in
1+0 records out
4096 bytes (4.1 kB) copied, 9.5696e-05 s, 42.8 MB/s

#sync

… repeat to find next bad LBA to write zero …

Anyway to speed up this process?

gogalthorp · April 24, 2016, 2:00pm

No way to speed it but what you have shows hardware problem. back your data and get a new drive

linuxvinh · April 24, 2016, 10:04pm

That’s it for the drive? It’s only five years old.

linuxvinh · May 12, 2016, 8:55pm

Following up on this:

I’ve purchased a new drive and will be installing Leap 42.1, to replace 13.2. The old drive has been rsyc’ed so recent data has been backed up. I’m still working through the old drive to fix bad sectors. There were originally 30 or so Current_Pending_Sector and I’ve whittled it down to 7 as of yesterday. (Hopefully done by tonight.)

I found another cause for the system sluggishness on boot…

While watching the boot process, I noticed a message that said it was waiting 1.5 minutes for something. (I’ll need to edit and add the message, if I can find it again.) Searching the message resulting in finding the solution, which was to turn swap off, remake the swap partition and get a new UUID, edit /etc/fstab, and turn swap back on.

swapoff
mkwap /dev/sda5 <— sda5 is /my/ swap partition
<edit /etc/fstab and use the new UUID for the swap partition line>
swapon -a

gogalthorp · May 12, 2016, 9:46pm

On modern drives That is also losing battle bad sectors are replaced form a pool of spares and when the pool is empty they start showing up.This is the main function of smart. In any case there is some hardware problem causing the bad sectors and that is simply not fixable. Don’t put anything important on that drive

note that systemd-analyze blame will show what is taking time

linuxvinh · May 14, 2016, 6:08am

Finally “Completed without error” and only 1 RAW VALUE of Current_Pending_Sector:

smartctl -a /dev/sda
…
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 1
…

1 Short offline Completed without error 00% 7776 -

…

The system feels much faster now, perhaps to what it was before.

systemd-analyze blame
5.620s systemd-udev-settle.service
3.481s apparmor.service
3.028s SuSEfirewall2_init.service
2.626s ModemManager.service
2.062s tpdaemon.service
1.865s display-manager.service
1.761s postfix.service
1.556s systemd-fsck@dev-disk-by\x2duuid-e2232a71\x2d491b\x2d4960\x2d8e
1.274s alsa-restore.service
1.257s systemd-user-sessions.service
1.256s nscd.service
1.256s avahi-daemon.service
1.254s wpa_supplicant.service
1.254s razerd.service
1.253s rc-local.service
1.129s polkit.service
1.019s SuSEfirewall2.service
804ms dev-disk-by\x2duuid-94717e0c\x2d32f0\x2d4a69\x2da47f\x2da720562
735ms NetworkManager.service
658ms systemd-random-seed.service
615ms sys-kernel-debug.mount
614ms dev-mqueue.mount
614ms dev-hugepages.mount
593ms systemd-tmpfiles-setup-dev.service
570ms \x2esnapshots.mount
555ms home.mount
462ms systemd-udev-trigger.service
460ms udisks2.service
455ms systemd-journald.service
451ms systemd-remount-fs.service
369ms systemd-udev-root-symlink.service
366ms lvm2-lvmetad.service
366ms dm-event.service
322ms var-tmp.mount
289ms var-spool.mount
277ms var-opt.mount
265ms var-lib-pgsql.mount
264ms var-lib-named.mount
259ms auditd.service
253ms var-lib-mailman.mount
234ms rtkit-daemon.service
230ms var-crash.mount
229ms usr-local.mount
213ms lvm2-activation-early.service
188ms systemd-tmpfiles-setup.service
187ms ntpd.service
182ms tmp.mount
171ms systemd-modules-load.service
162ms cycle.service
159ms systemd-logind.service
140ms systemd-udevd.service
135ms systemd-sysctl.service
131ms srv.mount
131ms plymouth-read-write.service
120ms systemd-readahead-replay.service
107ms opt.mount
100ms boot-grub2-x86_64\x2defi.mount
83ms bluetooth.service
83ms systemd-readahead-done.service
80ms systemd-update-utmp.service
74ms user@1000.service
67ms upower.service
60ms user@4.service
57ms systemd-vconsole-setup.service
46ms boot-grub2-i386\x2dpc.mount
36ms iscsi.service
31ms var-log.mount
14ms lvm2-activation.service
13ms plymouth-start.service
11ms systemd-readahead-collect.service
4ms systemd-journal-flush.service
3ms systemd-update-utmp-runlevel.service
3ms sys-fs-fuse-connections.mount
3ms kmod-static-nodes.service

I’m not sure how this compares with other systems, but I’ll have a basis to compare against with 42.1 and the new drive.

gogalthorp · May 14, 2016, 1:21pm

Bad or marginal sectors have to be reread multiple time this introduces large delays

linuxvinh · May 15, 2016, 11:53am

Leap 42.1 installed on new drive with old drive serving as alternative boot, which is handy to reference for checking settings.

Currently, new system boots slowly due to wicked.service:

systemd-analyze blame

    19.937s wicked.service
      1.887s dev-sdb1.device
      1.659s ModemManager.service
      1.545s SuSEfirewall2_init.service
      1.484s tpdaemon.service
       838ms home.mount
       799ms display-manager.service
       735ms boot-grub2-i386\x2dpc.mount
       704ms boot-grub2-x86_64\x2defi.mount
       607ms polkit.service
       586ms var-log.mount
       536ms var-lib-pgsql.mount
       535ms var-lib-mariadb.mount
       534ms var-lib-mailman.mount
       533ms var-crash.mount
       531ms opt.mount
       527ms var-lib-named.mount
       527ms var-lib-mysql.mount
       525ms var-lib-libvirt-images.mount
       524ms wpa_supplicant.service
       515ms srv.mount
       504ms var-opt.mount
       504ms systemd-journald.service
       453ms avahi-daemon.service
       440ms var-spool.mount
       436ms usr-local.mount
       435ms tmp.mount
       350ms systemd-fsck@dev-disk-by\x2duuid-f9dbd663\x2d46fd\x2d4934\x2dab
       326ms systemd-udev-trigger.service
       315ms postfix.service
       282ms apparmor.service
       279ms systemd-modules-load.service
       267ms systemd-readahead-replay.service
       267ms systemd-readahead-collect.service
       266ms dev-disk-by\x2duuid-bb388ce9\x2d96cb\x2d4f08\x2d9349\x2d1b858f0
       266ms systemd-udev-root-symlink.service
       264ms sys-kernel-debug.mount
       261ms auditd.service
       247ms systemd-tmpfiles-setup.service
       216ms nscd.service
       202ms kmod-static-nodes.service
       191ms SuSEfirewall2.service
       190ms ntpd.service
       181ms \x2esnapshots.mount
       180ms var-tmp.mount
       153ms systemd-tmpfiles-setup-dev.service
       151ms systemd-user-sessions.service
       125ms [email]user@1000.servic[/email]e
       117ms systemd-random-seed.service
        88ms systemd-remount-fs.service
        62ms plymouth-read-write.service
        52ms systemd-vconsole-setup.service
        51ms systemd-logind.service
        50ms systemd-tmpfiles-clean.service
        33ms udisks2.service
        30ms rtkit-daemon.service
        21ms [email]user@4.servic[/email]e
        19ms plymouth-start.service
        18ms [email]user@484.servic[/email]e
        11ms systemd-update-utmp.service
         8ms systemd-sysctl.service
         6ms systemd-udevd.service
         6ms upower.service
         6ms systemd-journal-flush.service
         4ms alsa-restore.service
         3ms wickedd-auto4.service
         3ms wickedd-dhcp6.service
         3ms iscsi.service
         3ms wickedd-nanny.service
         3ms systemd-update-utmp-runlevel.service
         3ms sys-fs-fuse-connections.mount
         3ms rc-local.service
         3ms wickedd.service
         3ms wickedd-dhcp4.service
         2ms bluetooth.service
         2ms systemd-readahead-done.service
         1ms dev-mqueue.mount
         1ms dev-hugepages.mount

Skype mic audio from Testing Service sounds sped up, like I’m a chipmunk! Basically, the mic works, but it either is recorded weird or the Testing Service receives it weird.

TrueCrypt no longer works. Interface doesn’t pop up.

linuxvinh · May 15, 2016, 1:13pm

I switched to Network Manager…

YaST->Global Options/Network Setup Method/NetworkManager Service (changed from Wicked Service)

Much faster boot…

systemd-analyze blame

      2.452s dev-sdb1.device
      1.741s display-manager.service
      1.177s SuSEfirewall2_init.service
      1.173s postfix.service
      1.138s ModemManager.service
      1.019s tpdaemon.service
       974ms systemd-udev-trigger.service
       877ms systemd-readahead-replay.service
       868ms NetworkManager.service
       712ms ntpd.service
       661ms home.mount
       655ms boot-grub2-i386\x2dpc.mount
       631ms boot-grub2-x86_64\x2defi.mount
       626ms var-lib-libvirt-images.mount
       614ms var-log.mount
       612ms var-lib-named.mount
       611ms var-lib-mysql.mount
       610ms var-lib-mailman.mount
       602ms var-lib-mariadb.mount
       599ms opt.mount
       595ms var-crash.mount
       587ms var-lib-pgsql.mount
       517ms systemd-journald.service
       463ms srv.mount
       453ms rtkit-daemon.service
       452ms var-opt.mount
       441ms sys-kernel-debug.mount
       435ms dev-mqueue.mount
       434ms dev-hugepages.mount
       420ms var-spool.mount
       408ms usr-local.mount
       367ms tmp.mount
       366ms SuSEfirewall2.service
       364ms var-tmp.mount
       349ms polkit.service
       332ms apparmor.service
       322ms auditd.service
       303ms systemd-tmpfiles-setup-dev.service
       247ms wpa_supplicant.service
       235ms systemd-tmpfiles-setup.service
       208ms nscd.service
       195ms avahi-daemon.service
       174ms systemd-remount-fs.service
       172ms dev-disk-by\x2duuid-bb388ce9\x2d96cb\x2d4f08\x2d9349\x2d1b858f0
       126ms systemd-fsck@dev-disk-by\x2duuid-f9dbd663\x2d46fd\x2d4934\x2dab
       112ms [email]user@1000.servic[/email]e
       103ms systemd-udev-root-symlink.service
       100ms bluetooth.service
        97ms \x2esnapshots.mount
        84ms kmod-static-nodes.service
        80ms plymouth-read-write.service
        53ms [email]user@4.servic[/email]e
        53ms systemd-vconsole-setup.service
        48ms systemd-udevd.service
        40ms systemd-readahead-collect.service
        40ms udisks2.service
        22ms systemd-journal-flush.service
        20ms [email]user@484.servic[/email]e
        13ms plymouth-start.service
         6ms systemd-update-utmp.service
         5ms systemd-user-sessions.service
         5ms upower.service
         5ms systemd-logind.service
         4ms alsa-restore.service
         4ms rc-local.service
         4ms systemd-sysctl.service
         3ms systemd-update-utmp-runlevel.service
         3ms sys-fs-fuse-connections.mount
         2ms iscsi.service
         2ms systemd-readahead-done.service
         1ms systemd-modules-load.service
         1ms systemd-random-seed.service

As for Skype chipmunk playback… “fixed” after reboot…

linuxvinh · May 16, 2016, 7:30am

Truecrypt was fixed by installing 32-bit versions of libraries it tries to load when run from the commandline.