Patch openSUSE-2021-60 - boot time improvement - sometimes

English Install/Boot/Login
1

Since the patch openSUSE-2021-60, on my desktop system I’ve noticed an improvement in the boot-time – network-pre target is reached within 3.3 seconds. Sadly, my laptop is not enjoying this improvement – var.cache mount chimes in at about 11.2 seconds.

Desktop: ext4 system partition on a SSD – AMD Ryzen 5 3400G CPU.
Laptop: Btrfs system partition on a SSHD – AMD A10-5750M APU.

Desktop: “Startup finished in 3.677s (kernel) + 923ms (initrd) + 25.598s (userspace) = 30.199s” – top of “blame” for today’s boot was “backup-rpmdb.service” – 11 seconds – therefore not an absolutely fair comparison …

Laptop: “Startup finished in 6.771s (kernel) + 9.511s (initrd) + 40.363s (userspace) = 56.646s” – top of “blame” is “firewalld.service” – 15 seconds …

  • Before the patch, the Laptop’s boot time was: “Startup finished in 5.129s (kernel) + 7.670s (initrd) + 23.657s (userspace) = 36.457s”
  • I didn’t take a look at the Desktop’s boot time before the patch – I was surprised by the improvement …

[HR][/HR]The only real difference that I can see is the file system of the system partitions – yes, I ran Btrfs Balance and Scrub before I rebooted with the patch changes on the Laptop and, then rebooted again to get rid of the “purge kernels” time …
[HR][/HR]Since the patch appeared yesterday, is anyone else seeing comparable results?

2

SSHD???

Is your network pre-target on WiFi, and your desktop on RG-6?

This is a desktop with no wireless hardware, with persistent journal, and fstrim run right before booting the .60 kernel:

# inxi -CDGISy
System:
  Host: asa88 Kernel: 5.3.18-lp152.60-default x86_64 bits: 64 Console: tty 3
  Distro: openSUSE Leap 15.2
CPU:
  Info: Quad Core model: **AMD A10-7850K** Radeon R7 12 Compute Cores 4C+8G
  bits: 64 type: MCP L2 cache: 2 MiB
  Speed: 1700 MHz min/max: 1700/3700 MHz Core speeds (MHz): 1: 1700 2: 1693
  3: 1697 4: 1683
Graphics:
  Device-1: AMD Kaveri [Radeon R7 Graphics] driver: amdgpu v: kernel
  Display: server: X.org 1.20.3 driver: amdgpu
  unloaded: fbdev,modesetting,vesa tty: 160x45
  Message: Advanced graphics data unavailable in console for root.
Drives:
  Local Storage: total: 476.94 GiB used: 35.47 GiB (7.4%)
  ID-1: /dev/sda model: T-FORCE 512GB size: 476.94 GiB
Info:...Shell: Bash **inxi: 3.2.02**

# mount | grep " / "
/dev/sda11 on / type ext4 (rw,noatime)

# systemd-analyze blame

         10.332s sshd.service
          2.061s kbdsettings.service
          1.704s systemd-udev-settle.service
          1.031s wicked.service
           765ms systemd-journal-flush.service
           374ms dracut-initqueue.service
           328ms initrd-switch-root.service
           231ms systemd-fsck@dev-disk-by\x2dlabel-tvgp06pub.service
           230ms systemd-fsck@dev-disk-by\x2dlabel-tvgp04usrlcl.service
           214ms systemd-fsck@dev-disk-by\x2dlabel-tvgp05home.service
           117ms initrd-parse-etc.service
           115ms disks-stw.mount
           108ms systemd-udevd.service
           104ms smartd.service

# systemd-analyze critical-chain

The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.

└─multi-user.target @13.548s
  └─sshd.service @3.215s +10.332s
    └─network.target @3.210s
      └─wicked.service @2.174s +1.031s
        └─wickedd-nanny.service @2.144s +26ms
          └─wickedd.service @2.116s +25ms
            └─wickedd-dhcp4.service @2.049s +60ms
              └─dbus.service @2.010s
                └─basic.target @2.007s
                  └─sockets.target @2.007s
                    └─dbus.socket @2.006s
                      └─sysinit.target @2.006s
                        └─systemd-udev-settle.service @301ms +1.704s
                          └─systemd-udev-trigger.service @238ms +61ms
                            └─systemd-udevd-control.socket @221ms
                              └─-.mount
                                └─system.slice
                                  └─-.slice
3

It’s a Seagate Firecuda – Solid State Hybrid Drive – there’s a cache which behaves like a small SSD in front of the HDD – within the drive …

  • It ain’t the systemd “sshd” service – the OpenSSH Daemon …

On this Desktop, the OpenSSH Daemon needs 78ms to initialise …


 # systemd-analyze blame | grep -i 'hd'
            78ms sshd.service
 #

I’ll be checking the Laptop later but yes, it’s Network Manager and usually WiFi – I’ve noticed a couple of other things which need to be done …

  • If you mean by RG-6 that the Desktop is using co-axial cable Ethernet – no, it’s using a “normal” Cat 6 cable …

I use IPv6 which may explain why Wicked is currently taking 8.384s to initialise …


 # systemd-analyze blame | grep -i 'wick'
          8.384s wicked.service
            40ms wickedd-auto4.service
            38ms wickedd-dhcp6.service
            37ms wickedd-dhcp4.service
            21ms wickedd-nanny.service
            20ms wickedd.service
 #

Looking at the systemd critical-chain, today, the network-pre target popped up at 2.980s …
[HR][/HR]BTW, I started this thread because of karlmistelberger’s report of the time needed to boot with the same AMD Ryzen GPU as the one I’m using but, running Tumbleweed – time to the Graphical target: 1.688s.
<https://forums.opensuse.org/showthread.php/548575-15-2-login-VERY-slow?p=2996011#post2996011&gt; – he’s not using Wicked or anything else which consumes time …

4

The issue with the Laptop seems to be –

  • A connector issue …

I removed the Battery, opened the case, removed the Disk and the Memory modules, plugged everything back in again, closed the case and reinserted the Battery.

The systemd Analysis is now reporting:
“Startup finished in 3.901s (kernel) + 7.849s (initrd) + 15.192s (userspace) = 26.944s”

  • Top of “blame” is: “dracut-initqueue.service” – 5.993s
  • In the “critical chain”, “var-cache.mount” kicks in at 2.664s – “basic.target” at 8.672s

 # systemd-analyze critical-chain 
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.

graphical.target @15.120s
└─display-manager.service @14.175s +943ms
  └─systemd-logind.service @13.691s +481ms
    └─nss-user-lookup.target @10.642s
      └─nscd.service @8.830s +1.811s
        └─basic.target @8.672s
          └─sockets.target @8.672s
            └─cups.socket @8.672s
              └─sysinit.target @8.460s
                └─apparmor.service @3.042s +5.417s
                  └─var-cache.mount @2.664s +375ms
                    └─dev-disk-by\x2duuid-fa6a0367\x2da191\x2d447e\x2d8298\x2d35761792b861.device @2.638s
 #
5

Some cheap hardware using default configuration, but etx4: https://forums.opensuse.org/showthread.php/533772-Running-Tumbleweed-on-HP-Laptop-15-da0xxx

Startup finished in 6.434s (firmware) + 3.071s (loader) + 4.032s (kernel) + 1.938s (initrd) + 4.042s (userspace) = 19.519s graphical.target reached after 4.019s in userspace

Note: network setup (network.target) is way faster than link configuration (network-online.target). By default wicked will wait for the links to be configured.

3400G:~ # journalctl -b -u systemd-networkd.service -o short-monotonic 
-- Logs begin at Sat 2020-12-12 12:09:37 CET, end at Tue 2021-01-19 12:53:05 CET. --
    5.485684] 3400G systemd[1]: Starting Network Service...
    5.485750] 3400G systemd-networkd[528]: Enumeration completed
**    5.506305] 3400G systemd[1]: Started Network Service.**
    6.193659] 3400G systemd-networkd[528]: wlan0: found matching network '/etc/systemd/network/wlan.network', based on potentially unpredictable ifname
    6.193768] 3400G systemd-networkd[528]: wlan0: IPv6 successfully enabled
    6.211737] 3400G systemd-networkd[528]: wlan0: Link UP
    7.303060] 3400G systemd-networkd[528]: wlan0: Gained carrier
    7.303307] 3400G systemd-networkd[528]: wlan0: Connected WiFi access point: FRITZ!Box Karl Mistelberger (08:96:d7:e2:d6:d3)
    7.303353] 3400G systemd-networkd[528]: wlan0: found matching network '/etc/systemd/network/wlan.network', based on potentially unpredictable ifname
**    8.336166] 3400G systemd-networkd[528]: wlan0: DHCPv4 address 192.168.178.26/24 via 192.168.178.1**
    8.512695] 3400G systemd-networkd[528]: wlan0: Gained IPv6LL
3400G:~ #

What really makes a difference is time to display-manager:

3400G:~ # systemd-analyze critical-chain display-manager.service
The time when unit became active or started is printed after the "@" character.
The time the unit took to start is printed after the "+" character.

display-manager.service +587ms
└─apache2.service @997ms +99ms
  └─time-sync.target @978ms
    └─chronyd.service @948ms +29ms
      └─nss-lookup.target @944ms
        └─systemd-resolved.service @768ms +172ms
          └─systemd-tmpfiles-setup.service @743ms +23ms
            └─systemd-journal-flush.service @351ms +389ms
              └─var.mount @326ms +17ms
                └─local-fs-pre.target @312ms
                  └─systemd-tmpfiles-setup-dev.service @301ms +10ms
                    └─kmod-static-nodes.service @252ms +31ms
                      └─systemd-journald.socket
                        └─system.slice
                          └─-.slice
3400G:~ #

I got the impression your disks are pretty lame. Showing more detail may help:

3400G:~ # journalctl -b -u init.scope -o short-monotonic|grep Reached
    3.271852] 3400G systemd[1]: Reached target System Initialization.
    3.273440] 3400G systemd[1]: Reached target Basic System.
    3.648036] 3400G systemd[1]: Reached target Initrd Root Device.
    3.960180] 3400G systemd[1]: Reached target Remote File Systems (Pre).
    3.960938] 3400G systemd[1]: Reached target Remote File Systems.
    4.296294] 3400G systemd[1]: Reached target Initrd Root File System.
    4.539823] 3400G systemd[1]: Reached target Initrd File Systems.
    4.540457] 3400G systemd[1]: Reached target Initrd Default Target.
    4.608033] 3400G systemd[1]: Reached target Switch Root.
    5.059468] 3400G systemd[1]: Reached target Local File Systems (Pre).
    5.091951] 3400G systemd[1]: Reached target Local File Systems.
    5.523597] 3400G systemd[1]: Reached target System Initialization.
    5.529405] 3400G systemd[1]: Reached target Paths.
    5.539502] 3400G systemd[1]: Reached target Sockets.
    5.540417] 3400G systemd[1]: Reached target Basic System.
    5.689679] 3400G systemd[1]: Reached target Network.
    5.690359] 3400G systemd[1]: Reached target Network is Online.
    5.691203] 3400G systemd[1]: Reached target Host and Network Name Lookups.
    5.725090] 3400G systemd[1]: Reached target System Time Synchronized.
    5.742389] 3400G systemd[1]: Reached target Timers.
    5.847295] 3400G systemd[1]: Reached target Login Prompts.
    6.197529] 3400G systemd[1]: Reached target Multi-User System.
    6.384900] 3400G systemd[1]: Reached target Sound Card.
    6.433007] 3400G systemd[1]: Reached target Graphical Interface.
3400G:~ #
6

Which sort of proves a suspicion that, I’ve had for the past few years – an ext4 system partition on a (cheap) Laptop boots faster than a Btrfs system partition on the same hardware.

7

I told you, there was a difference, but now I tell you that I am not aware that systems using btrfs are booting slower than other systems. I used ext4 and btrfs system partitions in parallel, but found no noticeable difference.

You present only fragments of your boot process and virtually no details of the hardware. Thus it’s not possible to comment on the difference in boot times observed. However your basic.target @8.672s (ext4 on Firecuda) vs. my basic.target @497ms (btrfs on NVME SSD) tells something about I/O performance.

8

Or, the I/O performance difference between a drive connected via a SATA cable and, a drive plugged into a PCI Express slot on the Mainboard.

  • BTW, bottom line on my Laptop issue – it pays to occasionally (possibly in biennial cycles – or, at least when the system’s boot time increases dramtically) – to “exercise” the drive and memory connectors …
9

Off topic a little bit, look into replacing your Firecuda ASAP for the following reasons:

  • 9/9 of 2.5" 2TB Firecudas I’ve had failed (7/9 failed within the first month, 2/9 wore out in less than 2 years all from different batches, some bought in Canada, some in Switzerland) and were replaced by Seagate warranty service with Barracuda.
  • Firecudas are no longer made for the exact reason above.
  • When the Cache is worn down, which happened rather quickly for me working with large data analysis, the performance does not drop into speed of a plain HDD (100MB/s). The performance drops to below 20MB/s due to error corrections happening in the cache component.

Replace this thing as soon as you can afford to.

10

As much as I do have my distaste for BTRFS, I disagree based on an experiment. I have the following setup:
SSD boot drive on PCIe + HDD data drive on SATA. I have two identical setups, one with ext4 root and ext4 home and the other with BTRFS boot and ext4 home and BTRFS typically boots faster by up to 5 seconds. All on the same laptop.

I’ve had the OP’s SSHD on several machines and it’s a single least reliable drive I’ve ever owned. I would consider any inconsistencies in such things like boot time as a sign that the cache is about to call it quits.

11

Being a “prefer to buy from local handlers (shops)” fan, I’ll have to wait until sometime next month – when the shops are allowed to open again …

  • I may well replace it by another Laptop anyway – it’s a 6 year old AMD dual-graphics APU machine with a very flaky battery …
12

Some operations are expensive with btrfs, but booting is mostly reading data, which is inexpensive.

Whenever feasible M.2 NVMe x4 or SATA III SSDs should be used for / and /home. Movies and pictures, which are written once and read multiple times may be stored on HDD:

**Drives:    Local Storage:****total:** 6.38 TiB **used:** 3.10 TiB (48.6%)  
           **ID-1:** /dev/nvme0n1 **vendor:** Samsung **model:** SSD 950 PRO 512GB **size:** 476.94 GiB  # system and /home, 3 GB/s
           **ID-2:** /dev/sda **vendor:** Western Digital **model:** WD40EZRX-22SPEB0 **size:** 3.64 TiB # backup, 150 MB/s
           **ID-3:** /dev/sdb **vendor:** Crucial **model:** CT2000BX500SSD1 **size:** 1.82 TiB  # media, 500 MB/s
           **ID-4:** /dev/sdc **vendor:** Samsung **model:** SSD 850 EVO 500GB **size:** 465.76 GiB # more systems, 500 MS/s

Large RAM will work as a high speed disk cache (20+ GB/s).

13

Yet another off topic… How/what would one mount on RAM to use it as cache? I have 32GB RAM and until I get my analysis going, I typically use less than 12.

14

Usage is fully automatic:

1687×906 188 KB

Older installations have /tmp on disk. Remove it from /etc/fstab:

**erlangen:~ #** df -h /tmp/ 
Filesystem      Size  Used Avail Use% Mounted on 
tmpfs            16G  408K   16G   1% /tmp 
**erlangen:~ #**

/tmp will occupy only space that is needed up to a maximum of 16G.

15

Beware: I/O is a bottleneck in many laptops

Samsung model: SSD 950 PRO 512GB size: 476.94 GiB

**erlangen:~ #** iops --num_threads 36 --time 2 /dev/nvme0n1                        
/dev/nvme0n1, 512.11 G, sectorsize=512B, #threads=32, pattern=random: 
 512  B blocks: 131576.3 IO/s,  67.4 MB/s (538.9 Mbit/s) 
   1 kB blocks: 124437.9 IO/s, 127.4 MB/s (  1.0 Gbit/s) 
   2 kB blocks: 118254.1 IO/s, 242.2 MB/s (  1.9 Gbit/s) 
   4 kB blocks: 104366.6 IO/s, 427.5 MB/s (  3.4 Gbit/s) 
   8 kB blocks: 69578.4 IO/s, 570.0 MB/s (  4.6 Gbit/s) 
  16 kB blocks: 39327.3 IO/s, 644.3 MB/s (  5.2 Gbit/s) 
  32 kB blocks: 38254.0 IO/s,   1.3 GB/s ( 10.0 Gbit/s) 
  65 kB blocks: 19590.8 IO/s,   1.3 GB/s ( 10.3 Gbit/s) 
 131 kB blocks: 9635.7 IO/s,   1.3 GB/s ( 10.1 Gbit/s) 
 262 kB blocks: 4872.7 IO/s,   1.3 GB/s ( 10.2 Gbit/s) 
 524 kB blocks: 2555.9 IO/s,   1.3 GB/s ( 10.7 Gbit/s) 
   1 MB blocks: 1576.1 IO/s,   1.7 GB/s ( 13.2 Gbit/s) 
   2 MB blocks:  961.9 IO/s,   2.0 GB/s ( 16.1 Gbit/s) 
   4 MB blocks:  536.4 IO/s,   2.2 GB/s ( 18.0 Gbit/s) 
   8 MB blocks:  286.2 IO/s,   2.4 GB/s ( 19.2 Gbit/s) 
  16 MB blocks:  146.4 IO/s,   2.5 GB/s ( 19.7 Gbit/s) 
  33 MB blocks:   76.8 IO/s,   2.6 GB/s ( 20.6 Gbit/s) 
  67 MB blocks:   39.8 IO/s,   2.7 GB/s ( 21.3 Gbit/s) 
 134 MB blocks:   20.6 IO/s,   2.8 GB/s ( 22.1 Gbit/s) 
 268 MB blocks:   10.1 IO/s,   2.7 GB/s ( 21.7 Gbit/s) 
 536 MB blocks:    4.5 IO/s,   2.4 GB/s ( 19.2 Gbit/s) 
**erlangen:~ #**

Western Digital model: WD40EZRX-22SPEB0 size: 3.64 TiB

[FONT=monospace]**erlangen:~ #** iops --num_threads 36 --time 2 /dev/sda 
/dev/sda,   4.00 T, sectorsize=4096B, #threads=32, pattern=random: 
 512  B blocks:   66.0 IO/s,  33.8 kB/s (270.4 kbit/s) 
   1 kB blocks:   65.3 IO/s,  66.9 kB/s (535.1 kbit/s) 
   2 kB blocks:   60.0 IO/s, 122.9 kB/s (983.1 kbit/s) 
   4 kB blocks:   66.3 IO/s, 271.5 kB/s (  2.2 Mbit/s) 
   8 kB blocks:   63.4 IO/s, 519.8 kB/s (  4.2 Mbit/s) 
  16 kB blocks:   63.9 IO/s,   1.0 MB/s (  8.4 Mbit/s) 
  32 kB blocks:   59.7 IO/s,   2.0 MB/s ( 15.6 Mbit/s) 
  65 kB blocks:   60.6 IO/s,   4.0 MB/s ( 31.8 Mbit/s) 
 131 kB blocks:   61.1 IO/s,   8.0 MB/s ( 64.1 Mbit/s) 
 262 kB blocks:   55.4 IO/s,  14.5 MB/s (116.1 Mbit/s) 
 524 kB blocks:   60.2 IO/s,  31.6 MB/s (252.6 Mbit/s) 
   1 MB blocks:   42.6 IO/s,  44.7 MB/s (357.4 Mbit/s) 
   2 MB blocks:   21.7 IO/s,  45.5 MB/s (364.0 Mbit/s) 
   4 MB blocks:   15.0 IO/s,  63.0 MB/s (503.7 Mbit/s) 
   8 MB blocks:    9.6 IO/s,  80.9 MB/s (647.4 Mbit/s) 
  16 MB blocks:    6.7 IO/s, 112.4 MB/s (899.3 Mbit/s) 
**erlangen:~ #**
[/FONT]