"2321" packages to upgrade . . . on reboot another "204" yet to go?? Why???

[karlmistelberger]

Using another OS, today was "Sid" day . . . the Sid guys on the forum take the same, "You don't know what you are doing" attitude on users of their fine offering when finding issues. BTW I run updates in all of my installs every week, today Sid had "103 packages" it dl'd and installed them in 3 or 4 minutes. Sid does everything fast. If I compare that time to the "505" packages that I upgraded in TW yesterday in half an hour . . . the Sid time beats TW by ten minutes . . . same "old" machine.

Nothing to compare in the above.

Provide some basic information:

• Upgrade: Tumbleweed 20220522-0 -> 20220523-0
• Download command: zypper dist-upgrade
• Number of packages: 36
• Size of download: 119,2 MiB
• CPU time of download: 5.536s
• CPU time of install: 10.984s

[non_space]

As spelled out in post #58, dup and dupa are local services. Local in a config file name always means the admin is entirely responsible for their content; hence, neither rpm nor zypper nor yast would know anything about them, unless possibly created from a packaged skeleton file.

@mrmazda:

OK, I saw the "local services" but just assumed that meant it or they were installed by user . . . . Still haven't seen the "how-to" to get those services. I'd like to be able to have my 4 TW variations maintained more efficiently.

Oh debian sid again? I suppose it's gcc 12 / glibc 2.35 over there already. That is fast!

@awrlang:

Yes . . . it's "Groundhog Day" all over again. As then in Feb, it's not per se the large number of packages, but the breakages that seem to increase commensurate with the huge number of packages being processed. In this recent 2321 something caused the machine to shut down while dl-ing, that makes it "difficult" to run these upgrades while twiddling my thumbs elsewhere, etc. "Supervision" seems to be required . . . also for answering dependency issue questions before zypper will run them through . . . .

Today is Gecko TW day, and zypper showed "252" packages compared to mainline TW's "525" from Monday. I assumed that zypper would crunch these packages quickly and efficiently . . . however total clock time to run them through was 47 mins . . . ???? Zypper bogged heavily time wise when it hit the kernel install, the virtualbox and the broadcom-wl packages . . . hanging for minutes on each of those packages . . . followed by thousands of "dracut" lines . . . . Essentially this is a "TW" system, but time used exceeded the "30 minutes" to install the 525 for TW.

That leads me to think that there is a "zypper" crisis??? Not looking at the numbers but Sid ran a similar number of packages, including kernel upgrade in a comparative "flash" . . . 3 or 4 minutes. A Sid is "pre-testing" . . . aka "unstable" . . . albeit, quick.

Nothing to compare in the above.

Provide some basic information:

• Upgrade: Tumbleweed 20220522-0 -> 20220523-0
• Download command: zypper dist-upgrade
• Number of packages: 36
• Size of download: 119,2 MiB
• CPU time of download: 5.536s
• CPU time of install: 10.984s

@karlmisteberger:

OK, I'm assuming that these datas would be supplied by "dup.service"?? because zypper isn't showing me those numbers.

[mrmazda]

Still haven't seen the "how-to" to get those services. I'd like to be able to have my 4 TW variations maintained more efficiently.

As an "enthusiast", it behooves you to learn what systemd services and timers are, and how to create and maintain custom ones, as karlmistelberger has done.

PS: try prefixing commands like zypper dup with the time command.

[malcolmlewis]

Hi
My only downtime is manually building the Nvidia driver....

I don't use virtualbox and derivatives, kvm/qemu/vagrant works OTB for me.

No crisis, your locale may be the issue, using third party products and repositories, I only use the oss and update repo. For other rpms I have a local repo and enable the google-chrome one when needed.

[karlmistelberger]

Today is Gecko TW day, and zypper showed "252" packages compared to mainline TW's "525" from Monday. I assumed that zypper would crunch these packages quickly and efficiently . . . however total clock time to run them through was 47 mins . . . ???? Zypper bogged heavily time wise when it hit the kernel install, the virtualbox and the broadcom-wl packages . . . hanging for minutes on each of those packages . . . followed by thousands of "dracut" lines . . . . Essentially this is a "TW" system, but time used exceeded the "30 minutes" to install the 525 for TW.

That leads me to think that there is a "zypper" crisis??? Not looking at the numbers but Sid ran a similar number of packages, including kernel upgrade in a comparative "flash" . . . 3 or 4 minutes. A Sid is "pre-testing" . . . aka "unstable" . . . albeit, quick. ... OK, I'm assuming that these datas would be supplied by "dup.service"?? because zypper isn't showing me those numbers.

Yep. Basically you may define a service by creating file /etc/systemd/system/dup.service and invoke the service by running "sytemctl start dup.service".

# /etc/systemd/system/dup.service
[Unit] 
Description=Dist Upgrade 

[Service] 
ExecStartPre=/usr/bin/sleep 11 
ExecStart=/usr/bin/zypper dist-upgrade --no-confirm --download-only 
erlangen:~ #

Get the log by running:

erlangen:~ # journalctl -b -u dup.service -g download 
May 26 10:45:41 erlangen zypper[26651]: Gesamtgröße des Downloads: 59,2 MiB. Bereits im Cache gespeichert: 0 B. Nur herunterladen. 
erlangen:~ # journalctl -b -u dup.service -g Pakete   
May 26 00:00:16 erlangen zypper[17875]: Installierte Pakete werden gelesen... 
May 26 10:45:41 erlangen zypper[26651]: Installierte Pakete werden gelesen... 
May 26 10:45:41 erlangen zypper[26651]: Die folgenden 67 Pakete werden aktualisiert: 
May 26 10:45:41 erlangen zypper[26651]: Die folgenden 2 NEUEN Pakete werden installiert: 
May 26 10:45:41 erlangen zypper[26651]: 67 Pakete werden aktualisiert, 2 neue, 1 zu entfernen. 
erlangen:~ # journalctl -b -u dup.service -g Consumed 
May 26 10:46:10 erlangen systemd[1]: dup.service: Consumed 5.943s CPU time. 
erlangen:~ #

The above command runs in the background with low priority. It will continue regardless what happens to the graphical login. Users running their GUI won't even notice this activity. Everything is logged to journal for later reference and forensics if needed.

There is no such thing as a zypper crisis. There are only users failing to deal appropriately with "zypper dist-upgrade". Today's dist-upgrade on host erlangen: 20220523-0 -> 20220524-0: 67 packages, 59,2 MiB; download 5.943s CPU time, log size 172 lines, 19906 bytes total size, install with comprehensive installation log consumed 9.663s CPU time, log size 592 lines, 56366 bytes total size.

You may test your connection by running all of the below commands. For ease of discussion don't make any changes. Include all output, also the command prompts.

erlangen:~ # zypper clean --all  
All repositories have been cleaned up. 
erlangen:~ # time zypper refresh 
Retrieving repository 'Packman' metadata ...............................................................................................................................................................................................[done] 
Building repository 'Packman' cache ....................................................................................................................................................................................................[done] 
Retrieving repository 'chrome' metadata ................................................................................................................................................................................................[done] 
Building repository 'chrome' cache .....................................................................................................................................................................................................[done] 
Retrieving repository 'qmapshack (openSUSE_Tumbleweed)' metadata .......................................................................................................................................................................[done] 
Building repository 'qmapshack (openSUSE_Tumbleweed)' cache ............................................................................................................................................................................[done] 
Retrieving repository 'jalbum' metadata ................................................................................................................................................................................................[done] 
Building repository 'jalbum' cache .....................................................................................................................................................................................................[done] 
Retrieving repository 'myrepo' metadata ................................................................................................................................................................................................[done] 
Building repository 'myrepo' cache .....................................................................................................................................................................................................[done] 
Retrieving repository 'Haupt-Repository (NON-OSS)' metadata ............................................................................................................................................................................[done] 
Building repository 'Haupt-Repository (NON-OSS)' cache .................................................................................................................................................................................[done] 
Retrieving repository 'Haupt-Repository (OSS)' metadata ................................................................................................................................................................................[done] 
Building repository 'Haupt-Repository (OSS)' cache .....................................................................................................................................................................................[done] 
Retrieving repository 'Hauptaktualisierungs-Repository' metadata .......................................................................................................................................................................[done] 
Building repository 'Hauptaktualisierungs-Repository' cache ............................................................................................................................................................................[done] 
All repositories have been refreshed. 

real    0m22.487s 
user    0m3.778s 
sys     0m0.503s 
erlangen:~ # zypper lr -uE 
#  | Alias                | Enabled | GPG Check | Priority | URI 
---+----------------------+---------+-----------+----------+--------------------------------------------------------------------------------------------- 
 6 | Packman              | Yes     | (r ) Yes  |   90     | https://ftp.fau.de/packman/suse/openSUSE_Tumbleweed/ 
18 | non-oss              | Yes     | (r ) Yes  |   99     | https://mirrorcache-eu.opensuse.org/tumbleweed/repo/non-oss/ 
20 | oss                  | Yes     | (r ) Yes  |   99     | https://mirrorcache-eu.opensuse.org/tumbleweed/repo/oss/ 
27 | update               | Yes     | (r ) Yes  |   99     | https://mirrorcache-eu.opensuse.org/update/tumbleweed/ 
 8 | chrome               | Yes     | (r ) Yes  |  100     | https://dl.google.com/linux/chrome/rpm/stable/x86_64 
13 | home_kukuk_qmapshack | Yes     | (r ) Yes  |  100     | https://mirrorcache-eu.opensuse.org/repositories/home:/kukuk:/qmapshack/openSUSE_Tumbleweed/ 
14 | jalbum               | Yes     | (  ) No   |  100     | https://jalbum.net/download/software/yumrepo/ 
17 | myrepo               | Yes     | (  ) No   |  100     | dir:/home/karl/Downloads/myrepo 
erlangen:~ #

[mrmazda]

It doesn't have to be bad. Today I upgraded a TW 20220218 to 20220515. There were 1177 transactions in just over 10 minutes to complete zypper dup. The PC is over 4 years old, but does run on NVME. It's quick to boot too:

# systemd-analyze
Startup finished in 12.938s (firmware) + 30.248s (loader) + 1.234s (kernel) + 1.405s (initrd) + 2.830s (userspace) = 48.657s
graphical.target reached after 2.824s in userspace

My newest PC

# inxi -SMCd
System:
  Kernel: 5.16.15-1-default arch: x86_64 bits: 64 Console: pty pts/1
    Distro: openSUSE Tumbleweed 20220525
Machine:
  Type: Desktop System: ASUS product: N/A v: N/A serial: N/A
  Mobo: ASUSTeK model: PRIME B560M-A v: Rev 1.xx serial: <filter>
    UEFI: American Megatrends v: 1410 date: 01/28/2022
CPU:
  Info: 6-core model: 11th Gen Intel Core i5-11400 bits: 64 type: MT MCP
    cache: L2: 3 MiB
  Speed (MHz): avg: 861 min/max: 800/4400 cores: 1: 801 2: 801 3: 801
    4: 801 5: 801 6: 801 7: 800 8: 801 9: 801 10: 800 11: 1528 12: 800
Drives:
  Local Storage: total: 238.47 GiB used: 11.95 GiB (5.0%)
  ID-1: /dev/nvme0n1 vendor: TeamGroup model: TM8FP6256G size: 238.47 GiB

just did 1046 transactions from already downloaded cache in 3m14s real, 0m46s user, 0m30s sys. I installed the latest kernel separately in 0m20.765s, 0m25.317s user, 0m5.265s sys.

# systemd-analyze
Startup finished in 12.488s (firmware) + 12.566s (loader) + 1.025s (kernel) + 1.336s (initrd) + 3.520s (userspace) = 30.937s
graphical.target reached after 3.501s in userspace

I still have to put in more time getting rid of unnecessary boot startups, but it is doing quite a bit better than my 4 year older Kaby Lake, which has faster but fewer cores. Possibly the extra time to graphical.target had to do with 3 connected displays instead of 1. Tried with only one next boot:

# systemd-analyze
Startup finished in 15.840s (firmware) + 11.874s (loader) + 1.050s (kernel) + 1.385s (initrd) + 2.746s (userspace) = 32.897s
graphical.target reached after 2.732s in userspace

Looks like could be.

[non_space]

As an "enthusiast", it behooves you to learn what systemd services and timers are, and how to create and maintain custom ones, as karlmistelberger has done.

PS: try prefixing commands like zypper dup with the time command.

@mrmazda:

My enthusiasm is for running a number of different OSs . . . "master of none." But, thanks for the "time" suggestion . . . did fiddle with that in my Leap 15.4 install . . . .

Hi
My only downtime is manually building the Nvidia driver....

I don't use virtualbox and derivatives, kvm/qemu/vagrant works OTB for me.

No crisis, your locale may be the issue, using third party products and repositories, I only use the oss and update repo. For other rpms I have a local repo and enable the google-chrome one when needed.

@malcolmlewis:

Alrighty, I'll check into cutting down the repos when I get a free minute.

Yep. Basically you may define a service by creating file /etc/systemd/system/dup.service and invoke the service by running "sytemctl start dup.service".

# /etc/systemd/system/dup.service
[Unit] 
Description=Dist Upgrade 

erlangen:~ # journalctl -b -u dup.service -g Pakete   
 
erlangen:~ # journalctl -b -u dup.service -g Consumed 
May 26 10:46:10 erlangen systemd[1]: dup.service: Consumed 5.943s CPU time. 
erlangen:~ #

You may test your connection by running all of the below commands. For ease of discussion don't make any changes. Include all output, also the command prompts.
[CODE]erlangen:~ # zypper clean --all
All repositories have been cleaned up.
erlangen:~ # time zypper refresh

@karlmistelberger:

So "close" and yet so far . . . . I booted TW this morning, out of rotation, to test out yr suggestion . . . tried to create that file, but I guess I don't know the SUSE way of doing that. In the ubuntu flavors we might use #mk xxxxxx to make the file . . . didn't work. I tried what I thought might be the SUSE way #mkconfig xxxxxxx but that also was "not found." Didn't have time to search the web for my answer . . . shopping on in my day. Another "159" packages are waiting for me in TW . . . .

Also, in your other command where "Pakete" is used . . . I'm assuming that is a linguistic "bleed" over into the Deutsch?? and would be Packet in the King's English????

[karlmistelberger]

just did 1046 transactions from already downloaded cache in 3m14s real, 0m46s user, 0m30s sys. -> 1046/(46 + 30) = 13.8 packages/second

Compare to Notebook with i5-8250U: 2553 packages, real 16m19.803s, user 7m1.228s, sys 2m58.022s -> 4.26 packages/second

Normalized values:
i5-11400: 2.6/4.4 GHZ, 6 cores: 13.8/(6*2.6) = 0.88
i5-8250U: 1.6/3.4 GHZ, 4 cores: 4.26/(4*1.6) = 0.67

The desktop yields better performance compared to the notebook. Given the many compromises the performance of the latter is still impressive.


A more significant metric of resources consumed is obtained by running zypper dist-upgrade as a service, which yields all of size of download, number of packages and total cpu time consumed based on cgroups in a nice and tidy report.

https://en.wikipedia.org/wiki/Cgroups

[deano_ferrari]

. . . tried to create that file, but I guess I don't know the SUSE way of doing that.

Hi. Create the file with an editor (as root) eg nano...

sudo nano /etc/systemd/system/dup.service

Add the required, and save when done.

Start and enable the service...

sudo systemctl --now enable dup.service

One of many guides...
https://www.suse.com/support/kb/doc/?id=000019672
https://www.linode.com/docs/guides/s...rvice-at-boot/

[mrmazda]

Compare to Notebook with i5-8250U: 2553 packages, real 16m19.803s, user 7m1.228s, sys 2m58.022s -> 4.26 packages/second

Normalized values:
i5-11400: 2.6/4.4 GHZ, 6 cores: 13.8/(6*2.6) = 0.88
i5-8250U: 1.6/3.4 GHZ, 4 cores: 4.26/(4*1.6) = 0.67

I don't understand why divide base freq by turbo freq?

I'm not sure this math is complete normalization.

1. My cache is on LAN server's rotating rust RAID1, not NVME SSD, so the packages are taken up over gigabit ethernet bottleneck instead of PCIe being the bottleneck. I need to try again when another 1k packages are need, and running zypper after copying the cache content to NVME.
2. I need to monitor core speeds. Turbo may or may not be being employed.