Why release such a large TW update?

I don’t understand. It would have taken less time to install TW from scratch.
I run a zypper dup every single day. And every now and then, the update is massive (1 hr 24 minutes today!)

What are the release managers thinking? Poor planning in my view.
(before someone suggests I don’t know what I’m talking about, I’m a Sr Technical Project Manager for a software company - been in the software field for 30+ years).

And now, I have to do the same massive update on another machine.

2298 packages to upgrade, 3 new, 3 to remove.
Overall download size: 1.72 GiB. Already cached: 0 B.
After the operation, 20.1 MiB will be freed.

yes, it took 1 hour and 24 minutes !!

2023-08-13 17:18:32 zypper -vvv dup
2023-08-13 18:42:14 zypper ps -s

One way to avoid monster updates is to trim the installation down to what you do or expect to need. Try running sudo zypper pa --unneeded to see what it suggests you have no need for. My TW installations are typically in the neighborhood of 1,200 total packages installed. One with Plasma just booted has 1,188.

Another way to avoid unneededs:

# grep onlyReq /etc/zypp/zypp.conf
 solver.onlyRequires = true
#

It doesn’t help a lot if you don’t configure it early in the life of the installation.

20230812 includs a new version of glibc, one of those packages that causes a massive need for rebuilding the packages that depend on or incorporate it.

3 Likes

Install/Boot/Login is a support forum. What’s your support request?

Mirrors take a time to catchup after big updates. I only download updates in advance of installation, so download times are not something bothers me.

1 Like

@aggie no poor planning, it’s always expected when glibc is rebuilt, as a Tumbleweed user it’s not unusual… Follow the link in the announcement if wanting more information…

4 Likes

You’re lucky. I just ran -dup- and I’ve got 2,808 packages updating (1 hr 5 min). Plus, I have three more machines to do !

Part of the nature of the distro, as I understand it - if a package is updated that is itself part of the process of building other packages, then anything built using that first package will be rebuilt to make sure that the distribution is consistent.

AFAIK isn’t that an intended effect of the Tumbleweed approach to deployment? So, I wouldn’t suggest that you don’t know what you’re talking about, but I would suggest that perhaps you aren’t familiar with what a Tumbleweed “release” is.

5 Likes

Thanks @mrmazda … I will run the zypper command and check the .conf file.

We can move this to Chat, if you wish @awerlang

And for @johnkizer … I have one TW machine that was installed in 2018 (I think) :+1:

So, in zypp.conf, I have
solver.onlyRequires = true

… and the zypper pa --unneeded shows me about 225 files.

(@mrmazda ) What is it that zypper is basing its decision on, which packages can be removed?

(I checked the man page and it didn’t have much about it)

It depends on several factors probably like connection speed, faster mirror, and cpu speed.
I started my zypper dup at 9:47 and was done by 10:29 Eastern Time for 2162 packages.
Finish the zypper dup in 42 minutes.

3 Likes

It’s showing you 225 packages installed that are not there because any currently installed package requires them. Choosing how to respond is an artful mental exercise. Responding can consume a lot of time. If you choose to remove any, you’ll almost certainly find removal of some packages will result in other packages joining the list. IOW, check, delete, check, delete, check, delete…where did the time go.

3 Likes

Did you mean from this news (the snapshot 20230811)? I’m running MicroOS (which is only the base packages), have tried to check upgrading the OS manually. Seems this rolling release (20230812) glibc was rebuilt again

The following 333 packages are going to be upgraded:
  MicroOS-release MicroOS-release-dvd NetworkManager NetworkManager-branding-openSUSE NetworkManager-tui aaa_base alts apparmor-parser audit bash
  bash-sh bc biosdevname boost-license1_82_0 btrfsprogs btrfsprogs-udev-rules busybox catatonit chkstat chrony chrony-pool-openSUSE cni cni-plugins
  compat-usrmerge-tools conmon container-selinux containerd coreutils cpio cracklib cracklib-dict-small crypto-policies cryptsetup curl dbus-1
  dbus-1-common dbus-1-daemon dbus-1-tools device-mapper dialog diffutils docker docker-compose dosfstools dracut dracut-transactional-update
  e2fsprogs ed efibootmgr elfutils file file-magic fillup findutils fuse-overlayfs gawk gdk-pixbuf-query-loaders gettext-runtime git git-core
  glib2-tools glibc glibc-locale glibc-locale-base gpg2 gptfdisk grep grub2 grub2-i386-pc grub2-snapper-plugin grub2-x86_64-efi gzip hostname
  ignition ignition-dracut-grub2 iproute2 iptables iputils kbd kbd-legacy kdump kernel-default kexec-tools kmod krb5 lastlog2 less libICE6 libX11-6
  libX11-data libXmu6 libXt6 libabsl2301_0_0 libacl1 libalternatives1 libapparmor1 libargon2-1 libasm1 libassuan0 libattr1 libaudit1 libaugeas0
  libauparse0 libblkid1 libboost_thread1_82_0 libbpf1 libbrotlicommon1 libbrotlidec1 libbtrfs0 libbtrfsutil1 libbz2-1 libcap-ng0 libcap2 libcares2
  libcom_err2 libcrack2 libcrypt1 libcryptsetup12 libcurl4 libdb-4_8 libdbus-1-3 libdevmapper-event1_03 libdevmapper1_03 libdialog15 libdw1 libeconf0
  libedit0 libefivar1 libelf1 libexpat1 libext2fs2 libfa1 libfdisk1 libffi8 libfido2-1 libfreebl3 libfreetype6 libfuse2 libfuse3-3 libgcc_s1
  libgcrypt20 libgdbm6 libgdbm_compat4 libgdk_pixbuf-2_0-0 libgio-2_0-0 libglib-2_0-0 libgmodule-2_0-0 libgmp10 libgnutls30 libgobject-2_0-0
  libgpg-error0 libgpgme11 libgpm2 libhogweed6 libidn2-0 libip4tc2 libip6tc2 libjbig2 libjitterentropy3 libjpeg8 libjson-c5 libkeyutils1 libkmod2
  libksba8 liblastlog2-1 libldap2 liblua5_4-5 liblz4-1 liblzma5 liblzo2-2 libmagic1 libmetalink3 libmount1 libmpdec3 libmpfr6 libncurses6 libndp0
  libnettle8 libnewt0_52 libnfnetlink0 libnghttp2-14 libnl-config libnl3-200 libnm0 libnotify-tools libnotify4 libnss_usrfiles2 libopenssl3
  libp11-kit0 libpci3 libpcre2-8-0 libpkgconf3 libpng16-16 libpopt0 libprocps8 libprotobuf-lite23_4_0 libproxy1 libpsl5 libpwquality1
  libpython3_11-1_0 libreadline8 libsasl2-3 libseccomp2 libselinux1 libsemanage-conf libsemanage2 libsepol2 libsha1detectcoll1 libslang2 libslp1
  libsmartcols1 libsnapper7 libsnappy1 libsoftokn3 libsolv-tools libsqlite3-0 libssh-config libssh4 libstdc++6 libsubid4 libsystemd0 libtasn1-6
  libtextstyle0 libtiff6 libtukit4 libudev1 libunistring5 libusb-1_0-0 libutempter0 libuuid1 libwrap0 libwtmpdb0 libxcb1 libxml2-2 libxml2-tools
  libxtables12 libxxhash0 libyajl2 libyaml-cpp0_7 libz1 libzck1 libzstd1 libzypp login_defs logrotate lsof makedumpfile mc microos-tools minikube
  mokutil mozilla-nspr mozilla-nss mozilla-nss-certs nano ncurses-utils netcfg openssh openssh-clients openssh-common openssh-server openssl
  openssl-3 p11-kit p11-kit-tools pam pam-config perl perl-Git perl-base permissions permissions-config pigz pinentry pkgconf pkgconf-m4
  pkgconf-pkg-config podman policycoreutils procps psmisc python311-base rebootmgr rpm rsync runc screen sed selinux-policy selinux-policy-targeted
  selinux-tools shadow shared-mime-info slirp4netns snapper sudo system-group-audit system-group-hardware system-group-kvm system-user-nobody
  system-user-root systemd systemd-coredump systemd-presets-branding-MicroOS systemd-rpm-macros sysuser-shadow tar terminfo-base
  thin-provisioning-tools timezone transactional-update transactional-update-zypp-config tukit udev unzip update-alternatives util-linux
  util-linux-systemd vim-data-common vim-small virtualbox-guest-tools virtualbox-kmp-default wget which wpa_supplicant wtmpdb xen-libs
  xtables-plugins xz zip zstd zypper zypper-needs-restarting

The following product is going to be upgraded:
  openSUSE MicroOS  20230811-0 -> 20230812-0

The following package requires a system reboot:
  kernel-default

333 packages to upgrade.
Overall download size: 415.1 MiB. Already cached: 0 B. After the operation, additional 1.9 MiB will be used.

    Note: System reboot required.
Continue? [y/n/v/...? shows all options] (y):

sorry, the last message was accidentally deleted.

@handevsar Hi, not that one, that’s the weekly summary, It’s the one starting New Tumbleweed Snapshot [DATE] https://forums.opensuse.org/t/new-tumbleweed-snapshot-20230812-released/

1 Like

I just ran the zypper on TW yesterday, thinking that 1672 packages was a lot, didn’t actually take too long as my TW install is running on a SSD . . . .

But, when I posted this same basic complaint some months back I got the suggestion to “make a script” to run the dup . . . saying something like, “shut up and dup” ( something like that which was a quote from a famous computer guy, but which I took umbrage with) . . . . Followed by the cryptic, “just dup it” . . . as the hint.

After I calmed down that gent then gave step by step instructions on how to set up two scripts, one to download and one to run the install) and I did set them up. Problem was you have to run some command to show you how the process is going, while it is running the script, in my case I boot TW on Monday mornings for a few hours . . . so I wouldn’t know if I was shutting down in the middle of some . . . dup-ing.

In the case of the OP if TW is the only system running, and better, if the machine is running all of the time, then setting up these scripts was actually pretty easy . . . even for me with no background in programming. Once the script is “started” it’s trolling for packages to update, it runs in the “background” . . . never have to watch the clock ticking as zypper runs through its thing.

2 Likes

I would tend to concur with the idea that a massive update after just 24 hours is excessive and poor production planning. Nevertheless, I would also suggest that as system manager, one should be sufficiently equipped to adapt to such a situation. All my machines run a root-based cronjob to do a —download-only update at every reboot. This avoids running the full dup, download and all, all in one go.
A systemd-inhibit can be run at a convenient moment - end of day, for example, to complete the update

2 Likes

a] Running on an SSD also.

b] I’m pretty sure I know exactly who is it … and no, not famous, as they constantly use the word that is the complete opposite. Shows lots of examples, but zero explanation.

c] I will search for the post to view the script - although, I’m leery that the script would do an automatic reboot when I’m in the middle of work. Not good. (unless I misunderstand).

d] (OP here) Machines are not running all the time - one is a laptop and the other is a desktop - only boot each when necessary (or to ensure it has the most recent update) … but yea, in general, boot every day.

Thanks for the details!

You could just install cronie if its not already installed and create a cronjob.

sudo zypper in cronie

then create a cron job to only download the packages. after that you can install whenever you want.

sudo crontab -e -u root

type into it:

@reboot sleep 1m && /usr/bin/zypper ref && /usr/bin/zypper --non-interactive dup --download-only --auto-agree-with-licenses >/dev/null 2>&1

@aggie

Well I have seen the guy quoted on mastodon for other quotes other than the “shut up and dup” quote . . . a professor at some school perhaps in the UK??

Anyway, my TW machine is shut down for the day, other stuff going on, but yes you could search my posts and see the similar "Why is this overload happening? thread, with similar, “that’s how TW does it’ answers, but in there is, as the recent poster suggesting 'cronjob”??? also posted.

I did mess with it, you write the script so I guess if you put “systemctl reboot” at the end of the script it would run that; if it’s not in there then it only does what you . . . “command” it to do . . . . The only caveat would be that it obviously is using cpu capacity to run and download and install the packages . . . ??? I would guess with modern machines that have “6 threads” function it prolly wouldn’t hurt.

As a 'control" type computer user I was also paranoid about just having a script running, “doing stuff without my explicit knowledge” so I spent more time “watching” it than was worth doing . . . but, as mentioned, you write the script.

The joy of TW is that it is “fresh horsies” . . . so the newest stuff is always coming along . . . hard to say whether time saved over not having to run an upgrade into the newest distro over having to run zypper dup every day or week . . . . But, with the script now that is “time saved.”

I can’t see a difference of running a script or simply open a terminal and run “zypper dup”. You can minimize the terminal and proceed with your work whilst the download and dup is running. After that a simple reboot and everything is done. You don’t need to rebot immediately if you have other things ongoing which prevent this step…Only took 20mins for download and installation of ~2500 packages …

“8 million stories in the Naked City” . . . title of an old TV show in LA when there were “8 million” people living there.

The OP reported essentially needing 1.5 hours to run the recent packages . . . i have had similar time frames used to get and install the sometime “balloon” upgrades that are associated with TW . . . . Now it doesn’t seem to take so long when I run a zypper dup . . . one reason on the download side is I changed to a FIOS internet connection some months back. For me it wasn’t always the download times that were at issue, but something in zypper seemed to bog down.

In my case my machine is not a “server” that is on all the time, I log in, check emails run the upgrade of the day and then shut down. I’ve got other stuff going on, so the time spent to run the large packages is . . . not an area of “increased personal pleasure” . . . knowing that my TW system is “totally uniquely fresh in every package” . . . doesn’t carry me too far through my day, etc.

It’s one of those 'rock and a hard place" issues, as far as how TW runs its system updates vs say how Manjaro runs their 'rolling" upgrades . . . 50 to 60 packages every once in awhile . . . not so time consuming . . . .

1 Like

This last update was the first one I’ve run that actually crashed X during the update. Luckily I was successful rebooting and finish updating the remaining 260+ packages. But from now on, I’m logging out of X, opening a virtual terminal with ctrl-alt-F1, and doing the update there.

Gene