Updating to a prior version of tumbleweed

Following these instructions last night I updated openSUSe Tumbleweed

basic install is - Gnome desktop with powertop, audacity and vlc added
Logout from GNOME, in the login screen hit Ctrl+Alt+F2. You’ll see a login prompt. Login as your normal user, and do

 Code:

 sudo zypper dup 

enter the root password and off it goes.

If the end of the ‘dup’ says you have to reboot, do
Code:
sudo reboot
If it doesn’t, do
Code:
exit
then hit Ctrl+Alt+F7 to return to the login screen.

The reboot had a problem purging old kernels.

I reloaded based on a 5/2020 DVD and updated again

When I reboot every two minutes I get an error
BUG: workspace lockup - pool cpus=4 node=0 flags=0x0 nice=0 active=4/512 refcnt=10

I have an Intel i7 with 6 cores and 12 logical processors if that matters.

What I think I need to do is restore 5/2020 from dvd again and update to a couple of days ago to get the machine working again.

Where can I find the tumbleweed release identifiers to selectively update to a specific release/date.

By being able to control the date of recovery I should be able to isolate the update causing my problem.

I can also get the machine back to operating temporarily with a backlevel version of the OS.

Thanks if you can help.

tumbleweed-cli could help: https://review.tumbleweed.boombatower.com/about.html

Have you tried running snapper to see if you have a snapshot of that prior TW version?
If you do, you might just rollback instead of re-install.

I’ve sometimes found TW re-installs and repairs problematic if packages and package versions are no longer available online.

BTW -
If you want to post code highlighted like the following example, just click on the hash butt on (#) to automatically create the Wikitext tags.

Example

TSU

If the OP cannot rollback then I do think tumbleweed-cli (AKA tumbleweed snapshots) might be the easiest way to get back to TW at a particular past date. Plus it’s relatively easy to switch between tumbleweed-cli and tumbleweed. The main issue with tumbleweed-cli is that sometimes packages from repos such as pacmac have been rebuilt for the latest TW and are ahead of tumbleweed-cli. But if you mainly stick with stock TW, then tumbleweed-cli should be quite usable.

A while back I wrote up some tumbleweed-cli usage notes to help explain the tumbleweed-cli concepts.

I reinstalled 20200516 (my full DVD version).

Using my gnome logon

I used the 1 click install from https://software.opensuse.org/package/tumbleweed-cli to install tumbleweed-cli

logging out and getting into the command line

tumbleweed list
got me a list of repo names

tumbleweed switch --install 20200609

sudo reboot

worked

tumbleweed switch --install 20200610

reboot was not required by I did it anyway

sudo reboot

worked

tumbleweed switch --install 20200611

sudo reboot

worked

tumbleweed switch --install 20200612

sudo reboot

failed to boot. Error message looked similar however I could not see the beginning of the error message the BUG: line.

I will wait till tomorrow to reload 20200516 and update to 20200611 to get a working machine back.

probably booting from the 20200516 full dvd?
Is there anything I should get from the machine that does not boot before I reload it to facilitate debugging?
Is there a way to undo the 20200612 update without reinstalling and updating again?

Thanks for the help so far. I now know how to select which update to load.

Use the grub menu to boot from a previous, working snapshot then rollback:
https://doc.opensuse.org/documentation/leap/reference/html/book.opensuse.reference/cha-snapper.html#sec-snapper-snapshot-boot

Doing this way, you don’t need to recover any logs, they would be still under /var, this subvolume is kept intact during system rollbacks.

Thanks – glad I waited.

If you have a wifi card you may have been affected by this bug:

https://bugzilla.opensuse.org/show_bug.cgi?id=1172905

It took me a while to track down why I couldn’t boot the 5.7.1 kernel. I have a wifi card on my motherboard (which I don’t use) and blacklisting iwlwifi solved the problem. Of course, if this is your problem and you are using wifi, then you will have to boot with an older kernel. As things are working with 5.7.2 it shouldn’t take look for this to be fixed.

I chose advanced options from the boot menu

In the boot menu I have

openSUSE Tumbleweed with Linux 5.7.1.1-default
openSUSE Tumbleweed with Linux 5.7.1.1-default (recovery mode)
openSUSE Tumbleweed with Linux 5.6.14.1-default
openSUSE Tumbleweed with Linux 5.6.14.1-default (recovery mode)
openSUSE Tumbleweed with Linux 5.6.12.1-default
openSUSE Tumbleweed with Linux 5.6.12.1-default (recovery mode)

I believe
5.7.1.1 is update 20200612
5.6.14.1 is update 20200611
there is no update 20200610 because the kernel did not change
5.6.12.1 is update 20200609
update 20200516 is missing for lack of disc space.

I chose openSUSE Tumbleweed with Linux 5.6.14.1-default

I logged into the console as before

since I used tumbleweed to do the updates instead of zypper (directly in any case)
I tried the following, If I am wrong I will learn.
tumbleweed revert --install

It did not request a reboot but I did one anyway (it wanted zypper to restart some tasks)

sudo reboot

When I got to the boot menu screen

I chose advanced options again

That gave me a menu where I lost one option from before
openSUSE Tumbleweed with Linux 5.7.1.1-default
openSUSE Tumbleweed with Linux 5.7.1.1-default (recovery mode)
openSUSE Tumbleweed with Linux 5.6.14.1-default
openSUSE Tumbleweed with Linux 5.6.14.1-default (recovery mode)
openSUSE Tumbleweed with Linux 5.6.12.1-default

I may have been wrong because the 5.7.1.1 kernel boots did get removed from the menu.

The 5.6.14.1 boot is functional in terms of what I do with the machine.

I have a wireless card however my error message indicates a workspace lockout and references cpus alot I do not think it is my wireless, but I could be wrong.

I tried to use journalctl --list-boots and journalctl -boot=-3 in my case to get any additional information the repeating message was not there. Perhaps in some other log file where I did not look. I may have been wrong but after this error looped about 5 times I rebooted the machine with the power switch.

Thanks to ALL

If it matters –
I also did a network install of 20200612 from the x86-64 cd (using my wireless to retrieve the install sources) it worked just fine until I tried to boot the kernel.

If you want to check out whether it is the wifi card, edit /etc/modprobe.d./blacklist.conf and add at the end


blacklist iwlwifi

and reboot to the 5.7.1 kernel. If successful you know this is the problem. If not, reboot to the 5.6.14 kernel and remove the blacklist entry; also do this if successful but you need the wifi. When a new kernel is released you can probably remove the blacklist.

In my /etc/modprobe.d directory there are about 31 *.conf files 29 of them are prefixed with a 2 digit number followed by a dash. The other files are tuned.conf and firewalld-sysctls.conf. The closest file matching the name you specified is 50-blacklist.conf. There is also a 50-iwl3945.conf file. Do you want me to add a new file or use one of the ones named. FYI my wireless device in Yast Hardware is identified as "Wireless-AC 9560 [Jefferson Peak] (wlo1) under Network Card., if it matters

The 50-blacklist,conf file opens read-only n the gnome text edit app.
Therefore if you choose that file I believe sudo edit from a terminal window will be required to edit it.
The 50-blacklist.conf file contains the following:

$Id$

Listing a module here prevents modprobe from loading it via modalias (only

aliases from /lib/modules/*/modules.alias). You may still load it explicitely.

We blacklist some modules becaus they may harm on certain devices or they

prevent other modules from grabbing the device.

Syntax: blacklist <driver name>

See ‘man modprobe’.

usbcore … module is loaded implicitly, ignore it otherwise

blacklist usbcore

tulip … de4x5, xircom_tulip_cb, dmfe (…) handle same devices

blacklist de4x5

At least 2.4.3 and later xircom_tulip doesn’t have that conflict

xircom_tulip_cb

blacklist dmfe

list all framebuffer drivers, some of them tend to crash during boot

they are either compiled into the kernel, or vesafb is active

X works fine without them, rcfbset can load them if really required

sed -e ‘//drivers/video/..(o|ko)$/{s@^./@@;s@…*$@@;p};d’

blacklist aty128fb
blacklist atyfb
blacklist cyber2000fb
blacklist g450_pll
blacklist hgafb
blacklist i2c-matroxfb
blacklist i810fb
blacklist intelfbdrv
blacklist intelfbhw
blacklist matroxfb_accel
blacklist matroxfb_base
blacklist matroxfb_crtc2
blacklist matroxfb_DAC1064
blacklist matroxfb_g450
blacklist matroxfb_maven
blacklist matroxfb_misc
blacklist matroxfb_proc
blacklist matroxfb_Ti3026
blacklist mdacon
blacklist neofb
blacklist pm2fb
blacklist pm3fb
blacklist radeonfb
blacklist rivafb
blacklist sisfb
blacklist sstfb
blacklist tdfxfb
blacklist tridentfb
blacklist vga16fb
blacklist vgastate

for kyrofb see Bug 35810

blacklist kyrofb

list was not complete (bug 106715)

blacklist arcfb
blacklist backlight
blacklist lcd
blacklist cirrusfb
blacklist gx1fb
blacklist intelfb
blacklist macmodes
blacklist nvidiafb
blacklist s1d13xxxfb
blacklist savagefb

additional modules since SLE11, bug 468964

blacklist arkfb
blacklist carminefb
blacklist gxfb
blacklist hecubafb
blacklist lxfb
blacklist s3fb
blacklist sm501fb
blacklist viafb
blacklist vmlfb
blacklist vt8623fb
#bug 846218
blacklist udlfb

ISDN modules are load from /lib/udev/isdn.sh

blacklist fcusb
blacklist fcusb2
blacklist fxusb
blacklist fxusb_CZ
blacklist fcdslusb
blacklist fcdslusb2
blacklist fcdslusba
blacklist fcdslslusb
blacklist fcdslslusb2
blacklist e2220pc
blacklist e5520pc
blacklist bfusb
blacklist b1isa
blacklist b1pci
blacklist b1pcmcia
blacklist c4
blacklist t1isa
blacklist t1pci
blacklist divas
blacklist act2000
blacklist hfc_usb
blacklist hisax
blacklist hisax_fcpcipnp
blacklist hisax_st5481
blacklist hysdn
blacklist icn
blacklist pcbit
blacklist sc
blacklist tpam
blacklist fcpci
blacklist fcclassic
blacklist fcdsl
blacklist fcdsl2

mISDN modules

blacklist hfcsusb
blacklist hfcpci
blacklist hfcmulti
blacklist l1oip
blacklist mISDN_dsp
blacklist mISDN_core

OSS PCI sound modules

blacklist ad1889
blacklist ali5455
blacklist btaudio
blacklist cmpci
blacklist cs4281
blacklist emu10k1
blacklist es1370
blacklist es1371
blacklist esssolo1
blacklist forte
blacklist i810_audio
blacklist maestro
blacklist maestro3
blacklist nm256_audio
blacklist opl3sa2 # Bug 219758
blacklist rme96xx
blacklist sonicvibes
blacklist trident
blacklist via82cxxx_audio
blacklist ymfpci

this is a debugging module which should only be loaded manually

blacklist evbug

These mtd drivers should be loaded manually.

blacklist scb2_flash
blacklist pci
blacklist l440gx
blacklist amd76xrom

job of rcdvb

blacklist snd-bt87x

This module seems to be good for nothing. See bug 129301.

blacklist dpt_i2o

These devices have bt878 chip without PCI Subsystem ID. Without that info bttv

does not know how to treat them properly. Therefore we disable autoloading of

modules for these devices.

See https://bugzilla.novell.com/show_bug.cgi?id=149588

To enable your device create a hardware configuration file for your device.

See man hwup for details.

You will probably have to specify an option to identify your card. Have a

look in /usr/src/linux/Documentation/video4linux/CARDLIST.bttv.

alias pci:v0000109Ed0000036Esv00000000sd00000000bc04sc00i00 bttv_skip_it
alias pci:v0000109Ed00000878sv00000000sd00000000bc04sc80i00 bttv_skip_it
install bttv_skip_it echo “module alias skipped (bt878 chip without PCI Subsystem ID)”

For some bridges both intel-agp and i82875p_edac are loaded. If i82875p_edac

is loaded first it will grab the device. Then intel-agp doesn’t work.

Therefore we disable automatic loading of 82875p_edac. (Bug 213840)

blacklist i82875p_edac

Blacklist the IBM s390 module for I/O dynamic configuration support

Bug bnc#478601

blacklist chsc_sch

Blacklist acpi_power_meter. The device requires processing ACPI AML code to

update average power measurement. This may be at a high frequency and has

been observed executing every 500ms. This has a noticable impact on latency

sensitive applications that experience delays on workqueue executions. As

very few applications require the data, blacklist the module by default

(bnc#974373)

blacklist acpi_power_meter

\

Also to tsu2

Continuing my learning

This morning I booted to 5.6.14.1-default and logged into console and entered

snapper ls

The result was
The config ‘root’ does not exist. Likely snapper is not configured.
see ‘man snapper’ for further instructions.

Meaning any consideration of a rollback would require some after the fact configuration or an opportunity to reinstall and configure snapper to work with tumbleweed-cli better.

I logged out of console and went back into the gnome desktop

Using Yast Software with the down arrow I searched for snapper

The following are installed

grub2-snapper-plugin
libsnapper5
snapper
snapper-zypp-plugin
yast2-snapper

NOT installed
pam-snapper

Did I Not configure snapper when I should have? (probably after the install of 20200516)

(the following may mean I read too much that I do not understand)
home is currently a separate xfs partition for one user, me.
Should home be a btrfs partition and should pam-snapper for my user be setup to snapshot changes like the root (/) directory should have had?
so things like firefox preferences and bookmarks are saved for the appropriate version when rolling back?

Add the line at the end of 50-blacklist.conf

It’s been a long time since I’ve used gnome. You do have to open the file in su mode. I remember dropping to a shell and then doing

sudo gedit /etc/modprobe.d/50-blacklist.conf

You can always use nano or similar.

I generally assume that all TW are installed with the default filesystem BTRFS which should automatically enable snapper.
Snapper is installed in all systems but activated only if BTRFS is your root partition filesystem… If you installed ext4 or some other filesystem, snapper won’t work or activate.
If you installed with ext4, you should install timeshift (it’s in the openSUSE repos) and you’d have to manually configure your snapshot schedule.

Up to and including LEAP 15.0, BTRFS was the default for the root parrtiton and /home was deployed as a separate partition formatted as XFS.
Starting with LEAP 15.1, the entire system except /boot has been formatted as a single big BTRFS partition, with /home deployed as a BTRFS sub-volume.
If you’ve been upgrading from a system originally built as 15.0 or earlier, then your system will have the older layout with /home formatted as XFS.

Regardless of your layout, numerous directory locations are excluded from a snapshot by default, the intention of snapshots is to enable system recovery without wiping out personal files. This means that where you store your personal files has become important, so for instance don’t create a directory in your root directory and store any files you want to survive a rollback there.

TSU

Did you see the option ‘Start from a read-only snapshot’ in grub menu? I’m afraid you only have the option to boot another kernel, but from the same system installation.

# is there a config for automatic snapshots on /?
cat /etc/snapper/configs/root

# are there any snapshots? look for subvolumes under @/.snapshots
btrfs subvolume list / 
sudo ls /.snapshots

If your / partition is not btrfs or smaller than a threshold (~40GiB) then you won’t have snapshots.

You can find instructions on how to create a snapper config on https://en.opensuse.org/openSUSE:Snapper_Tutorial

Root Partition is btrfs but too small only about 32gb. Followed a recommendation I found somewhere apparently Leap15 related based on your comments. Now is the time to reload and alter the partition setup. thinking 64 GiB for root including home. Tumbleweed should support several snapshots I would think. IS it worth considering using pam-snapper with the home directory? Learning to utilize snapshots correctly is the reason home can co-exist with root in the same BTRFS partition. I’m an old guy and separate partitions were the normal solution.

I will test the blacklist shortly (before reloading). My machine has two GPU’s an intel and an nvidea. I did not install the experimental nvidea driver, but I saw a proprietary nvidea driver issue on this site. That could also be a possible issue.

 21:37:19  zypper --quiet info pam_snapper
...
Description    :
    A PAM module for calling snapper during user login and logout. 

For /home, possibly. For / (root subvolume), zypper and YaST create snapshots every system change. But for home, you need the find another way to create these snapshots. It seems this module can fulfill this need. Also, snapper can create periodic snapshots (timeline). Beware, if you change lots of data (a few gigabytes between snapshots) you might get out of space frequently, since 64GiB might not support this pattern.

I think you meant subvolumes. But yeah. Each subvolume it’s like a separate partition except they allocate empty blocks from the same pool. A snapshot it’s a read-only (usually) window into a subvolume at a given point in time.

You should be able to use nvidia driver with kernel 5.6.x in the meantime. If you reinstall with kernel 5.7.x, you’ll need nouveau. Or, you’ll need follow then “the hard way”.

I just tried the blacklist iwlwifi rebooting 5.7.1.1 I SUCCESSFULLY got to the Gnome Desktop. no nouveau (maybe I am only using the intel gpu)

I will keep your sizing in mind bumping the work machine to 82GB + 18GB swap. I store very little data on home mostly the stuff installs put there like app data (firefox). I store everything from that box on an external drive. My other machine I will probably use a larger partition size. Thanks to all I think this is for me to do now. You were great.