Upgraded to 16.0 from 15.6 - failed

I upgraded a server from 15.6 to 16 using the migration script. (I would offer a more complete name but the server is down.)

Booting after the upgrade took me to the RESCUE system. Hardly what I expected.

The last few entries in the journal are (paraphrased):

Received SIGTERM+21 from pid 523 (Plymouth)
...
rfkill hostnamed
...
hostnamed.service successfully entered the 'dead' state

What do I do now?

I assume you’re referring to the openSUSE Migration Tool.

Try to resolve manually

If you can access a TTY terminal, you can identify which services have failed using
systemctl --failed

then try restarting them with
sudo systemctl restart [service_name].service

If that restarts, then
systemctl status plymouth-start.service

Roll-back

If your \ is on a BTRFS partition, then you could use snapper via the command line to roll back to Leap 15.6.

Fresh install

The last resort is to reinstall everything using Agama.

# systemctl --failed
UNIT LOAD ACTIVE SUB DESCRIPTION

0 loaded units listed
#

Nothing failed?

Here are some (transcribed) messages in the journal.

Starting default.target
Failed to start default.target: unit may not be isolated
Fallback to single user shell

I presume this means I’d have to rebuild our system from scratch.

:: This is for general information only
SIGRTMIN+21 … Disables display of status messages on the console, as controlled via systemd.show_status=0 on the kernel command line.
::

Okay, there is a remote possibility that the issue is corruption or invalid entries in /etc/fstab. Might be worth checking it out.

.
( Sidenote for Wolfheri: not running Windows :slight_smile:

Hello,

I had a similar problem when upgrading. For me it was that / was mounted read-only. In the rescue system I did the following:

mount / -o remount,rw
systemctl start graphical.target

The cause was that default.targetwas not set and additionally I needed to modify
/etc/default/grub
to include rootflags=rw,relatime as a kernel parameter, then run update-bootloader.

So, is your / read-only? Is default.target set?

Best regards,

George

Whoo-hoo! That got it started!

Where, exactly, is that to be added? I do not see any explicit KERNEL options.

Here is /etc/fstab:

UUID=bdb8baee-bade-4cf0-b8b0-173683386949  swap               swap   defaults                    0  0
UUID=b33a1356-36ce-48f2-a87c-6e8eab6186a8  /                  btrfs  defaults                    0  0
UUID=fa19f3c3-ec6a-4e40-8e5e-1c6b7261c007  /usr               btrfs  defaults                    0  0
UUID=fbf9fff4-fd4c-43b7-9d2a-7d69aecf93e6  /home              btrfs  defaults                    0  0
UUID=5ea91d89-4420-4bda-bf25-426e148eb7f5  /data01            btrfs  defaults                    0  0
UUID=93f1dd74-059b-4b4b-9851-0d8a5a892b54  /data02            btrfs  defaults                    0  0
UUID=47e333f8-122f-47fb-8b90-4c656ccc2aad  /data03            btrfs  defaults                    0  0
UUID=b33a1356-36ce-48f2-a87c-6e8eab6186a8  /var/lib/machines  btrfs  subvol=/@/var/lib/machines  0  0

Hello,

in /etc/default/grub edit the line

`GRUB_CMDLINE_LINUX_DEFAULT=“splash=verbose showopts security=selinux selinux=1 rootfstype=ext4 rootflags=rw,relatime mitigations=auto rd.driver.blacklist=nouveau”

GRUB_CMDLINE_LINUX_RECOVERY=“showopts apm=off noresume edd=off powersaved=off nohz=off highres=off processor.max_cstate=1 nomodeset x11failsafe security=selinux selinux=1 rootfstype=ext4 rootflags=rw,relatime”

`

I included mine; Yours will look similar but no exactly the same.

Best,

George

Well, crap. I booted the system. It did not start; it reverted to the single user mode. After doing the disk remount, it did start. Only now Apache2 and named do not start. Sheesh!

named is complaining about a /usr/lib/bind/named.prep file it cannot execute. There was not even a /usr/lib/bind/ directory.

What is that prep script supposed to do and how can I (re-)create it?

I tried zypper in --force named to get the installation requirements. zypper refused indicating there was nothing to do.

# zypper in --force named
Refreshing service 'openSUSE'.
Loading repository data...
Reading installed packages...
'named' not found in package names. Trying capabilities.
No provider of 'named' found.
Resolving package dependencies...
Nothing to do.

Oops. That should have been zypper in --force bind. And it made no difference. It still failed to start because of the missing prep file.

Hello Jim

I do not use bind / named on this machine so I cannot help. One comment though, I suspect that part of the config is not being written properly (why would default.target be missing) and in my case apparmor was still used.

I did not encounter these issues on a clean install on another computer.
For me upgrading brings some subtle differences, probably due to the multitude of configurations floating around, which you need to deal on an ad-hoc basis. What helped in some cases is to

(a) look for version information with for example
zypper se --details bind
(b) de-install a package (take care to backup any /etc/ config files)
(c) re-install it taking into account the information from (a)

I experienced that in some cases incompatible packages had been installed.

Best regards

George

Another thread sent me to SELinux Troubleshooting.

It suggested this:

# setenforce 0
#

This set SELinux to “permissive.” And all of the previously aborted services started without a problem.

In fact this system exhibits the exact problem the site describes and requests a bug report about it.

Did you perform the suggested autorelabel?

No, not yet. Now that I have fully functioning server again, I am reluctant to change things with wild abandon. We must be cautious.

The command systemctl start selinux-autoreload does not work because it does not exist on this host.
The commands touch /.autoreload; reboot made no difference. It was again relegated to a single user mode. Not a clean start.
The command setenforce 0 is still required to start services. setenforce 0 is transient.
From there I can start the Graphical/multi-user interface (systemctl start graphical.target) after reboot.

These are the current kernel parameters:
showopts resume=/dev/sda1 security=selinux enforcing=1 selinux=0 rootflags=rw

Where do you have this commands from? It is autorelabel. Not autoreload.

https://en.opensuse.org/Portal:SELinux

https://en.opensuse.org/Portal:SELinux/Setup

My mistake. autorelabel it is.

I followed the steps to here:

$ sudo update-bootloader 
$ sudo dracut --regenerate-all --force
/usr/bin/ldd: line 159: /lib/ld-linux.so.2: cannot execute binary file: Exec format error
dracut-install: Failed to find module 'pata_atiixp'
dracut[E]: FAILED: /usr/lib/dracut/dracut-install -D /var/tmp/dracut.P3ScYx/initramfs -N ^i2o_scsi$ --kerneldir /usr/lib/modules/6.12.0-160000.32-default -m pata_atiixp ata_generic fan ahci
/usr/bin/ldd: line 159: /lib/ld-linux.so.2: cannot execute binary file: Exec format error
/usr/bin/ldd: line 159: /lib/ld-linux.so.2: cannot execute binary file: Exec format error
/usr/bin/ldd: line 159: /lib/ld-linux.so.2: cannot execute binary file: Exec format error
$

I am guessing that going further is not helpful.

$ ausearch -m avc,user_avc,selinux_err,user_selinux_err -ts boot
<no matches>
$