Hi All,
We are experiencing a strange problem with our installation of Leap 42.1 on our server. The installation is on an LLVM partition. The system has been on automatic updates for a few months now & had not been restarted for more than a month despite (possibly) major/crucial updates.
When we tried rebooting the machine a couple of days ago, it started to hang in the boot sequence & never proceeds to show the login screen. I have pasted below the initial parts of a normal boot sequence (as seen in /var/log/boot.log) which one would expect to see (from other machines in our network with the same OS). The point where our server hangs is marked by a comment.
^[32m OK ^[[0m] Started Show Plymouth Boot Screen.
^[32m OK ^[[0m] Reached target Paths.
^[32m OK ^[[0m] Reached target Basic System.
^[32m OK ^[[0m] Found device ST31000524AS 3.
^[32m OK ^[[0m] Found device /dev/disk/by-uuid/9dcad953-5e95-4ab4-a632-d9796c60e5d9.
^[32m OK ^[[0m] Found device /dev/mapper/volgroup-user.
^[32m OK ^[[0m] Started dracut initqueue hook.
^[32m OK ^[[0m] Reached target Remote File Systems (Pre).
^[32m OK ^[[0m] Reached target Remote File Systems.
Starting dracut pre-mount hook…
^[32m OK ^[[0m] Started dracut pre-mount hook.
Starting File System Check on /dev/disk/by-uuid/9dcad953-5e95-4ab4-a632-d9796c60e5d9…
^[32m OK ^[[0m] Started File System Check on /dev/disk/by-uuid/9dcad953-5e95-4ab4-a632-d9796c60e5d9.
Mounting /sysroot…
^[32m OK ^[[0m] Mounted /sysroot.
^[32m OK ^[[0m] Reached target Initrd Root File System.
Starting Reload Configuration from the Real Root…
Starting File System Check on /dev/volgroup/user…
^[32m OK ^[[0m] Started Reload Configuration from the Real Root.
^[32m OK ^[[0m] Started File System Check on /dev/volgroup/user.
Mounting /sysroot/usr…
^[32m OK ^[[0m] Mounted /sysroot/usr.
^[32m OK ^[[0m] Reached target Initrd File Systems.
^[32m OK ^[[0m] Reached target Initrd Default Target.
Welcome to ^[0;32mopenSUSE Leap 42.1 (x86_64)^[[0m!
HANGS HERE!!!
[^[[32m OK ^[[0m] Stopped Switch Root.
^[32m OK ^[[0m] Stopped target Switch Root.
^[32m OK ^[[0m] Stopped target Initrd File Systems.
Stopping File System Check on /dev/disk/by-uuid/9dcad953-5e95-4ab4-a632-d9796c60e5d9…
^[32m OK ^[[0m] Stopped File System Check on /dev/disk/by-uuid/9dcad953-5e95-4ab4-a632-d9796c60e5d9.
Stopping File System Check on /dev/volgroup/user…
…
…
& SO ON
We tried rebuilding the initrd file (ran mkinitrd from the hard drive after mounting the installation from a rescue session with a DVD), re-installed the boot loader (using grub2-install). We also tried adding a debug option on the grub2 config file during bootup, so it shows more details during the boot sequence. It spews a lot of details which can neither be piper to a file nor does it save that output to a log file on disc. Below I have uploaded a screen shot of the output at the point where the boot sequence hangs with the debug option enabled.
When that has happened here, I took it as a clue that I should use CTRL-ALT-F1 to get a terminal login. And then I would reinstall the Nvidia driver. That was back when I was installing it “the hard way”.
I would guess that you have some kind of video driver problem. Not knowing your hardware, it’s hard to guess beyond that.
Thanks for the reply. There were no video card drivers that were updated in the recent past, we checked the log files. We don’t believe this is due to any hardware drivers. No hardware was also changed. And there is no terminal login spawned when we go to ctrl+alt+F1. I have attached images of what it shows when we type ctrl+alt+F1 & ctrl+alt+F10. ctrl+alt+F7 takes us back to the boot sequence screen posted above…
So we are pretty much stumped with this problem. Any clues how we can make systemd more talkative to reveal more information about what exactly it is trying to find… ?
It cannot “hang here” because you have output going on, so something obviously happens after the point you marked with “HANGS HERE”. And your screenshots are made too early, while it is still in initrd. So please wait until it really stops any output and make screenshots then.
To capture console output you may try to use either serial console or netconsole, depending on what hardware you have.
As I have explained in the first post: “I have pasted below the initial parts of a normal boot sequence (as seen in /var/log/boot.log) which one would expect to see (from other machines in our network with the same OS). The point where our server hangs is marked by a comment.”
We let this system like this for 2 days over the weekend, to see if it continues the boot sequence, it doesn’t … After seeing that we can’t seem to get any more output, we took the screen shots… It really does hang at after displaying " Welcome to openSUSE Leap 42.1 (x86_64)! "
Booting to previous kernel version doesn’t help, it shows & gets stuck at the same point in the boot sequence. Unfortunately we don’t have the machine on btrfs…
But we found a solution, we have the same problem as reported here: https://forums.opensuse.org/showthread.php/520200-Boot-hangs-after-recent-update?p=2794767
the workaround suggested worked for us as well. Thanks a lot for the replies! Much appreciated…
