Dracut emergency shell: cannot read log, journal or mount

My system boot into dracut emergency shell (I assume, it’s a bash shell, ver. 4.4).

It created this error log /run/initramfs/rdsosreport.txt

I now got two problems
1: I want to read the log
2: I want to copy the log to my USB stick
3: Read the journal

Read log:
The less command is not available in the emergency shell :frowning: How can I read the log pages one at a time? Using cat scroll fast to the end-line of course.

Mount:
If I try to mount I get this error:
2895.949491] FAT-fs (sdb1): codepage cp437 not found

Journal:
The provided journalctl in this emergency shell doesn’t include the pager, again it jump fast to the end-line. Any solution for this?

Obviously I don’t access this shell through SSH terminal and therefore I can’t use SHIFT-PgUp to scroll up. This is the VGA console.

Any help would be appreciated.

I believe there is either more or vi command.

 2895.949491] FAT-fs (sdb1): codepage cp437 not found

Try to mount as msdos explicitly (mount -t msdos …)

The provided journalctl in this emergency shell doesn’t include the pager

journalctl never “includes” any pager, it calls external command. If more is available you can use it.

If you have another system you could check which commands are available using lsinitrd.

No, it ain’t a Bash shell – your system can’t find the root ‘/’ partition when booting – <https://linuxtroubleshootblog.wordpress.com/2016/05/27/first-blog-post/&gt;.

[HR][/HR]Bottom line –

  • It seems that, somehow, some way, an entry in the GRUB2 configuration was changed – the entry which points to the partition where the “/” (root) filesystem is located …

Neither more or vi is available.

If I execute “help” is says “GNU bash, version 4.4.23…” so yes, I would say it’s a bash shell. But of course it’s not the same as the one on a “nornal” full booted openSUSE shell but I newer said or claimed that!

And it didn’t help to add “-t msdos” when mount…I get the error “unknown filesystem type ‘msdos’”

In my view it make the emergency shell use case very limited. One of the most important actions is to be able to read the error-log file which isn’t really possible in practice.
Do you agree and should I request a fix?

And to be able to mount would also be helpful.

I briefly tested Leap 15.2, 15.3 and Tumbleweed and all of them have less in initrd.

 2895.949491] FAT-fs (sdb1): codepage cp437 not found

While that is a problem, in general you will not even have vfat support. It is added only on UEFI systems due to ESP.

If you stop in initrd due to misconfiguration you should be able to mount one of your disks (or volumes or whatever) and copy rdsosreport there. Actually you should be able to load necessary kernel module (in this case nls_cp437) from root directly.

That said it is certainly not nice, and there are multiple issues here.

  1. vfat filesystem is not functional even when dracut decices to add it. This is actually upstream issue.

  2. Should dracut always include vfat suport to facilitate collecting troubleshooting info. Each configuration is different, someone may prefer networking support etc. You certainly can arrange for necessary modules being present. Whether this should be default is subject to discussion.

Strange! If I search for “less” in the initrd image I get this result:

localhost:~ # lsinitrd /boot/efi/EFI/kernels/openSUSE_15.3/initrd.img | grep less
-rw-r–r-- 1 root root 6200 Jul 16 09:54 lib/modules/5.3.18-59.16-default/kernel/drivers/input/ff-memless.ko.xz

Just using a “default” created image using dracut.

Could it have been injected manually in your initrd?

Can you boot the installation media?
If yes, please follow the openSUSE trouble shooting instructions –

Please stay on the subject which is: one cannot in practice read the log files in the emergency shell which is one of the most important thing to do in the shell.

Subject has nothing to do with booting problems…I got that fully under control.

And you didn’t answer how less ended up in you initrd image? Are you using a custom made initrd or default automatic created?

bor@leap15:~> sudo lsinitrd /boot/initrd-5.3.18-59.16-default | grep less
-rw-r--r--   1 root     root         6200 Jul 16 10:54 lib/modules/5.3.18-59.16-default/kernel/drivers/input/ff-memless.ko.xz
-rwxr-xr-x   1 root     root       168064 May 25  2018 usr/bin/less
bor@leap15:~> 

Could it have been injected manually in your initrd?

I have pretty much vanilla installation, no customization. Anyway, “less” is added by base dracut module since the very beginning. The only reason it could be missing is if it is not found when initrd is being created (“less” is installed as optional).

Here also with a default Leap 15.3 installation – no changes to the Kernel or initrd – no modifications to the Kernel or initrd –


 # lsinitrd /boot/initrd-5.3.18-59.16-default | grep -i 'less'
-rw-r--r--   1 root     root         6200 May 10 16:57 lib/modules/5.3.18-59.16-default/kernel/drivers/input/ff-memless.ko.xz
-rwxr-xr-x   1 root     root       168064 May 25  2018 usr/bin/less
 # 
 # uname -a
Linux xxx 5.3.18-59.16-default #1 SMP Thu Jul 15 11:28:57 UTC 2021 (0b62bdb) x86_64 x86_64 x86_64 GNU/Linux
 # 
 # find /boot/ -iname '*initrd*'
/boot/initrd-5.3.18-59.16-default
/boot/initrd-5.3.18-59.13-default
/boot/initrd
/boot/initrd-5.3.18-59.10-default
 # 
 # l --almost-all /boot/efi/
insgesamt 40
-rwxr-xr-x 1 root root 6322  3. Okt 2020  db*
-rwxr-xr-x 1 root root 3724  3. Okt 2020  dbx*
drwxr-xr-x 4 root root 8192 18. Aug 2020  EFI/
-rwxr-xr-x 1 root root 3573  3. Okt 2020  KEK*
-rwxr-xr-x 1 root root  886  3. Okt 2020  PK*
 # l --almost-all /boot/efi/EFI/
insgesamt 16
drwxr-xr-x 2 root root 8192 28. Aug 2020  boot/
drwxr-xr-x 3 root root 8192 24. Aug 2020  opensuse/
 # l --almost-all /boot/efi/EFI/*/
/boot/efi/EFI/boot/:
insgesamt 1840
-rwxr-xr-x 1 root root 939800 14. Jul 17:34 bootx64.efi*
-rwxr-xr-x 1 root root  86352 14. Jul 17:34 fallback.efi*
-rwxr-xr-x 1 root root 846240 14. Jul 17:34 MokManager.efi*

/boot/efi/EFI/opensuse/:
insgesamt 3184
-rwxr-xr-x 1 root root      58 14. Jul 17:34 boot.csv*
drwxr-xr-x 2 root root    8192 24. Aug 2020  fw/
-rwxr-xr-x 1 root root   63744 24. Aug 2020  fwupdx64.efi*
-rwxr-xr-x 1 root root     125 14. Jul 17:34 grub.cfg*
-rwxr-xr-x 1 root root 1222656 14. Jul 17:34 grub.efi*
-rwxr-xr-x 1 root root  143360 14. Jul 17:34 grubx64.efi*
-rwxr-xr-x 1 root root  846240 14. Jul 17:34 MokManager.efi*
-rwxr-xr-x 1 root root  939800 14. Jul 17:34 shim.efi*
 # 

But, you have an “initrd.img” located in “/boot/efi/EFI/kernels/openSUSE_15.3/” –

  • Are you building your own Kernels and/or initrd?

How to debug Dracut problems

https://fedoraproject.org/wiki/How_to_debug_Dracut_problems#Using_the_dracut_shell

It doesn’t show that less is included in the initrd-image so it will be available in the emergency shell

In regards to my initrd location:
No, not at all. I’m just using the boot-manager systemd-boot instead of grub.
systemd-boot is a lot simpler and you have full control :slight_smile:
With grub you are in the hands of an wizard-script and it’s very difficult to have 100% control (well, at least that’s the case for me, I’m sure there are some out there who actually have full understanding of how grub works).
The only down-side of systemd-boot is it can only read the initrd from the efi partition whereas grub is able to read it from your root drive direcly.
But you can place it wherever want in the efi partition. You specify the path in the systemd-boot config file.

An example of the systemd-boot config file:


title      "SUSE_15x"
linux      "/EFI/kernels/SUSE_15x/vmlinuz-5.3.18-59.10-default"
initrd     "/EFI/kernels/SUSE_15x/initrd-5.3.18-59.10-default"
options    root=UUID=7e9215e3-f469-4d72-bf2a-2e4c795a228b *(and put your additional kernel options here)*

(this example is from another system)

A thing of beauty…compare that with the grub config file, yark!

When your initrd-image is rebuild e.g. at a kernel update you have to copy the initrd to the efi partition (but I have a script for that, so no hazle)

In my old grub days I often ended up with a non-booting system and was totally lost if the chroot + grub-wizard-regenerate-script didn’t work. Which happen rather often.
With systemd-boot I can ALWAYS get my system booted by modifying the “options” line in the systemd-boot config file.

This guide doesn’t seem to show you how to actually read log file FROM the emergency shell! And that’s because you can’t as the necessary tools are not available in the emergency shell e.g. less or more only cat. Try to read a 2000 lines file with cat!
And that’s where your need it most, when your system will not boot.

Oh yes, indeed, now things are beginning to be clear – and, there was this post – <https://forums.opensuse.org/showthread.php/542959-ALL-Systemd-boot&gt;.

I hope that, you noticed the following in the systemd-boot man (7) page –

Note that unless configured otherwise in the UEFI firmware, systemd-boot will use the US keyboard layout, so key labels might not match for keys like +/-.

Given that, SUSE and openSUSE are both extremely quiet with respect to “systemd-boot”, I’m beginning to wonder if, an error has crept in with Leap 15.3 …

Beginning with kernel 2.6.13, the initrd has been replaced by the initramfs (initial RAM file system), which does not require a file system driver to be mounted. openSUSE Leap exclusively uses an initramfs.

When I say initrd it is implied that I’m actually talking about initramfs, which “all” distro has switched too. I will claim that initrd doesn’t really exist anymore in practice, only as an alias for **initramfs
**
After a second test my first accusation about “less” was missing from the dracut emergency shell (the initrd image) was wrong.
“less” does seem to be added automatically by dracut and therefore available in the Emergency shell :-/

systemd-boot work perfectly with suse/openSUSE. I’m sure it’s not updated automatically by yast or when you update the kernel with zypper, but I wouldn’t expect it would.
Just copy the new initrd image to the EFI partition at kernel updates yourself.

Yes, and, “lsinitrd” is –

tool to show the contents of an initramfs image

  • Looks as if, the maintainers have retained the “older” tool name – to confuse people like me … :shame: