Repair a broken UEFI/GRUB2/openSUSE boot scenario

English Forums Feedback
1

I would like to share the line of action i use to investigate and (hopefully) repair broken UEFI boot scenarios (It may even serve to manually install GRUB2 for UEFI boot).

Please be aware:

  • I use openSUSE 42.3 on UEFI-based machines with GRUB2 as bootloader.
  • “secureboot” is always disabled.
  • The UEFI is set to boot in UEFI-mode (no “Compatibility Support Mode (CSM)” aka “Legacy mode” or “MBR mode”).
  • All disks have a GPT partition scheme and ext4 as file system.
  • RAID or LVM are not used.
  • There are no separat boot partition or any encrypted system/swap partition(s).

If your setup is different the following might probably be of no use to you.

The UEFI specification offers some freedom to vendors on how to implement an UEFI (and there even might be faulty UEFI implementations). So some UEFI specific stuff mentioned below may not work with your UEFI (it might be worth to check your motherboard vendors website for the latest UEFI version).

Before you start:
Your system may become completely unusable if you follow the guidelines below! Make sure you have a working and up to date backup of all your data.

When your system does not boot any more the openSUSE-Installation-media (DVD/USB stick) can be used to repair the broken boot scenario:

  1. Make sure your UEFI is set up to boot in UEFI-mode.
  2. Make sure “secureboot” and “FastBoot” are disabled in your UEFI.
  3. Boot the openSUSE installation media [01] (DVD/USB stick).
    Make sure it really started in UEFI mode; i.e at the bottom of the very first screen there is NO options menu (F1, F2, …) displayed. If there is an options menu the media booted in CSM (legacy mode / MBR mode).
  4. Select “More …” and then “Rescue system”. This will give you a working command-line-only system.
  5. Log in as “root” (no password needed).
  6. It’s high time to make sure you have a backup of your data.
# od -An -t u1 /sys/firmware/efi/vars/SecureBoot-8be4df61-93ca-11d2-aa0d-00e098032b8c/data
0
#

This will check the “secureboot”-status of your UEFI. If the result is not equal “0” then you still need to disable “secureboot” in your UEFI (or your UEFI returns invalid values). In case the command fails completely make sure you booted in UEFI-mode.

  1. The UEFI spec allows for more than one UEFI system partition. If you plan to (re)install GRUB2 you can use any EFI system partition available on your system (as long as it has enough free space to hold the openSUSE GRUB2 bootloader files) or - if you have some spare disk space available (min. 128 MB; better 500 MB) - you may even create a new EFI system partition (Type 0xEF00; file system FAT32).
# blkid | grep EFI

Will help you to find the EFI system partition(s) available on your system. If you want to use a specific one (e.g. the one that was used in a once working boot scenario) and are not sure about which it is you can inspect each in order to determine which is the one you are looking for.

  1. Mount the EFI system partitions one by one. Use “blkid” to get the proper value(s) for “sdxz”.
# mount /dev/sdxz /mnt

If you already had a working boot scenario you can now look for a file “/mnt/efi/opensuse/grubx64.efi”. If you can’t find that file check the other EFI system partitions. If there aren’t any other EFI system partitions you need to reinstall GRUB2 as described later on.

  1. “grubx64.efi” is the booatloader which needs to know where to find the system it is supposed to boot. This knowledge will either be build into “grubx64.efi” when it gets installed or there is a file “/mnt/efi/opensuse/grub.cfg”.
# cat /mnt/efi/opensuse/grub.cfg
search --fs-uuid --set=root a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6
set prefix=(${root})/boot/grub2
configfile $prefix/grub.cfg

Make sure that “a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6” fits the UUID of the partition which hosts the root file system of the system you want to boot. Use “blkid” to get the UUID of your openSUSE root file system partition.

If “grubx64.efi” was build to boot from a partition that no longer exists (or you just don’t want to boot from any longer) then you need to reinstall GRUB2 as described later on.

# umount /mnt

Finaly unmount the EFI system partition you inspected.

  1. Setup a changeroot environment.
# mount /dev/sdxy /mnt

Mounts your openSUSE root file system partition. Use “blkid” to get the proper value for “sdxy”.

# mount /dev/sdxz /mnt/boot/efi

Mounts your EFI system partition. Use “blkid” to get the proper value for “sdxz”.

# mount -o bind /dev /mnt/dev
# mount -o bind /run /mnt/run
# mount -o bind /sys /mnt/sys
# mount -o bind /proc /mnt/proc
# chroot /mnt

Completes and starts the changeroot environment. From here on all actions take place in the already installed openSUSE system.

# zypper se -si grub2

Check that GRUB2 is installed (that should be the case if there once was a working boot scenario). If no grub2-packages are listed GRUB2 needs to be installed (a working internet connection is needed for this!)

# zypper in grub2

# zypper up grub2

If GRUB2 is already installed this makes sure you have the latest version of the grub2-packages (a working internet connection is needed for this!)

  1. Check the GRUB2 configuration file
# cat /etc/default/grub | grep "GRUB_USE_LINUXEFI"
GRUB_USE_LINUXEFI="true"
# cat /etc/default/grub | grep "GRUB_DISABLE_OS_PROBER"
GRUB_DISABLE_OS_PROBER="false"

If you do not see the results shown above use an editor (e.g. vi) to change the file “/etc/default/grub”

You may also want to check the “GRUB_CMDLINE_LINUX_DEFAULT=” and “GRUB_CMDLINE_LINUX=” settings in that file. If there is something like “resume=/dev/disk/by-uuid/a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6” make sure that “/dev/disk/by-uuid/a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6” really exists and is the partition you want to use for resume.

# cat /etc/fstab | grep boot
UUID=1234-ABCD   /boot/efi  vfat umask=0002,utf8=true  0  0

Make sure that “1234-ABCD” fits the UUID of the EFI system partition you want to use. If there is no entry with “/boot/efi” the EFI system partition will not be mounted at system startup so you need to add an entry for the EFI system partition to “/etc/fstab”.

# efibootmgr -v
BootCurrent: 0003
Timeout: 2 seconds
BootOrder: 0003,0005,0001,0000
Boot0000* Win...
Boot0003* opensuse    HD(5,GPT,a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6,0xacbe800,0x200000)/File(\EFI\OPENSUSE\GRUBX64.EFI)..BO
Boot0005* Ubu...

This will check the UEFIs NVRAM. If there is no entry (Bootnnnn) for opensuse you need to create one (replace “sdx” with the device that carries the EFI system partition you want to use; in “–part n” replace “n” with the number of this EFI system partition)

# efibootmgr --create --disk /dev/sdx --part n --label "openSUSE" --loader \\EFI\\opensuse\\grubx64.efi

# blkid

will help you to find the correct value for “/dev/sdx” and

# gdisk -l /dev/sdx

will help you to determine “n”

# efibootmgr -b 0004 -B

Changes the bootsequence stored in the UEFIs NVRAM (e.g. make entry “Boot0004” to be the first one in the “BootOrder”)

  1. If you could not fix the problem so far you can reinstall GRUB2
# grub2-install --target=x86_64-efi

and create a new GRUB2 boot menu

# grub2-mkconfig -o /boot/grub2/grub.cfg
  1. Leave the changeroot environment
# exit
  1. Reboot your system
# init 6

Hopefully your system will boot again.

Last but not least a few things worth knowing about UEFI:

  • EFI firmware contains knowledge about the partition structure of various devices and can understand legacy MBR, GPT, and “El Torito”. An EFI system partition supports backward compatibility with legacy systems.
  • The file system (for EFI system partitions) supported by EFI is based on the FAT file system and includes support for long file names. The EFI firmware must support the FAT32, FAT16, and FAT12 variants of the EFI file system. EFI encompasses the use of FAT32 for a EFI system partition and FAT12 or FAT16 for removable media.
  • FAT defines that all files in a directory must have a unique name and unique is defined as a case insensitive match.
  • An EFI system partition that is present on a hard disk must contain in the root directory a directory named EFI. All OS loaders and applications will be stored in subdirectories below EFI. The choice of the subdirectory name is up to the vendor, but all vendors must pick names that do not collide with any other vendor’s subdirectory name. This applies to system manufacturers, operating system vendors, BIOS vendors, and third party tool vendors, or any other vendor that wishes to install files on an EFI system partition. There must also only be one executable EFI image for each supported processor architecture in each vendor subdirectory. If more than one executable EFI image is present, then the boot behavior for the system will not be deterministic. There may also be an optional vendor subdirectory called BOOT. This directory contains EFI images that aide in recovery if the boot selections for the software installed on the EFI system partition are ever lost.
  • The UEFI boot sequence looks like this:

The boot order list (shown as “BootOrder” by “efibootmgr -v”) is read from the UEFI NVRAM. It holds a list of NVRAM variables (shown as “Boot0000” to “BootFFFF” by “efibootmgr -v”) that contain information about what is to be booted. Modifications to this variable are only guaranteed to take effect after the next platform reset.
Each NVRAM variable (“Boot0000” to “BootFFFF”) contains a pointer to the hardware device and to a file on that hardware device that contains the UEFI image to be loaded. Also it defines a name for the boot option that can be displayed to a user. Furthermore the variable might contain paths to the OS partition and directory along with other configuration specific directories. The UEFI will try to boot the “Bootnnnn” entries in the order given by “BootOrder” until a “Bootnnnn” can be booted successfully. If no “Bootnnnn” could be booted successfully (or no “Bootnnnn” was defined) then the UEFI will try to boot /EFI/BOOT/BOOT.efi".

Some helpful links:

[01] openSUSE install media
http://download.opensuse.org/distribution/openSUSE-current/iso/
[02] GRUB2 Manual
https://www.gnu.org/software/grub/manual/grub/
[03] UEFI specification
http://www.uefi.org/specifications

Regards

susejunky

2

Your advice is mostly good. But I’ll add some comments.

I have been leaving “secure-boot” enabled. OpenSUSE can handle that without a problem, though that was broken when first installing Leap 42.1. But it is back to working well. However, some other linux versions do not support secure-boot.

As far as I know, openSUSE, Ubuntu and Fedora are the main distributions that do support secure-boot. Linux Mint mostly supports it, because it is based on Ubuntu.

On one of my computers, there are two UEFI boot entries for “ubuntu”. So it might be false that system names have to be unique.

You mention “fastboot” (and suggest turning if off). I agree that it should be turned off. But “fastboot” usually refers to something in Windows, rather than something in the UEFI firmware.

1 Like
3

Thank you very much for your feedback.

I’m aware that openSUSE can handle secure boot but i never tried it. However at the moment i have no idea how to manually install (using the Rescue System and no yast) shim/MokManager.

Please, could you provide a brief description? I’m very intrested to learn how to do it (and i would like to enhance my checklist).

The descriptions at the very bottom of my post are taken from the UEFI specification (Chapter 13.3.1.3). As far as i understood only the directories below “/EFI” must have unique names. The names in the NVRAM boot entries (“Boot0000”-“BootFFFF”) have not to be unique. But i’m no native english speaker so i can be wrong.

The UEFI of my desktop motherboard (MSI Z97 Gaming 3) does have an option “FastBoot”. Because i thought it to be “the” FastBoot option i have turned it of. But probably that was not necessary?

Regards

susejunky

4

Okay. I’ll do a separate reply about secure-boot (following this reply).

As far as i understood only the directories below “/EFI” must have unique names. The names in the NVRAM boot entries (“Boot0000”-“BootFFFF”) have not to be unique.

That’s probably correct. The file system (FAT) already requires unique directory names. And it appears to be allowable (but confusing) to have two UEFI boot entries with the same name.

The UEFI of my desktop motherboard (MSI Z97 Gaming 3) does have an option “FastBoot”.

Fair enough. The term “fastboot” in ambiguous. I have seen a “fastboot” BIOS option going back many years. Windows introduced a “fastboot” option for Win 8 and later, and made that the default.

It is the “fastboot” of Windows that is the main concern to linux users. It leaves the Windows file system in an unstable state. So that windows “fastboot” is best disabled.

The “fastboot” in the BIOS or UEFI firmware is a different thing. Normally, when the computer boots, it goes through POST (Power On Self Test). The BIOS “fastboot” shortens this POST. It does a quick memory check and skips the more thorough memory check. It might skip initializing USB devices, unless you are booting from a USB. Unless you are having hardware issues, it is probably okay to leave the BIOS “fastboot” turned on.

1 Like
5

Notes on secure-boot in UEFI systems.

Most UEFI computers come with secure-boot. You can enable or disable that in the BIOS.

What is secure-boot?

With secure-boot, your UEFI firmware comes with a certificate (public-key) for checking signatures. Usually, the public key is from Microsoft, though it could be from your computer vendor. The bare computer might come without a key, but that key is installed when the first software is installed.

The idea is to check everything. So the boot files are supposed to all be digitally signed. And then the signature is checked. If the signature check fails, the system refused to boot.

However, it is not that simple. Since the check is for a Microsoft signature, the files have to be digitally signed at the factory. However, some of the files needed for booting are specific to your computer, so cannot be signed. In Windows, the BCD database (boot configuration data) is not signed. With linux/grub, the grub menu is not signed, and the “initrd” file is not signed. So the security checks of secure-boot are not complete. Secure-boot gives some limited protection, but it is not complete protection.

How does linux handle secure-boot?

For most linux distros, linux needs you to disable secure-boot in the UEFI firmware (BIOS). Some distros do support secure-boot, and openSUSE is one of those.

When you boot your computer, the UEFI firmware runs the system loader. For openSUSE with secure-boot support, that system loader is the file “shim.efi”. This file is signed by Microsoft, so it passes the secure-boot test.

The file “shim.efi” contains a public key for verifying signatures made by openSUSE (instead of by Microsoft). So “shim.efi” check other boot files with that opensuse signature. The first such file is “grub.efi”, which contains the grub boot code. This file is signed by openSUSE. The grub boot code next loads the kernel at what is called the “EFI stub”. And it asks “shim.efi” to verify the signature on the kernel. Thereafter, booting proceeds as it would without secure-boot.

Installing secure-boot support

On a UEFI box, the default for openSUSE is to install secure-boot support. During install, if you go to the booting section of the installer, there is a check box for “secure-boot”. And that is checked by default, though you can turn it off.

It should be harmless to install secure-boot support. If you disable secure-boot in your BIOS, it should still work. The effect of disable secure-boot in the BIOS is that the digital signatures are not checked. But everything else should still work the same.

If you did not originally install secure-boot support (you unchecked that box during install), then it is fairly easy to install later. If the system is already up and running, just use Yast bootloader, and check that box. If you are running the rescue-system, then you must mount disks as needed for going into a “chroot” session. Then, in that “chroot” session, you will need to run

shim-install

Note that you must boot into UEFI mode to do this. You can successfully run “shim-install” even if secure-boot is disabled. But you cannot run it if you booted into MBR mode instead of UEFI mode. (Well, you can run it, but it will give an error).

Whether running the rescue system, or using Yast, I suggest that you check “/etc/default/grub” to make sure that it contains the line:

GRUB_USE_LINUXEFI="true"

Otherwise secure-boot won’t work.

And if you had to change that, then run


grub2-mkconfig -o /boot/grub2/grub.cfg

If you are using the rescue system, then run that from the “chroot” session.

I should note that if you are trying to boot a 32-bit kernel, that won’t work. So don’t even try using secure-boot if you need to be able to boot a 32-bit kernel.

Is secure-boot worth the trouble?

That’s for you to decide. I normally leave it turned on. But, in all honesty, I don’t think it adds any useful security. I leave it turned on as a way of testing whether it works (and reporting bugs when it doesn’t work).

6

Thank you very much for your description on “secureboot”! I will definitely give it a try.

All i had found on “secureboot” so far looked very complicated to me (generating keys and installing them in the UEFI and the like). Because i was afraid to mess up my system i never tried that.

Regards

susejunky

7

IMHO Secure boot is security theater. If a bad actor can modify the boot stack then they already own the machine. At best it is marginal protection but the protection is to brick the machine if things do not add up. At least you have to clean and reinstall the complete boot stack and all certificates.

8

That’s only needed if you want to sign a kernel yourself – only needed if you are compiling kernels yourself.

9

For an inexperience linux user, whose new computer has secure-boot enabled, the openSUSE support means that he can install openSUSE without having to mess with UEFI settings.

10

MY point is the whole concept is smoke an mirrors and saves you from very very few things. Just something forced on us by Microsoft of dubious value

11

Hi,

That was a great info i definitely had this bookmarked. My only suggestion i repeat just a suggestion is avoid doing

cat file | grep pattern

because grep does not need cat since it can parse files by itself.

grep --help | head -n 4

the output would be something like below.

Usage: grep [OPTION]... PATTERN [FILE]...
Search for PATTERN in each FILE or standard input.
PATTERN is, by default, a basic regular expression (BRE).
Example: grep -i 'hello world' menu.h main.c

ie.

grep boot /etc/fstab

grep -E '^(GRUB_USE_LINUXEFI|GRUB_DISABLE_OS_PROBER)' /etc/default/grub

or

grep -E '^(GRUB_)USE_LINUXEFI|DISABLE_OS_PROBER' /etc/default/grub
12

I have been leaving “secure-boot” enabled. OpenSUSE can handle that without a problem, though that was broken when first installing Leap 42.1. But it is back to working well. However, some other linux versions do not support secure-boot.

13

Thanks for your great instructions!

Although, I found when having a btrfs root filesystem it needs another mount in order to let grub2-mkconfig succeed, see below.

After this command the “Boot0004” entry is no longer listed. Is this expected?
And just for my unterstanding, how do you know that “0004” is the correct entry?

And when doing

I get the error

Found openSUSE Tumbleweed on /dev/sdb6
ERROR: failed to lookup root id: Inappropriate ioctl for device

To fix this it is required to do

# mount /.snapshots/

(according to https://www.reddit.com/r/openSUSE/comments/7y7l6m/tw_updating_bootloader_failed/)

Could you please add this to your instructions?

14

Thank you very much for reading and testing my post.

As i mentioned at the start of my post i use “ext4” not “btrfs” and therefor do not know if my instructions will work when “btrfs” is used.

The command

# efibootmgr -b 0004 -B

delets a boot entry from the UEFIs NVRAM.

I must admit my description here is misleading. Deleting an entry will indeed change the bootorder however that may not be the real reason to use this command.

The command

# efibootmgr -o XXXX,YYYY,ZZZZ

will change the boot order to XXXX first, YYYY second and ZZZZ third.

You can check the contents of the UEFIs NVRAM with

# efibootmgr -v
BootCurrent: 0003
Timeout: 2 seconds
BootOrder: 0003,0005,0001,0000
Boot0000* Win...
Boot0003* opensuse    HD(5,GPT,a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6,0xacbe800,0x200000)/File(\EFI\OPENSUSE\GRUBX64.EFI)..BO
Boot0005* Ubu...

This tells you the current boot order (0003,0005,0001,0000) and what each boot entry points to (e.g. 0003 will start openSUSE). So you can delete a specific entry or just rearrange the boot order using the command above.

Sorry! I definitely would like to do so. But in this forum there is now way to change an existing post that is more than few minutes old. One must read a thread to the very end to gather all its information.

Regards

susejunky

15

Hello.
Very useful for me.
I just use it.
Concerning the use which I have just had, I have to make some remarks on the points which required a reflection from my part.
POST #1 - N°9 : better to use insensitive pattern. In my case label was in small characters.

blkid | grep -i EFI

POST #1 - N°10 : For people who runs into problem with a dual boot opensuse - windows ( like me ) after mounting the efi partition :

mount  /dev/sdxy  /tmp
ls /tmp
'$RECYCLE.BIN'  EFI  'System Volume Information'

This is surely a window efi partition.
but

ls /tmp
EFI

This is probably a linux efi partition.

POST #1 - N°11

Make sure that “a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6” fits the UUID of the partition which hosts the root file system of the system you want to boot. Use “blkid” to get the UUID of your openSUSE root file system partition.

do that and If partition still exists you get :

blkid | grep "a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6" 
/dev/sdvw: LABEL="SOME_LABEL" UUID="a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6" TYPE="ext4" PARTLABEL="primary" PARTUUID="bc325cec-438b-4e15-bd27-a6d8807de242"

else

blkid | grep "a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6"

POST #1 - N°13 -a

# mount /dev/sd**xy** /mnt

Mounts your openSUSE root file system partition. Use “blkid” to get the UUID…

More simpler : Mounts your openSUSE root file system partition using the proper value for “sdxy” found at step 11

POST #1 - N°13 -b

# mount /dev/sd**xz** /mnt/boot/efi

Mounts your EFI system partition. Use “blkid” to get the proper value for “sdxz”.

[/QUOTE]
More simpler : Mounts your openSUSE efi partition using the proper value for “sdxz” found and/or choosen at step 10

POST #1 - N°13 -c
The command cannot work because there is no “boot/efi” folder in /tmp.
Just mount in /tmp.

# mount /dev/sd**xz** /mnt

This point is more sophisticated and should be infirm or confirm by opensuse guru :The current system have a physical /boot partition which is different of the root file system partition.
or

The current system do not have a physical /boot partition. Then the /boot folder is inside the root file system partition and the installer mount the efi partition in “/boot/efi”.
This does not make difference because after mounting the 2 partitions ( /dev/sdxy and /dev/sdxz ) every folders are in the right place.

POST #1 - N°14 -15
Of course you need a working internet conection.

POST #1 - N°18

This will check the UEFIs NVRAM. If there is no entry (Bootnnnn) for opensuse you need to create one (replace “sdx” with the device that carries the EFI system partition you want to use; …)

replace “sdx” … : More simpler : use the proper value for “sdxz” found and/or choosen at step 10

in “–part n” replace “n” with the number of this EFI system partition

For me the numbers use by efibootmgr was 0001 and 0002.
I choose 0003.
'efibootmgr -v’ show then 0000 0001 0002 ? ? ?

blkid

will help you to find the correct value for “/dev/sdx”

More simpler : use the proper value for “sdx” found and/or choosen at step 10 for efi partition “sdxz”

And thanks again for your great job.

16

I appreciate the documenting of this recovery process. However I would like to point out that in certain circumstances there can be an easier way which works for me.

Occasionally when updating my TW and Leap systems I see an error saying that there is no room to update the UEFI settings and executing efibootmgr -v shows no boot order set, this in my case is a f/w error on the mb.

When booting the USB stick in UEFI mode there is an option to boot from hard drive, if you select this and it works then go into yast and set up grub and in each case I have done that it has worked.May be worth a try initially before having to resort to a recovery boot up from USB.

Stuart

17

Thank you very much for reading and testing my post.

I agree! Before one goes through all the steps i described it’s definitely worth to try what you recommend.

Regards

susejunky

18

Your thread is great stuff. I postponed switching to UEFI for several years. When I switched several months ago a tutorial like the above would have been great help.