efi redundancy - cannot complete boot when /boot/efi is missing

English Install/Boot/Login
1

Hi,
I’m experimenting with LEAP 15.2 to find the optimal solution for a complete redundancy through the use of 2 disks and software raid. I’ve a fully redundant layout in my LEAP 42.3 installation and I’m trying to reproduce the same layout in 15.2. Theory says two approach are possible: efi on a raid1 partition or two efi partitions manually aligned each time it is necessary. In LEAP 42.3 only the first approach was successfully, the second one suffers of a bug that make it useless (https://bugzilla.suse.com/show_bug.cgi?id=1059169)

In LEAP 15.2 i tried to reproduce the same strategy I adopted for LEAP 42.3 but I encountered a problem that I’m still investigating (https://bugzilla.suse.com/show_bug.cgi?id=1179981, https://forums.opensuse.org/showthread.php/547946-grub2-install-error-with-efi-on-raid1-(error-at-boot-quot-Verification-requested-but-nobody-cares-quot-) , please note that i cannot enable secureboot in bios)
So while investigating the problem I’ve just mentioned, I started experimenting the second approach to see if it is now, in LEAP 15.2, a possible way to obtain efi redundancy.

Unfortunately I discovered that LEAP 15.2 suffer the same problem I’ve had with LEAP 42.3: when the partition mounted on /boot/efi is missing, due to a failed disk, the bios can start the boot using the second efi partition (which I called /boot/efi2) but the system (dracut I suppose) is not able to complete the boot and for some reason complaints about the missing /boot/efi even if it is no more needed to complete the boot.
This is the disk layout:


vda                        253:0    0   20G  0 disk  
├─vda1                     253:1    0  500M  0 part  /boot/efi
├─vda2                      253:2    0    1G  0  part                                                                                                   
│ └─md0                      9:0    0 1024M  0 raid1  /boot                                                                                            
├─vda3                     253:3    0    1G  0  part                                                                                                   
│ └─md1                      9:1    0 1024M  0  raid1                                                                                                  
│   └─cr_swap              254:4    0 1024M  0 crypt  [SWAP]                                                                                           
├─vda4                     253:4    0   15G  0  part                                                                                                   
│ └─md2                      9:2    0   15G  0  raid1                                                                                                  
│   ├─sysVG-rootLV         254:0    0   10G  0 lvm    /                                                                                                
│   ├─sysVG-privateLV      254:1    0  480M  0  lvm                                                                                                    
│   │ └─cr_sysVG-privateLV 254:5    0  478M  0 crypt  /private                                                                                         
│   ├─sysVG-varLV          254:2    0  1.4G  0 lvm    /var                                                                                             
│   └─sysVG-homeLV         254:3    0    1G  0 lvm    /home                                                                                            
└─vda5                     253:5    0    1G  0 part   /vm1                                                                                             
vdb                        253:16   0   20G  0  disk                                                                                                   
├─vdb1                     253:17   0  500M  0 part   /boot/efi2                                                                                       
├─vdb2                     253:18   0    1G  0  part                                                                                                   
│ └─md0                      9:0    0 1024M  0 raid1  /boot                                                                                            
├─vdb3                     253:19   0    1G  0  part                                                                                                   
│ └─md1                      9:1    0 1024M  0  raid1                                                                                                  
│   └─cr_swap              254:4    0 1024M  0 crypt  [SWAP]                                                                                           
├─vdb4                     253:20   0   15G  0  part                                                                                                   
│ └─md2                      9:2    0   15G  0  raid1                                                                                                  
│   ├─sysVG-rootLV         254:0    0   10G  0 lvm    /                                                                                                
│   ├─sysVG-privateLV      254:1    0  480M  0  lvm                                                                                                    
│   │ └─cr_sysVG-privateLV 254:5    0  478M  0 crypt /private
│   ├─sysVG-varLV          254:2    0  1.4G  0 lvm   /var
│   └─sysVG-homeLV         254:3    0    1G  0 lvm   /home
└─vdb5                     253:21   0    1G  0 part  /vm2
vdc                        253:32   0    1G  0 disk  
└─vdc1                     253:33   0 1023M  0 part

when I simulate a failure on disk2 everything is working fine, when I simulate a failure on disk1 dracut drops me into the emergency shell. Please note that both efi and efi2 mount point have the nofail option in fstab.

I opened a bug with all the details: https://bugzilla.suse.com/show_bug.cgi?id=1180383

Any ideas on how can i workaround this issue? Alternatively, are there different approach to obtain efi redundancy?

Thank you in advance!

2

I don’t have any experience with RAID.

I have used a system where the “/etc/fstab” entry for “/boot/efi” gave the “noauto” option, so that “/boot/efi” is never mounted. And that works fine unless there is a grub update that requires writing to that file system.

Presumably, when I had it setup that way there was no attempt of “dracut” to mount “/boot/efi”.

3

The issue is the that the grub point to vmlinuz and initrd that are on the missing drive and the one in /boot on the good drive also points to the missing drive.

The cure if needed is to boot a recovery disk of OpenSUSE and mount the drive on /mnt and mount /dev /proc /sys and /run onto /mnt - chroot to /mnt and run mkinitrd to point it to the existing drive.

LVM for mirroring has the same issue. I’ve had to recover many LVM systems where the boot drive has failed. Doesn’t hurt to have supported 1000’s of SLES machines. You learn a lot when a main system is down and you have to get it back up. Luckily I had some test systems to practice on and prove the recovery. You have to undo it when you replace and rebuild the mirror - it now points to the alternate drive - not the boot drive.

hint - mount -B /dev /mnt/dev

don’t forget to umount them when done.

4

Yes, dracut adds /boot/efi device as hard requirment in hostonly mode.

Please note that both efi and efi2 mount point have the nofail option in fstab.

Timeout happens inside of dracut own event loop which is completely unrelated to /etc/fstab content.

Any ideas on how can i workaround this issue?

Disable hostonly mode, creating host independent initrd. This will results in much larger size and may uncover different bugs. You may also try --no-hostonly-default-device option (or equivalent conf.d setting). This will probably have less impact.

Did you try to actually read and understand what was written?

5 
LLR1:/boot/efi/EFI/opensuse # cat grub.cfg
set btrfs_relative_path="yes"
search --fs-uuid --set=root 75837315-86e5-4e19-af88-621f330d2be5
set prefix=(${root})/boot/grub2
source "${prefix}/grub.cfg"
LLR1:/boot/efi/EFI/opensuse #

Yes, it has a hard coded UUID - that is where it fails when the primary drive is missing. - That in itself gets you the “emergency menu”.
You cannot have a mirror set with it different in each mirror.

This was the one flaw in Veritos (What LVM is based on). As far as I know that was never really solved.

I guess you could also change via recovery disk all the UUID’s of the secondary drive to be those of the primary drive and get it to work also. Then the missing drive is the secondary drive. That would require a file with the blkid’s save where you could access them - I know that info is somewhere in /etc/lvm - I don’t use LVM anymore so I cannot access that info.

6

Which tells you exactly the same as I told you - the reason is hardcode device for /boot/efi.

I opened a bug with all the details: 1180383 – Cannot boot on degraded raid with dual EFI partitions - dracut exits on emergency shell

Which is full duplicate of the first bug.

7

Thank you very much for your help! Please note that in my layout /boot and /boot/efi are two different filesystem! /boot is on a soft raid1 so it is always available. Only /boot/efi and /boot/efi2 are two separated vfat partitions. The problem is not in the grub pointing to the boot partition, but dracut that absolutely need /boot/efi even if it is no more necessary.

8

Thank you very much. When I was dealing with the same bug in LEAP 42.3, disabling hostonly mode did not worked. With hostonly mode disabled the system cannot complete the boot even with without any failed disk (1059169 – cannot boot on degraded raid - dracut exits on emergency shell (leap 42.3 after update)). I can try again the same approach to see what happens with LEAP 15.2. I’ll let you know of course.