No boot after update of encrypted system

I’ve installed Leap 42.3 on two relatively new systems. Both have the same problem. During installation, unencrypted /boot/efi and /boot partitions and an encrypted volume containing / and swap are created on a gpt disk. After installation, the kernel is 4.4.67. At this point, the system works. It will reboot, present the prompt to decrypt the disk, present the login prompt and all is well. Also after installation, an update to zypper and numerous subsequent updates are waiting. These take the kernel to 4.4.90. After that, it loses the ability to decrypt the LVM volume. It never presents the disk decryption password prompt. It simply spins through startup until it finally errors out because the root partition is unavailable. Dropping back to the older kernel at boot time does not fix the problem. Whatever has changed through the updates affects it also.

It does not matter if I disable UEFI and create a new MBR partition table on the disk. The installation boots and installs to MBR but the behavior is the same. The systems do need firmware updates for their Intel processors, but applying these separately before update does not help.

I have an older system without UEFI running Leap 42.2 installed in a similar way, with an encrypted volume. That one has been upgraded steadily and currently runs the 4.4.90 kernel successfully. Installing 42.2 on the newer systems does not help, however. They still to lose access to the encrypted volume after the first batch of updates.

I’ve looked through the startup messages, trying to find something helpful. I do see /dev/mapper/system-root presented as a kernel startup parameter. It just doesn’t work.

openSUSE is the only LInux distribution that meets all of my requirements for a working laptop. If I can’t make this work, I’m going to have to revert to Windows and work through all the problems that come with that. Any way I can avoid that will be most appreciated.

I’m not having that problem here.

Here’s something to try:

On the grub boot line, hit the ‘e’ key. That should allow you to edit the grub boot line.

Scroll down until you find a line that begins “linux” or “linuxefi”. That’s the line you will want to change.

Scroll to the right, until you find the text “splash=silent quiet”. The delete the “splash=silent”. It’s okay to also delete the “quiet”, but maybe leave that there for the first try.

Now hit CTRL-X to proceed with booting.

You will now see a lot of text on the screen. If it all works, then there should be a prompt for your encryption key. If there are related error messages, then write them down, and report back here.

That is a neat trick, editing the boot command on the fly. Thanks for that.

In any case, in quiet, but not silent mode, the system finds a disk with a short ID, my FAT-formatted /boot/efi partition, then finds a disk with a full UUID, my /boot partition, then hangs. It never finds my third, volume group partition. In full verbose mode, it even finds my unpartitioned second disk that I left out of the installation, but goes no farther.

Using a live CD of (ahem) another distribution, I can verify that the volume group partition is available. I can open it with cryptsetup, mount it and view and edit the contents.

I’ll make a wild guess. It’s probably wrong. But it might be worth checking.

If your encrypted LVM setup is similar to mine, then the entry in “/etc/crypttab” specifies the partition using “/dev/disk/by-id/some-funky-string”. And maybe something has changed in the rules that define that funky string. I had it happen to me a while ago for a USB drive (while running 42.1).

So here’s what I would suggest, if it is possible:

(1) edit “/etc/crypttab”, and in place of “/dev/disk/by-id/some-funky-string” you should put “UUID=some-uuid-string”. You can find the UUID of that partition with something like:

# blkid /dev/sda5

except use the device for your encrypted partition.

You should be able to do that step while booted to that live CD.

(2) This is the trickier step. You need to rebuild the “initrd” for that system.

You will have to boot to live media, and it will have to match the architecture (i.e. be 64bit). Best would be to use the 42.3 install media. The Tumbleweed rescue CD should also work. Maybe that live CD will work, but the Tumbleweed rescue CD or the 42.3 install DVD or NET install CD (booted to rescue mode) would be safer.

Once booted, you need to get to a command line (as root):

And then:

cryptsetup luksOpen /dev/xxx cr_lvm ### or something similar.  You've done that before.
ls /dev/mapper  ### check if your lvm volumes are available
vgchange -a y   ### only needed if lvm volumes are not yet visible
mount /dev/mapper/system-root  /mnt   #mount the root file system
mount /dev/sdax /mnt/boot  ### change "sdax" to what is appropriate
mount /dev/sday /mnt/boot/efi  ### mount the EFI partition, again change that "sdy" to what is appropriate.
### your system should now be mounted
mount --bind /dev  /mnt/dev
mount --bind /proc /mnt/proc
mount --bind /sys /mnt/sys
### next, chroot into the mounted system
chroot /mnt
mkinitrd  ### rebuild the initrd
exit    ### leave the chroot environment
umount -R /mnt   ### unmount everything.

Then reboot and see what happens

Scanning recent bug reports, I am noticing bug 1063249 Encrypted LVM password no longer requested after systemd update
That is possibly the same problem, particularly if you are using an “nvme” device. It might be worth scrolling through that report. And, incidentally, my suggestion of switching to UUID in “crypttab” would probably work in that case.

I had the same wild thought, and I tried it, but without mkinitrd. I hoped that using the same identifier in /etc/crypttab would allow it to go through. In the past, I’ve found that the system will pick up changes until the mapping name is changed, then it’s time to rebuild the boot. In any case, my approach did not fix the problem.

I think it will be best to change /etc/crypttab before applying updates and see if that helps. I’ll try that. I have never used the openSUSE rescue system, because I’ve never needed it.

I’ve now scanned the bug report also. Looks just like my condition. I do have an nvme device. I’ll try the workaround using UUID instead of /disk/by-id, but it looks like this one will get fixed soon in any case.

It won’t work without the “mkinitrd”. There a copy of “/etc/crypttab” in the “initrd”, and that is used while booting. The “mkinitrd” will update that to match your changes to the “cryptab” on the real root file system.

SOLVED. Thanks for your help, kind sir.

The problem occurs when an nvme device is used to host an encrypted partition containing root or a logical volume for root. It happens because the nvme device is not properly mapped by ID in /etc/crypttab.

The workaround:

  • Install openSUSE Leap 42.3, but DO NOT apply updates during installation.

  • After logging in to the installed system, DO NOT apply updates immediately.

  • In the installed result, obtain the UUID of the encrypted partition containing the root directory using

sudo blkid

In UEFI systems, the partition will likely be /dev/nvme0np3. In MBR systems, the partition will likely be /dev/nvme0np2.

  • Edit /etc/crypttab to replace the loading of the partition by ID, loading it instead by UUID. The line in /etc/crypttab will look something like:

crypt_name UUID=uuuuuuuu-vvvv-wwww-xxxx-yyyyyyyyyyyy none none

  • Run the following command to regenerate the booting kernel.

sudo mkinitrd

  • Now apply all pending updates and reboot the system.

I’m glad you have it working.