how to re-install grub: chroot for encrypted lvm

English Install/Boot/Login
1

I was trying to install Tumbleweed alongside an opensuse Leap, which was on an encrypted LVM. I did not provide passwords of the encrypted LVM while installing tumbleweed. After installation, I can boot into tumbleweed, but no boot option is available for Leap.
I know that it’s possible to boot from a rescue disk and chroot into the previous encrypted lvm and then re-install grub, but I can’t remember the detailed procedure! Can you please guide me?

P.S: The Tumbleweed is installed on another LVM, but it’s not encrypted.

2

Start by trying the easy method.

1: Boot into Tumbleweed;
2: Open the encrypted LVM for your 15.1 system


cryptsetup luksOpen /dev/sdxy cr_lvm151  ## change "sdxy" to the appropriate device

do that as root, and provide the passphrase as requested. That “cr_lvm151” can be changed to something else, as long as there is no name conflict.

3: Update grub2 in Tumbleweed (again as root)


grub2-mkconfig -o /boot/grub2/grub.cfg

This will usually add a boot entry for 15.1 to your Tumbleweed boot menu.

If you really want to reinstall grub for 15.1, then I will need to know more about your system. Does it use UEFI booting, or legacy booting? And if legacy BIOS booting, then I will need to know where grub is to be installed.

I will note, however, that there is no perfect solution to this problem. If Tumbleweed controls the boot menu, then you will lose access to 15.1 whenever Tumbleweed updates grub (usually after every kernel update and sometimes when there isn’t a kernel update involved). If you reinstall grub for 15.1, then a future update for Tumbleweed may reinstall grub for Tumbleweed anyway, and you will be back where you started.

Here’s what I do: I add an entry to boot 15.1 to the Tumbleweed menu, using “/etc/grub.d/40_custom”. And I also add an entry to boot Tumbleweed to the 15.1 menu, using “/etc/grub.d/40_custom” (but this time in the 15.1 install). That way, whichever is in charge of booting, I can boot the other.

3

Thanks, but I knew about the easy method. What I really want is to do that from inside a live session. And yes, I would like to have the Leap as the main OS and in-charge. I will try to I have assigned /dev/sda1 as the boot partition. My boot partition is mounted on /boot/efi, so most likely it’s UEFI booting. Can you please check what is lacking from the following code? I got that from an ubuntu forum and very likely it needs some modifications to work properly on openSUSE.


modprobe dm-crypt 
cryptsetup luksOpen /dev/sda4 linux 
vgscan 
vgchange -a y [NAME] 
lvscan 
mkdir /media/linux 
mount /dev/[NAME]/root /media/linux 
mount -o bind /proc /media/linux/proc 
mount -o bind /dev /media/linux/dev 
mount -o bind /sys /media/linux/sys  
chroot /media/linux /bin/bash 
mount /dev/sda1 /boot 
grub2-install /dev/sda
4

You need to check that, because if affects how grub is installed.

Take a look at “/boot/grub2/x86_64-efi”. If if contains a bunch of files, then you are likely using UEFI booting. If that directory does not exist or is empty, then you are not using UEFI booting.

Maybe check that on both 15.1 and Tumbleweed.

Can you please check what is lacking from the following code? I got that from an ubuntu forum and very likely it needs some modifications to work properly on openSUSE.

Yes, it needs some changes. I normally use “/mnt” for mounting when doing a rescue. So I’ll give my suggestion using that, otherwise loosely based on your code.


cryptsetup luksOpen /dev/sda4 linux
### no need for the "vgscan" and "vgchange" -- these days that is automatic.  But try:
ls /dev/mapper  ## this should tell you if the lvm partitions are already mapped.
mount /dev/[NAME]/root  /mnt
mount --bind /dev /mnt/dev
mount --bind /proc /mnt/proc
mount --bind /sys /mnt/sys
### that should get things ready for chroot

At this point, it depends somewhat on what you are doing with UEFI. If both of your systems are using UEFI booting, then “/dev/sda1” is already mounted as “/boot/efi” on Tumbleweed. So you should be able to use:


mount --bind /boot/efi /mnt/boot/efi

That makes the EFI partition available to the system that you are rescuing.

Continuing


chroot /mnt  /bin/bash
mount -a   ### this completes any other needed mounts.  If you are using "btrfs", then this will mount the subvolumes
shim-install
exit

Normally, UEFI booting uses “shim-install”, which then calls “grub2-install” behind the scenes as needed. If you have disabled secure-boot, and don’t want to use “shim”, then you can replace the “shim-install” with “grub2-install” (no additional arguments needed).

Note that this all assumes UEFI booting. I can give the non-UEFI version if you want it. The most important thing missing from that ubuntu suggested code, is the “mount -a” for mounting “btrfs” subvolumes. If you are not using “btrfs” then it would probably be fine.

There’s actually another problem that you will face. When you update Tumbleweed, there’s a good chance that will put Tumbleweed back in charge of the booting. It won’t happen for every Tumbleweed update, but it will happen whenever “grub2” is updated or “shim” is updated, and perhaps for a few other updates.

I’ll post another reply shortly, on how to handle that in a UEFI situation.

5

With UEFI booting, the important files are in “/boot/efi/EFI/opensuse”.

That is probably the same directory shared between Tumbleweed and Leap 15.1.

What I do, is make a backup of the files.


cd /boot/efi/EFI/opensuse
mkdir tw 15_1

The subdirectory “tw” is for Tumbleweed boot files, and the subdirectory “15_1” is for the 15.1 boot files. I use “15_1” for the directory name, to avoid using any “.” in the name.
To make a backup:


cd /boot/efi/EFI/opensuse
cp *.*  tw/.   ## Do this if Tumbleweed controls booting.

### copy, instead, to 15_1, if 15_1 more recently installed booting.

Then, to switch which controls booting, just copy back from that backup directory. And it is probably sufficient to just copy back “grub.cfg” which is the file that contains the information on what system to boot.

6

Big thanks, nrickert! You solved my problem!

There were some minor issues and some questions that I would like to ask.

**1. **There was a minor error running the codes you shared. The following would raise an error along the lines, "mount point (/mnt/boot/efi) does not exist":

mount --bind /boot/efi /mnt/boot/efi

I guess this is minor and IIRC I got around it by first creating the path: “mkdir /mnt/boot/efi”.

**2. **I ran your code from inside a Live CD. After running it, I was able to boot into my good old Leap. There was also a menu entry for Tumbleweed, but it wouldn’t boot and would throw an error about “You have to load the kernel module first.” Not sure why.
I solved this one by booting into Leap and running Boot Loader (present in YAST) from inside there.

**3. **This one is important. In your first post you said:

Here’s what I do: I add an entry to boot 15.1 to the Tumbleweed menu, using “/etc/grub.d/40_custom”. And I also add an entry to boot Tumbleweed to the 15.1 menu, using “/etc/grub.d/40_custom” (but this time in the 15.1 install). That way, whichever is in charge of booting, I can boot the other.

Would you please expand on this? How exactly should I do that? I already have used the other method you talked about (cp . ./15_1), but this method seems to solve the problem for once and ever.

**4. **Why should this whole issue happen? Previously, I had tried dual booting openSUSE Leap with Debian and Ubuntu and there were no such problems at all. It seems this is related to both Tumbleweed and Leap using the same name “opensuse” in /boot/efi/EFI/. If so, shouldn’t there be a bug report or something to solve the problem fundamentally?

7

Glad to have helped. But now, I am not sure how.

The following would raise an error along the lines, “mount point (/mnt/boot/efi) does not exist”:

Well, okay.

That seems to contradict your comment (in post #3 above), that the boot partition is mounted at “/boot/efi”.

My guess, at this stage, is that you installed Leap 15.1 for legacy BIOS booting and you then installed Tumbleweed for UEFI booting. And your BIOS prefers UEFI if it has a choice.

I guess this is minor and IIRC I got around it by first creating the path: “mkdir /mnt/boot/efi”.

That was a good decision.

So now, I think you have converted 15.1 to use UEFI booting. But perhaps that leaves an inconsistency. It may be booting with UEFI, but parts of the system still think it is using legacy booting. That can be fixed, but for now I’ll just mention. I don’t want this reply to be too long.

**2. **I ran your code from inside a Live CD. After running it, I was able to boot into my good old Leap. There was also a menu entry for Tumbleweed, but it wouldn’t boot and would throw an error about “You have to load the kernel module first.” Not sure why.
I solved this one by booting into Leap and running Boot Loader (present in YAST) from inside there.

Okay. That may have already fixed the possible inconsistency that I mentioned.

**4. **Why should this whole issue happen? Previously, I had tried dual booting openSUSE Leap with Debian and Ubuntu and there were no such problems at all.

It happens because Leap 15.1 and Tumbleweed are both opensuse. So they both store their boot files in “/boot/efi/EFI/opensuse”. They use the same directory, and what one writes will overwrite the other.

When you use “ubuntu”, it uses a directory named “ubuntu”. And when you use “debian”, it uses a directory name “debian” (or maybe named “grub”). So you don’t have one overwriting the other.

It seems this is related to both Tumbleweed and Leap using the same name “opensuse” in /boot/efi/EFI/. If so, shouldn’t there be a bug report or something to solve the problem fundamentally?

There have been bug reports. It is far from clear what to do about it. You will actually run into the same problem if you install both “ubuntu” and “kubuntu”, or even if you install both “ubuntu” and “deepin” (because “deepin” uses the ubuntu booting, even though otherwise debian based).

It is possible to have different directory names – say “opensuse” for Leap 15.1 and “tumbleweed” for Tumbleweed. We can further discuss that below if you like. But it isn’t a perfect solution either.

On that “/etc/grub.d/40_custom”, here’s what I am currently using:


### Entry to boot openSUSE Tumbleweed on sdb2
menuentry "configfile for openSUSE Tumbleweed on /dev/sdb2"  {
        search --fs-uuid --set=bootdir b76c6bf4-772c-4419-8398-c5de5a538235
        configfile (${bootdir})/grub2/grub.cfg
}

Note that this is appended to the header information already in that file.

Note that I use a separate “/boot” for Tumbleweed, and the UUID given is for that “/boot” partition. If you are not using a separate “/boot”, then you need the UUID of the Tumbleweed root file system (“/dev/mapper/something”), and you will need to replace the “/grub2/grub.cfg” with “/boot/grub2/grub.cfg”.

8

God this is thorough and so informative! I appreciate your effort and thoughtful explanations.

I think you have converted 15.1 to use UEFI booting. But perhaps that leaves an inconsistency. It may be booting with UEFI, but parts of the system still think it is using legacy booting. That can be fixed, but for now I’ll just mention. I don’t want this reply to be too long.

Please explain if possible. I would like to know.

It is possible to have different directory names – say “opensuse” for Leap 15.1 and “tumbleweed” for Tumbleweed. We can further discuss that below if you like. But it isn’t a perfect solution either.

Yes, I would like to know more about this method and why it’s not perfect!

On that “/etc/grub.d/40_custom”, here’s what I am currently using:

This should be the best method all around, right?

Note that I use a separate “/boot” for Tumbleweed

What’s the reason for that? Should I consider having multiple boot partitions in the future?

9

As for the custom menu entries, I did as per your instructions from inside Leap and now I have a custom grub entry that can switch to Tumbleweed in the case of emergency. How can I do the same from inside Tumbleweed to have a custom menu entry for Leap? As I said, Leap is on an encrypted LVM while Tumbleweed was on a plain LVM. So Leap could easily see the UUID of Tumbleweed’s root logical volume. But Tumbleweed can’t see Leap’s logical volumes. What should I do?

10

Check the output from

grep LOADER_TYPE /etc/sysconfig/bootloader

If it shows as “grub2-efi” then the system knows you are using UEFI booting. If it shows “grub2”, then it is still set for legacy booting.

You can change that with Yast bootloader. You may also need to make sure that the EFI partition is mounted at “/boot/efi” before running Yast bootloader.

Yes, I would like to know more about this method and why it’s not perfect!

To use “tumbleweed” as the directory name (“/boot/efi/EFI/tumbleweed”), set

GRUB_DISTRIBUTOR="Tumbleweed"

in "/etc/default/grub, and then run Yast bootloader and maybe change something (change the timeout by 1 second) to force an update.

This is not perfect, because you still have to tell your BIOS which boot entry to use as default. And that may change with updates. I found it easier to just keep the “opensuse” directory for everything, and change the content of that directory (via the suggested backups).

This should be the best method all around, right?

I find it the best for me. If I always boot with Leap, and use the Tumbleweed entry from the standard Leap boot menu, then it will use the Tumbleweed kernel as of the time that Leap booting was last updated. And that might be wrong. It might give “You need to load the kernel” message. But, using the entry via “/etc/grub.d/40_custom”, I am essentually using the Tumbleweed boot menu called by a subroutine. And that will get the kernel right.

What’s the reason for that?

I use a separate “/boot” because I am using an encrypted LVM. If I don’t use a separate “/boot”, then I am prompted twice for the encryption key (once by “grub” and once by the kernel). But it’s a choice. If using “btrfs”, then it is better to not have a separate “/boot”. I’m using “ext4” at present.

11

Thank you sir!
Since I’m afraid you may have unwittingly omitted my last question at post #9, please let me repeat it here. Basically, how can I add a custom menu entry to Tumbleweed, given that Leap is on an encrypted LVM?

12

Here’s an entry that I am using in a similar situation:


### Entry to boot TW3 on sda-root3
menuentry "configfile for TW3 on sda-root3"  {
        set btrfs_relative_path="yes"
        cryptomount -u 7428e7b830da407ab4ec6b53ac372022
        search --fs-uuid --set=bootdir 0b34f9bd-2d71-408c-a104-617efe2ad70f
        configfile (${bootdir})/boot/grub2/grub.cfg
}

The UUID in the “cryptomount” line is the UUID for the encrypted LVM itself. In this case, it is the UUID for “/dev/sda5”. However, all of the hyphens have been removed from that UUID to get the string shown.

The UUID in the search line is the UUID for the root file system (which is a volume within the encrypted LVM).

I have no idea about the “btrfs_relative_path” part as I just copied that from elsewhere and I don’t use “btrfs”.

I’m not sure, but you might need:

GRUB_ENABLE_CRYPTODISK="y"

in “/etc/default/grub”. Actually, I think you only need that on your Leap 15.1 system, and it is probably already set there.

13

Unlike nrickert, I find GRUB_DISTRIBUTOR= to be the better solution. Both my UEFI PCs use this method, which is required for *buntu derivatives because every one of them uses the same GRUB_DISTRIBUTOR=, making every one of them overwrite at each kernel update. I have /boot/efi/EFI/opensusetw, /boot/efi/EFI/opensuse150 and /boot/efi/EFI/opensuse151 in addition to those for the lesser used distros.

As to “changes with updates”, I omit the ESP partition from all fstabs except the one I wish to be booting from. In fact, I purge Grub from most installations, if I even allow the installer to include a bootloader. Every installation does not require having its own bootloader. I’m actually booting from a hand-made custom.cfg with symlinks to the kernels and initrds via /etc/grub.d/06_custom (40_custom renamed).

Whatever grub2-mkconfig generates using os-prober is normally ignored by me, because my 06_custom entries are at the top of the menu. Using efibootmgr I’ve removed all entries from the UEFI firmware except for Tumbleweed’s. About the only time I boot from an auto-generated stanza is when I wish a prior kernel used.

Multiboot is as much art as science, many ways to fail, and to succeed.

14

So in my setup (dual-booting Leap and Tumbleweed), can I safely use “GRUB_DISTRIBUTOR=tumbleweed” parameter and then remove ESP from fstab of Tumbleweed? Is that all that needs to be done?

15

It can be anything you please, as long as the quote characters are in the right places. :wink:

…and then remove ESP from fstab of Tumbleweed?
Yes, as long as you won’t let yourself be bothered by an occasional harmless in fact error message that seems dire because the ESP is not available when desired by various update/installation scripts. It may be that I am not bothered by any due to having purged Grub from my other OS installations. It is reputedly true that Grub isn’t actually needed to be able to boot, due to UEFI having full boot capability all by itself - I’ve never tried testing it.

Is that all that needs to be done?
At some point you should ensure TW is bootable via Leap’s Grub’s boot menu. :slight_smile: You may wish to purge the TW entry from the UEFI NV RAM if there are entries for both it and Leap.

16

This does not work for me. The menu item that’s supposed to switch back to Leap will prompt for decryption and then just shows the same Tumbleweed grub menu items; just like a loop.

17

If I purge Grub from tumbleweed, would those errors disappear? If not, then I think I’m going with custom boot menu entries, because I can’t really get along with those errors! And yes, I have saw them in a previous installation.

Thanks for all the great info!

18

If you want to do that, then you must first update (or reinstall) grub before you remove the ESP from “fstab”. You could do that by running Yast bootloader and changing something (such as a 1-second change in timeout).

I disagree with mrmazda as to whether this is a good way of doing things. But now you get to decide which way to go.

19

That’s not quite true.

Yes, you can do without grub. But UEFI does not have “full boot capability”. Rather, UEFI has the ability to run EFI applications.

The kernel is built to look like an EFI application, and when called that way, it can load itself into memory and thus boot. But it isn’t just UEFI doing it – it is partly the EFI stub loader that is built into the kernel.

To illustrate, I have a system (a virtual machine) using 32-bit EFI. I have to use grub to boot that. The EFI stub loader in the kernel is for 64-bit EFI, and won’t run with 32-bit EFI. Presumably one could build a kernel with a 32-bit EFI stub loader, but no distro does that (as far as I know).

At some point you should ensure TW is bootable via Leap’s Grub’s boot menu. :slight_smile: You may wish to purge the TW entry from the UEFI NV RAM if there are entries for both it and Leap.

This is where my approach to booting works well. There is only one openSUSE boot entry in the NVRAM. I control what that entry boots by controlling the content of the directory “/boot/efi/EFI/opensuse”.

20

What it should do, is ask for decryption, and then show the boot menu from Leap.

If, instead, you are seeing the boot menu from Tumbleweed, then something went wrong. Unfortunately, grub doesn’t give good information on this. In case of error, it just jumps back to the beginning of its original menu.

Check those UUIDs carefully.

For the first of those, I use:

blkid /dev/sda5

where “/dev/sda5” is the device for the encrypted LVM. And then, as mentioned, I strip all hyphens from that UUID before putting in the “cryptodisk” line.

For the second, I need the encrypted LVM to be unlocked, so that a device file appears for that. It probably shows up as “/dev/mapper/system-root” and also as “/dev/system/root”, except the LVM name might be something other than “system”. Using “system” for the moment, you need:

blkid /dev/mapper/system-root

or, equivalently

blkid /dev/system/root

but change “system” to whatever name you are using (the installer default name is “system”). And this time, do not remove the hyphens.

And this all assumes that “/boot” is part of your root file system and not a separate partition.