Boot fail after adding a new device to encrypted btrfs root filesystem

Hello, guys!

So, I want to add new a device to the existing encrypted btrfs root filesystem (Tumbleweed) to expand my storage. The new device is a SD card formatted as LUKS2 encrypted btrfs, just the same as the root filesystem. After configuring /etc/sysconfig/fde-tools and /etc/crypttab following this thread and other instructions, I can now decrypt root and new device at boot time. After booting into system, now I have /dev/mapper/cr_root and /dev/mapper/cr_sdcard (the new device).

However, after I added the new device to /, the next boot would fail and boot into grub command line. I can boot again if I remove the device from Live CD.

To add the new device, I use the following commands:

sudo btrfs add -f /dev/mapper/cr_sdcard /
sudo btrfs balance start /
sudo fdectl regenerate-key
sudo dracut --force
sudo update-bootloader --refresh

Here is my /etc/sysconfig/fde-tools and /etc/crypttab:

/etc/crypttab
cr_root    UUID=804b9a28-118a-4b3f-97c7-f20665d4af20  /.root.key  x-initrd.attach
cr_sdcard  UUID=f05cf65d-4dfe-4456-acd6-95b5c247fed2  /.root.key  x-initrd.attach
/etc/sysconfig/fde-tools
# Tunables for full disk encryption

# Configure whether to use old-style PCR policies, or TPMv2 authorized policies.
# Set to yes/no
FDE_USE_AUTHORIZED_POLICIES=yes

# List of PCRs to seal the LUKS key to
FDE_SEAL_PCR_LIST=0,2,4,7,9
FDE_SEAL_PCR_BANK=sha256

# It appears that 128 is the maximum size of what TPM2_Load is willing to handle
FDE_KEY_SIZE_BYTES=128

# These need to match exactly what grub2 uses to create the SRK
FDE_TPM2_SRK_ATTRS="userwithauth|restricted|decrypt|fixedtpm|fixedparent|noda|sensitivedataorigin"

# The PBKDF algorithm to use for deriving LUKS keys from a given password
# For grub2 based schemes, you have to use pbkdf2 for now.
FDE_LUKS_PBKDF="pbkdf2"

# Enable/disable tracing output
FDE_TRACING=true

# This is used by the installer to inform "fdectl tpm-enable" about a key
# to enroll on the next reboot
FDE_ENROLL_NEW_KEY=""

# Specify the devices to be managed by fdectl
# NOTE: Those devices must use the same recovery password.
FDE_DEVS="/dev/disk/by-uuid/804b9a28-118a-4b3f-97c7-f20665d4af20 /dev/disk/by-uuid/f05cf65d-4dfe-4456-acd6-95b5c247fed2"

# Configure whether to update the authorized policy in the sealed key after
# the bootloader update
# Set to yes/no
FDE_TPM_AUTO_UPDATE="yes"
FDE_AUTHORIZED_POLICY="authorized-policy"

Show the content of /boot/efi/EFI/opensuse/grub.cfg.

Thank you in advance!

/boot/efi/EFI/opensuse/grub.cfg
set btrfs_relative_path="yes"
tpm_record_pcrs 0-9
tpm2_key_protector_init -T $prefix/sealed.tpm
if ! cryptomount -u 804b9a28118a4b3f97c7f20665d4af20 --protector tpm2; then
    cryptomount -u 804b9a28118a4b3f97c7f20665d4af20
fi
search --fs-uuid --set=root 92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
set prefix=(${root})/boot/grub2
source "${prefix}/grub.cfg"

As is obvious, only one device is unlocked.

This only rebuilds /boot/grub2/grub.cfg. You need to reinstall bootloader to update grub.cfg on ESP. It needs to be done afer adding the second device (so grub can detect it).

update-bootloader --reinit

This time, after adding the new device to the root partition and balancing, I did the following:

sudo fdectl regenerate-key
sudo dracut --force
sudo update-bootloader --reinit
sudo update-bootloader --refresh

And indeed, /boot/efi/EFI/opensuse/grub.cfg showed the second device!

/boot/efi/EFI/opensuse/grub.cfg
set btrfs_relative_path="yes"
tpm_record_pcrs 0-9
tpm2_key_protector_init -T $prefix/sealed.tpm
if ! cryptomount -u 804b9a28118a4b3f97c7f20665d4af20 --protector tpm2; then
    cryptomount -u 804b9a28118a4b3f97c7f20665d4af20
fi
tpm2_key_protector_init -T $prefix/sealed.tpm
if ! cryptomount -u f05cf65d4dfe4456acd695b5c247fed2 --protector tpm2; then
    cryptomount -u f05cf65d4dfe4456acd695b5c247fed2
fi
search --fs-uuid --set=root 92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
set prefix=(${root})/boot/grub2
source "${prefix}/grub.cfg"

So did /boot/grub2/grub.cfg:

/boot/grub2/grub.cfg
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub2-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
set btrfs_relative_path="y"
export btrfs_relative_path
if [ -f ${config_directory}/grubenv ]; then
  load_env -f ${config_directory}/grubenv
elif [ -s $prefix/grubenv ]; then
  load_env
fi

if [ "${env_block}" ] ; then
  set env_block="(${root})${env_block}"
  export env_block
  load_env -f "${env_block}"
fi

if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   if [ "${env_block}" ] ; then
     save_env -f "${env_block}" next_entry
   fi
   set boot_once=true
else
   set default="${saved_entry}"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    if [ "${env_block}" ] ; then
      save_env -f "${env_block}" saved_entry
    else
      save_env saved_entry
    fi

  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

if [ x$feature_default_font_path = xy ] ; then
   font=unicode
else
insmod part_gpt
insmod part_gpt
insmod cryptodisk
insmod luks2
insmod gcry_rijndael
insmod gcry_rijndael
insmod gcry_sha256
insmod cryptodisk
insmod luks2
insmod gcry_rijndael
insmod gcry_rijndael
insmod gcry_sha256
insmod btrfs
cryptomount -u 804b9a28118a4b3f97c7f20665d4af20
cryptomount -u f05cf65d4dfe4456acd695b5c247fed2
set root='cryptouuid/804b9a28118a4b3f97c7f20665d4af20
cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root --hint='cryptouuid/804b9a28118a4b3f97c7f20665d4af20'  --hint='cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'  92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
else
  search --no-floppy --fs-uuid --set=root 92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
fi
    font="/usr/share/grub2/unicode.pf2"
fi

if loadfont $font ; then
  if [ "${grub_platform}" = "efi" ]; then
    echo "Please press 't' to show the boot menu on this console"
  fi

  set gfxmode=auto
  load_video
  insmod gfxterm
  set locale_dir=$prefix/locale
  set lang=en_US
  insmod gettext
fi
terminal_input console

for i in gfxterm; do
  if [ x${use_append} = xtrue ]; then
     terminal_output --append $i
  elif terminal_output $i; then
     use_append=true;
  fi
done

insmod part_gpt
insmod part_gpt
insmod cryptodisk
insmod luks2
insmod gcry_rijndael
insmod gcry_rijndael
insmod gcry_sha256
insmod cryptodisk
insmod luks2
insmod gcry_rijndael
insmod gcry_rijndael
insmod gcry_sha256
insmod btrfs
cryptomount -u 804b9a28118a4b3f97c7f20665d4af20
cryptomount -u f05cf65d4dfe4456acd695b5c247fed2
set root='cryptouuid/804b9a28118a4b3f97c7f20665d4af20
cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root --hint='cryptouuid/804b9a28118a4b3f97c7f20665d4af20'  --hint='cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'  92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
else
  search --no-floppy --fs-uuid --set=root 92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
fi
insmod gfxmenu
loadfont ($root)/boot/grub2/themes/openSUSE/ascii.pf2
loadfont ($root)/boot/grub2/themes/openSUSE/DejaVuSans10.pf2
loadfont ($root)/boot/grub2/themes/openSUSE/DejaVuSans12.pf2
loadfont ($root)/boot/grub2/themes/openSUSE/DejaVuSans-Bold14.pf2
insmod png
set theme=($root)/boot/grub2/themes/openSUSE/theme.txt
export theme
if [ x${boot_once} = xtrue ]; then
  set timeout=0
elif [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=2
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=2
fi
if [ -n "$extra_cmdline" ]; then
  menuentry "Help on bootable snapshot #$snapshot_num" {
    echo "Select the default entry of the snapshot boot menu."
    echo "Examine the snapshot, and if it's OK,"
    echo "   run 'snapper rollback' and reboot."
    echo "See 'System Rollback by Booting from Snapshots'"
    echo "   in the manual for more information."
    echo "  ** Hit Any Key to return to boot menu **  "
    read
  }
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/00_tuned ###
set tuned_params=""
set tuned_initrd=""
### END /etc/grub.d/00_tuned ###

### BEGIN /etc/grub.d/05_crypttab ###
### END /etc/grub.d/05_crypttab ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'openSUSE Tumbleweed'  --class opensuse --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a' {
	load_video
	set gfxpayload=keep
	insmod gzio
	insmod part_gpt
	insmod part_gpt
	insmod cryptodisk
	insmod luks2
	insmod gcry_rijndael
	insmod gcry_rijndael
	insmod gcry_sha256
	insmod cryptodisk
	insmod luks2
	insmod gcry_rijndael
	insmod gcry_rijndael
	insmod gcry_sha256
	insmod btrfs
	cryptomount -u 804b9a28118a4b3f97c7f20665d4af20
	cryptomount -u f05cf65d4dfe4456acd695b5c247fed2
	set root='cryptouuid/804b9a28118a4b3f97c7f20665d4af20
	cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'
	if [ x$feature_platform_search_hint = xy ]; then
	  search --no-floppy --fs-uuid --set=root --hint='cryptouuid/804b9a28118a4b3f97c7f20665d4af20'  --hint='cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'  92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
	else
	  search --no-floppy --fs-uuid --set=root 92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
	fi
	echo	'Loading Linux 6.7.2-1-default ...'
	linux	/boot/vmlinuz-6.7.2-1-default root=UUID=92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a  ${extra_cmdline} splash=silent quiet security=apparmor mitigations=auto
	echo	'Loading initial ramdisk ...'
	initrd	/boot/initrd-6.7.2-1-default
}
submenu 'Advanced options for openSUSE Tumbleweed' --hotkey=1 $menuentry_id_option 'gnulinux-advanced-92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a' {
	menuentry 'openSUSE Tumbleweed, with Linux 6.7.2-1-default' --hotkey=2 --class opensuse --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.7.2-1-default-advanced-92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a' {
		load_video
		set gfxpayload=keep
		insmod gzio
		insmod part_gpt
		insmod part_gpt
		insmod cryptodisk
		insmod luks2
		insmod gcry_rijndael
		insmod gcry_rijndael
		insmod gcry_sha256
		insmod cryptodisk
		insmod luks2
		insmod gcry_rijndael
		insmod gcry_rijndael
		insmod gcry_sha256
		insmod btrfs
		cryptomount -u 804b9a28118a4b3f97c7f20665d4af20
		cryptomount -u f05cf65d4dfe4456acd695b5c247fed2
		set root='cryptouuid/804b9a28118a4b3f97c7f20665d4af20
		cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'
		if [ x$feature_platform_search_hint = xy ]; then
		  search --no-floppy --fs-uuid --set=root --hint='cryptouuid/804b9a28118a4b3f97c7f20665d4af20'  --hint='cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'  92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
		else
		  search --no-floppy --fs-uuid --set=root 92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
		fi
		echo	'Loading Linux 6.7.2-1-default ...'
		linux	/boot/vmlinuz-6.7.2-1-default root=UUID=92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a  ${extra_cmdline} splash=silent quiet security=apparmor mitigations=auto
		echo	'Loading initial ramdisk ...'
		initrd	/boot/initrd-6.7.2-1-default
	}
	menuentry 'openSUSE Tumbleweed, with Linux 6.7.2-1-default (recovery mode)' --hotkey=3 --class opensuse --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.7.2-1-default-recovery-92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a' {
		load_video
		set gfxpayload=keep
		insmod gzio
		insmod part_gpt
		insmod part_gpt
		insmod cryptodisk
		insmod luks2
		insmod gcry_rijndael
		insmod gcry_rijndael
		insmod gcry_sha256
		insmod cryptodisk
		insmod luks2
		insmod gcry_rijndael
		insmod gcry_rijndael
		insmod gcry_sha256
		insmod btrfs
		cryptomount -u 804b9a28118a4b3f97c7f20665d4af20
		cryptomount -u f05cf65d4dfe4456acd695b5c247fed2
		set root='cryptouuid/804b9a28118a4b3f97c7f20665d4af20
		cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'
		if [ x$feature_platform_search_hint = xy ]; then
		  search --no-floppy --fs-uuid --set=root --hint='cryptouuid/804b9a28118a4b3f97c7f20665d4af20'  --hint='cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'  92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
		else
		  search --no-floppy --fs-uuid --set=root 92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
		fi
		echo	'Loading Linux 6.7.2-1-default ...'
		linux	/boot/vmlinuz-6.7.2-1-default root=UUID=92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a single  ${extra_cmdline}
		echo	'Loading initial ramdisk ...'
		initrd	/boot/initrd-6.7.2-1-default
	}
	menuentry 'openSUSE Tumbleweed, with Linux 6.7.1-2-default'  --class opensuse --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.7.1-2-default-advanced-92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a' {
		load_video
		set gfxpayload=keep
		insmod gzio
		insmod part_gpt
		insmod part_gpt
		insmod cryptodisk
		insmod luks2
		insmod gcry_rijndael
		insmod gcry_rijndael
		insmod gcry_sha256
		insmod cryptodisk
		insmod luks2
		insmod gcry_rijndael
		insmod gcry_rijndael
		insmod gcry_sha256
		insmod btrfs
		cryptomount -u 804b9a28118a4b3f97c7f20665d4af20
		cryptomount -u f05cf65d4dfe4456acd695b5c247fed2
		set root='cryptouuid/804b9a28118a4b3f97c7f20665d4af20
		cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'
		if [ x$feature_platform_search_hint = xy ]; then
		  search --no-floppy --fs-uuid --set=root --hint='cryptouuid/804b9a28118a4b3f97c7f20665d4af20'  --hint='cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'  92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
		else
		  search --no-floppy --fs-uuid --set=root 92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
		fi
		echo	'Loading Linux 6.7.1-2-default ...'
		linux	/boot/vmlinuz-6.7.1-2-default root=UUID=92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a  ${extra_cmdline} splash=silent quiet security=apparmor mitigations=auto
		echo	'Loading initial ramdisk ...'
		initrd	/boot/initrd-6.7.1-2-default
	}
	menuentry 'openSUSE Tumbleweed, with Linux 6.7.1-2-default (recovery mode)' --hotkey=1 --class opensuse --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.7.1-2-default-recovery-92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a' {
		load_video
		set gfxpayload=keep
		insmod gzio
		insmod part_gpt
		insmod part_gpt
		insmod cryptodisk
		insmod luks2
		insmod gcry_rijndael
		insmod gcry_rijndael
		insmod gcry_sha256
		insmod cryptodisk
		insmod luks2
		insmod gcry_rijndael
		insmod gcry_rijndael
		insmod gcry_sha256
		insmod btrfs
		cryptomount -u 804b9a28118a4b3f97c7f20665d4af20
		cryptomount -u f05cf65d4dfe4456acd695b5c247fed2
		set root='cryptouuid/804b9a28118a4b3f97c7f20665d4af20
		cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'
		if [ x$feature_platform_search_hint = xy ]; then
		  search --no-floppy --fs-uuid --set=root --hint='cryptouuid/804b9a28118a4b3f97c7f20665d4af20'  --hint='cryptouuid/f05cf65d4dfe4456acd695b5c247fed2'  92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
		else
		  search --no-floppy --fs-uuid --set=root 92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a
		fi
		echo	'Loading Linux 6.7.1-2-default ...'
		linux	/boot/vmlinuz-6.7.1-2-default root=UUID=92dc4c9a-fef1-4b23-9bd7-32e2453c3d2a single  ${extra_cmdline}
		echo	'Loading initial ramdisk ...'
		initrd	/boot/initrd-6.7.1-2-default
	}
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###

### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/20_memtest86+ ###
### END /etc/grub.d/20_memtest86+ ###

### BEGIN /etc/grub.d/25_bli ###
if [ "$grub_platform" = "efi" ]; then
  insmod bli
fi
### END /etc/grub.d/25_bli ###

### BEGIN /etc/grub.d/30_os-prober ###
### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/30_uefi-firmware ###
if [ "$grub_platform" = "efi" ]; then
	menuentry 'UEFI Firmware Settings' $menuentry_id_option 'uefi-firmware' {
                fwsetup --is-supported
                if [ "$?" = 0 ]; then
                        fwsetup
                else
                        echo "Your firmware doesn't support setup menu entry from a boot loader"
                        echo "Press any key to return ..."
                        read
                fi
        }
fi
### END /etc/grub.d/30_uefi-firmware ###

### BEGIN /etc/grub.d/35_fwupd ###
### END /etc/grub.d/35_fwupd ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg
fi
### END /etc/grub.d/41_custom ###

### BEGIN /etc/grub.d/80_suse_btrfs_snapshot ###
btrfs-mount-subvol ($root) /.snapshots @/.snapshots
if [ -f "/.snapshots/grub-snapshot.cfg" ]; then
  source "/.snapshots/grub-snapshot.cfg"
fi
### END /etc/grub.d/80_suse_btrfs_snapshot ###

### BEGIN /etc/grub.d/90_persistent ###
### END /etc/grub.d/90_persistent ###

### BEGIN /etc/grub.d/95_textmode ###
if [ "${grub_platform}" = "efi" ]; then
  # On EFI systems we can only have graphics *or* serial, so allow the user
  # to switch between the two
  hiddenentry 'Text mode' --hotkey 't' {
    set textmode=true
    terminal_output console
  }
fi
### END /etc/grub.d/95_textmode ###

Strangely and sadly, the problem remained. Once I rebooted, the system again booted into grub command line.

In grub CLI do

set pager=1
set

and post photos of the screen with output (you will likely have two pages).

The output from grub command line looks like this.

grub fails to detect btrfs filesystem. Show

cryptsetup luksDump /dev/disk/by-uuid/804b9a28-118a-4b3f-97c7-f20665d4af20
cryptsetup luksDump /dev/disk/by-uuid/f05cf65d-4dfe-4456-acd6-95b5c247fed2

These commands showed these information:

804b9a28
LUKS header information
Version:       	2
Epoch:         	122
Metadata area: 	16384 [bytes]
Keyslots area: 	16744448 [bytes]
UUID:          	804b9a28-118a-4b3f-97c7-f20665d4af20
Label:         	(no label)
Subsystem:     	(no subsystem)
Flags:       	(no flags)

Data segments:
  0: crypt
	offset: 16777216 [bytes]
	length: (whole device)
	cipher: aes-xts-plain64
	sector: 512 [bytes]

Keyslots:
  0: luks2
	Key:        512 bits
	Priority:   normal
	Cipher:     aes-xts-plain64
	Cipher key: 512 bits
	PBKDF:      pbkdf2
	Hash:       sha256
	Iterations: 1579180
	Salt:       67 15 d0 1a 25 05 74 7b 96 f5 a8 58 c2 cf 35 48 
	            16 ec 76 cb ca 89 c9 8f 2d ca 15 6b d0 54 ba 3c 
	AF stripes: 4000
	AF hash:    sha256
	Area offset:32768 [bytes]
	Area length:258048 [bytes]
	Digest ID:  0
  2: luks2
	Key:        512 bits
	Priority:   normal
	Cipher:     aes-xts-plain64
	Cipher key: 512 bits
	PBKDF:      argon2id
	Time cost:  4
	Memory:     1048576
	Threads:    4
	Salt:       65 09 67 e5 27 08 a7 fd e2 2d 0c 5e e6 b6 04 8b 
	            3a f2 10 3e 41 76 8a fa f5 87 0f 89 ed 4b fe d7 
	AF stripes: 4000
	AF hash:    sha256
	Area offset:548864 [bytes]
	Area length:258048 [bytes]
	Digest ID:  0
  3: luks2
	Key:        512 bits
	Priority:   normal
	Cipher:     aes-xts-plain64
	Cipher key: 512 bits
	PBKDF:      pbkdf2
	Hash:       sha256
	Iterations: 1000
	Salt:       38 ae 96 bc ec b7 d9 ba 57 aa 14 b2 26 d5 b7 8c 
	            22 d9 a4 23 3a 3c d4 05 f4 e4 b0 e2 09 1d a0 b3 
	AF stripes: 4000
	AF hash:    sha256
	Area offset:806912 [bytes]
	Area length:258048 [bytes]
	Digest ID:  0
Tokens:
  1: grub-tpm2
	timestamp:  2024-02-06 12:00:54 UTC
	Keyslot:    3
Digests:
  0: pbkdf2
	Hash:       sha256
	Iterations: 98996
	Salt:       fb 14 a9 08 6c 16 bc 95 4a 72 75 0c 1c 20 69 1e 
	            0b 78 c5 52 2e cb 21 ec 3f 2d c1 87 87 0f a0 3c 
	Digest:     c8 3a 93 9a 82 17 d1 79 e7 cc 7e 0f 69 8f ba 8b 
	            b3 9c 79 ef df 53 8c 88 b7 b3 02 b6 0a 33 de d7 
f05cf65d
LUKS header information
Version:       	2
Epoch:         	111
Metadata area: 	16384 [bytes]
Keyslots area: 	2064384 [bytes]
UUID:          	f05cf65d-4dfe-4456-acd6-95b5c247fed2
Label:         	(no label)
Subsystem:     	(no subsystem)
Flags:       	(no flags)

Data segments:
  0: crypt
	offset: 2097152 [bytes]
	length: (whole device)
	cipher: aes-xts-plain64
	sector: 512 [bytes]

Keyslots:
  0: luks2
	Key:        512 bits
	Priority:   normal
	Cipher:     aes-xts-plain64
	Cipher key: 512 bits
	PBKDF:      pbkdf2
	Hash:       sha256
	Iterations: 1558062
	Salt:       13 77 51 7c 1c d6 3d 15 60 57 67 db 98 aa 2b 24 
	            d4 05 7f cf 51 bb d7 a1 0e 6a e0 5c 63 45 01 48 
	AF stripes: 4000
	AF hash:    sha256
	Area offset:32768 [bytes]
	Area length:258048 [bytes]
	Digest ID:  0
  2: luks2
	Key:        512 bits
	Priority:   normal
	Cipher:     aes-xts-plain64
	Cipher key: 512 bits
	PBKDF:      argon2id
	Time cost:  5
	Memory:     1048576
	Threads:    4
	Salt:       74 95 18 35 34 76 9f a6 9e ce 55 b1 30 05 ea 9f 
	            b9 4f 4f 4f b0 32 8a 70 68 a2 40 4e 8c 27 3f 3c 
	AF stripes: 4000
	AF hash:    sha256
	Area offset:548864 [bytes]
	Area length:258048 [bytes]
	Digest ID:  0
  3: luks2
	Key:        512 bits
	Priority:   normal
	Cipher:     aes-xts-plain64
	Cipher key: 512 bits
	PBKDF:      pbkdf2
	Hash:       sha256
	Iterations: 1000
	Salt:       58 d9 a6 45 43 94 15 eb f3 cd 9c 7d 07 bc 76 37 
	            1a 1c 8f 14 7f f4 27 af d8 ec f3 00 bb a2 8a c1 
	AF stripes: 4000
	AF hash:    sha256
	Area offset:806912 [bytes]
	Area length:258048 [bytes]
	Digest ID:  0
Tokens:
  1: grub-tpm2
	timestamp:  2024-02-06 12:00:57 UTC
	Keyslot:    3
Digests:
  0: pbkdf2
	Hash:       sha256
	Iterations: 97234
	Salt:       e4 01 22 7b de 9f 5e 18 1b 01 6c 6f d4 93 1d fc 
	            fe d0 e6 10 09 2e 07 ce cf 7d 83 e8 cf 64 ac a3 
	Digest:     ff 7b 03 e7 7a 20 80 73 8f 22 db 64 ad 25 49 47 
	            aa b3 e8 c9 

What is the output of efibootmgr?

Here is the output of efibootmgr when I have removed the second device:

efibootmgr
BootCurrent: 0000
Timeout: 0 seconds
BootOrder: 0000,0007,0006,0008
Boot0000* opensuse-secureboot	HD(1,GPT,b8fcc8a0-fc63-4192-b3a4-1bf8c777e3e8,0x800,0x100000)/File(\EFI\opensuse\shim.efi)
Boot0001* MsTemp	PciRoot(0x0)/Pci(0x14,0x0)/USB(12,0)/HD(2,GPT,1a354f0f-5968-fc6a-3787-47c9ee3a592a,0xea4ffd8,0x10000)
Boot0006* Internal Storage	FvVol(a881d567-6cb0-4eee-8435-2e72d33e45b5)/FvFile(50670071-478f-4be7-ad13-8754f379c62f)53444400
Boot0007* USB Storage	FvVol(a881d567-6cb0-4eee-8435-2e72d33e45b5)/FvFile(50670071-478f-4be7-ad13-8754f379c62f)55534200
Boot0008* PXE Network	FvVol(a881d567-6cb0-4eee-8435-2e72d33e45b5)/FvFile(50670071-478f-4be7-ad13-8754f379c62f)50584500

After adding the second device and doing update-bootloader operations, the output didn’t seem to change.

How do you boot? It shows the current boot openSUSE, but it stops in grub. What do you do to boot the system after it stopped?

Oh, I can’t manage to boot into the system when it shows grub cli.

When the system fails to boot after adding the new device, I boot into a Live CD and following this instruction to chroot to root filesystem and remove the new added device there. After that I reinstall bootloader and update grub config, then I can boot to my system again (on its original device).

So I can’t really figure out what happens when the system boots into grub cli.

Did you wait for the balance to complete (now stabbing in the dark)?

Yes.

I’ve tried sudo btrfs balance start / and sudo systemctl start btrfs-balance.service on different runs. I know the latter one will calculate usage and apply filters accordingly.

The results are the same though: boot fail.

You said it is SD card. Is it visible in UEFI at all? What shows ls in grub CLI?

ls in grub cli shows:
(crypto0) (memdisk) (proc) (hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd2)

Thank you for all your efforts.

Is it after adding the second device to your filesystem and reinstalling bootloader?

Yes, it is.

Can you manually unlock the second encrypted device in GRUB CLI?

cryptomount -u f05cf65d4dfe4456acd695b5c247fed2

You should be asked for the password.