No boot on raid1 after change hard drive

catarino · July 13, 2023, 1:00pm

Hi there i need some heldf from the comunity

I had a server with raid1 software opensuse 15.5 btrsf file system

hdd1 → sda
hdd2 → sdb

the hdd2 disk was damage, so i change and i rebuild the raid

Then the system stays:
hdd1 → sda
ssd2 → sdb

Everything was woking 100%

I resolve to change the hdd1 to a ssd drive
I change the drive and rebuild the raid1, everything was syncronized to 100%

ssd1 → sda
ssd2 → sdb

after boot the system dont start, no boot device.

my system have this raid:
/dev/sda1 * boot partition
/dev/sda2 system

/dev/sdb1 * boot partition
/dev/sdb2 system

/dev/sda1 + /dev/sdb1 = md0 | /boot/efi

/dev/sdb2 + /dev/sdb2 = md127 | /srv /home /and so one…

I thried to mount through rescue cd opensuse 15.5 system the md0 and md127 with no sucess.
all monted folders are empty example /, /srv, /opt, /home, /boot/grub2/x86_64-efi, /boot/grub2/i386-pc,…

Any solution?

gogalthorp · July 13, 2023, 1:10pm

Legacy or EFI booting??

In either case the boot stuff is outside of the BTRFS RAIDED partitions thus not reproduced .

For Legacy (MBR) boot you need MBR CODE installed
For EFI boot you need a correctly provisioned FAT partition to hold the boot files and the BIOS needs to point to the correct place.

arvidjaar · July 13, 2023, 1:19pm

How exactly did you replace disks? Show full command sequence you did.

Educated guess is that your partitions on new disks do not match EFI boot entry (entries). Show complete output of

efibootmgr -v
blkid
fdisk -l

catarino · July 13, 2023, 1:32pm

I start with Mark the disk as failed
mdadm --manage /dev/md0 --fail /dev/sdb1
mdadm --manage /dev/md127 --fail /dev/sdb2

Remove the disk
mdadm --manage /dev/md0 --remove /dev/sdb1
mdadm --manage /dev/md127 --remove /dev/sdb2

Replace the disk
copy the partitions
sfdisk -d /dev/sda | sfdisk /dev/sdb

Create the mirror of the disk:
mdadm --manage /dev/md0 --add /dev/sdb1
mdadm --manage /dev/md127 --add /dev/sdb2

To check the process i made "cat /proc/mdstat "
and it reach 100%

For the first hdd i rebooted and i was ok.

Then i make the same process to the second hdd, it reach 100% but with no boot after reboot.

catarino · July 13, 2023, 1:37pm

I still have the working hdd hard drive and the result of the commands are:

efibootmgr -v
EFI variables are not supported on this system.

blkid
/dev/md127: UUID=“e5188064-8dcf-41f6-be18-246204930cb7” UUID_SUB=“4466deff-d5ab-4691-902c-95d2f47e2a01” BLOCK_SIZE=“4096” TYPE=“btrfs”
/dev/md0: SEC_TYPE=“msdos” UUID=“F454-C7BD” BLOCK_SIZE=“512” TYPE=“vfat”
/dev/sda2: UUID=“e1221ff4-fd8d-e8fa-9251-2f49016df0bd” UUID_SUB=“418fd794-0494-ce32-70aa-270a6af3d931” LABEL=“any:sistema” TYPE=“linux_raid_member” PARTUUID=“0006740a-02”
/dev/sda1: UUID=“de856a6a-c671-c08b-f579-53967ced17d8” UUID_SUB=“e266c65f-6f6b-ba7f-928b-67cfc42be1fd” LABEL=“any:0” TYPE=“linux_raid_member” PARTUUID=“0006740a-01”

fdisk -l
Disco /dev/sda: 931,51 GiB, 1000204886016 bytes, 1953525168 sectores
Modelo do disco: WDC WD10EARX-00N
Unidades: sectores de 1 * 512 = 512 bytes
Tamanho do sector (lógico/físico): 512 bytes/4096 bytes
Tamanho de E/S (mínimo/óptimo): 4096 bytes/4096 bytes
Tipo de rótulo de disco: dos
Identificador de disco: 0x0006740a

Dispositivo Arranque Início Fim Sectores Tamanho Id Tipo
/dev/sda1 * 2048 206847 204800 100M fd Linux raid auto-detectada
/dev/sda2 206848 1953525167 1953318320 931,4G fd Linux raid auto-detectada

Disco /dev/md127: 931,41 GiB, 1000098824192 bytes, 1953318016 sectores
Unidades: sectores de 1 * 512 = 512 bytes
Tamanho do sector (lógico/físico): 512 bytes/4096 bytes
Tamanho de E/S (mínimo/óptimo): 4096 bytes/4096 bytes

Disco /dev/md0: 99,94 MiB, 104792064 bytes, 204672 sectores
Unidades: sectores de 1 * 512 = 512 bytes
Tamanho do sector (lógico/físico): 512 bytes/4096 bytes
Tamanho de E/S (mínimo/óptimo): 4096 bytes/4096 bytes
Tipo de rótulo de disco: dos
Identificador de disco: 0x00000000

arvidjaar · July 13, 2023, 1:50pm

You were asked whether you were using legacy BIOS or EFI and you silently ignored this question. You have /boot/efi so I assumed you were using EFI. If you are using legacy BIOS, you simply need to reinstall bootloader to your boot device. Boot device is determined by your BIOS setup, most likely it was the first HDD (now SSD).

Show output of

LANG=C LC_ALL=C cat /etc/default/grub_installdevice
LANG=C LC_ALL=C grep -Ev '^$|^#' /etc/sysconfig/bootloder

Please paste computer text as preformatted (button </> in editor).

catarino · July 13, 2023, 2:01pm

I dont ignore, i will reboot and see the bios configuration, for now only remote acess thru ssh.

catarino · July 13, 2023, 2:05pm

I using the commands in the working old hdd . the two ssd are disconected trying to get one solution.

LANG=C LC_ALL=C cat /etc/default/grub_installdevice
/dev/disk/by-id/ata-WDC_WD10EARX-00N0YB0_WD-WMC0S0321217
/dev/disk/by-id/ata-WDC_WD10EARS-00Y5B1_WD-WCAV5N180031
activate

LANG=C LC_ALL=C grep -Ev ‘^$|^#’ /etc/sysconfig/bootloder
grep: /etc/sysconfig/bootloder: No such file or directory

Thank you for your time.

arvidjaar · July 13, 2023, 2:16pm

This is /etc/sysconfig/bootloader, sorry. And please, use preformatted text.

catarino · July 13, 2023, 2:28pm

cat bootloader

## Path:        System/Bootloader
## Description: Bootloader configuration
## Type:        list(grub,grub2,grub2-efi,none)
## Default:     grub2
#
# Type of bootloader in use.
# For making the change effect run bootloader configuration tool
# and configure newly selected bootloader
#
#
LOADER_TYPE="grub2"

## Path:        System/Bootloader
## Description: Bootloader configuration
## Type:        yesno
## Default:     "no"
#
# Enable UEFI Secure Boot support
# This setting is only relevant to UEFI which supports Secure Boot. It won't
# take effect on any other firmware type.
#
#
SECURE_BOOT="no"

## Path:        System/Bootloader
## Description: Bootloader configuration
## Type:        yesno
## Default:     "no"
#
# Enable Trusted Boot support
# Only available for legacy (non-UEFI) boot.
#
TRUSTED_BOOT="no"

arvidjaar · July 13, 2023, 2:35pm

OK, so you are definitely using legacy BIOS boot. You said you replaced disks one by one. Did you do it online, without powering off your system? Or did you boot from some other live distro to do it?

catarino · July 13, 2023, 3:36pm

Hi, always online.
I change the first, reboot several times and two oi three weeks later change the other one.

LANG=C LC_ALL=C cat /etc/default/grub_installdevice
/dev/disk/by-id/ata-WDC_WD10EARX-00N0YB0_WD-WMC0S0321217
/dev/disk/by-id/ata-WDC_WD10EARS-00Y5B1_WD-WCAV5N180031
activate

Looks like the sistem as the boot on the two hdd drives.
I change one hdd and it worked ok because the other hdd is listed above, but when i change the other one, the two are missing in the grub_installdevice.

Sorry my english

arvidjaar · July 13, 2023, 6:13pm

In this case add the second disk (SSD), let it rebuild the RAID and post output of

ls -l /dev/disk/by-id

catarino · July 14, 2023, 12:16pm

Hi,
Now i have 2 drives, one hdd and one ssd

ls -l /dev/disk/by-id

lrwxrwxrwx 1 root root  9 jul 14 13:10 ata-T-CREATE_T253TA001T_112101040070018 -> ../../sdb
lrwxrwxrwx 1 root root 10 jul 14 13:10 ata-T-CREATE_T253TA001T_112101040070018-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 jul 14 13:10 ata-T-CREATE_T253TA001T_112101040070018-part2 -> ../../sdb2
lrwxrwxrwx 1 root root  9 jul 14 13:10 ata-WDC_WD10EARX-00N0YB0_WD-WMC0S0321217 -> ../../sda
lrwxrwxrwx 1 root root 10 jul 14 13:10 ata-WDC_WD10EARX-00N0YB0_WD-WMC0S0321217-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 jul 14 13:10 ata-WDC_WD10EARX-00N0YB0_WD-WMC0S0321217-part2 -> ../../sda2
lrwxrwxrwx 1 root root  9 jul 14 13:10 md-name-any:0 -> ../../md0
lrwxrwxrwx 1 root root 11 jul 14 13:10 md-name-any:sistema -> ../../md127
lrwxrwxrwx 1 root root  9 jul 14 13:10 md-uuid-de856a6a:c671c08b:f5795396:7ced17d8 -> ../../md0
lrwxrwxrwx 1 root root 11 jul 14 13:10 md-uuid-e1221ff4:fd8de8fa:92512f49:016df0bd -> ../../md127
lrwxrwxrwx 1 root root  9 jul 14 13:10 scsi-0ATA_T-CREATE_T253TA0_112101040070018 -> ../../sdb
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-0ATA_T-CREATE_T253TA0_112101040070018-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-0ATA_T-CREATE_T253TA0_112101040070018-part2 -> ../../sdb2
lrwxrwxrwx 1 root root  9 jul 14 13:10 scsi-0ATA_WDC_WD10EARX-00N_WD-WMC0S0321217 -> ../../sda
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-0ATA_WDC_WD10EARX-00N_WD-WMC0S0321217-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-0ATA_WDC_WD10EARX-00N_WD-WMC0S0321217-part2 -> ../../sda2
lrwxrwxrwx 1 root root  9 jul 14 13:10 scsi-1ATA_T-CREATE_T253TA001T_112101040070018 -> ../../sdb
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-1ATA_T-CREATE_T253TA001T_112101040070018-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-1ATA_T-CREATE_T253TA001T_112101040070018-part2 -> ../../sdb2
lrwxrwxrwx 1 root root  9 jul 14 13:10 scsi-1ATA_WDC_WD10EARX-00N0YB0_WD-WMC0S0321217 -> ../../sda
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-1ATA_WDC_WD10EARX-00N0YB0_WD-WMC0S0321217-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-1ATA_WDC_WD10EARX-00N0YB0_WD-WMC0S0321217-part2 -> ../../sda2
lrwxrwxrwx 1 root root  9 jul 14 13:10 scsi-350014ee2069c35ee -> ../../sda
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-350014ee2069c35ee-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-350014ee2069c35ee-part2 -> ../../sda2
lrwxrwxrwx 1 root root  9 jul 14 13:10 scsi-SATA_T-CREATE_T253TA0_112101040070018 -> ../../sdb
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-SATA_T-CREATE_T253TA0_112101040070018-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-SATA_T-CREATE_T253TA0_112101040070018-part2 -> ../../sdb2
lrwxrwxrwx 1 root root  9 jul 14 13:10 scsi-SATA_WDC_WD10EARX-00N_WD-WMC0S0321217 -> ../../sda
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-SATA_WDC_WD10EARX-00N_WD-WMC0S0321217-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 jul 14 13:10 scsi-SATA_WDC_WD10EARX-00N_WD-WMC0S0321217-part2 -> ../../sda2
lrwxrwxrwx 1 root root  9 jul 14 13:10 wwn-0x50014ee2069c35ee -> ../../sda
lrwxrwxrwx 1 root root 10 jul 14 13:10 wwn-0x50014ee2069c35ee-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 jul 14 13:10 wwn-0x50014ee2069c35ee-part2 -> ../../sda2

The WD10EARX-00N_WD-WMC0S0321217 is hdd
The T-CREATE_T253TA0_112101040070018 is the ssd

cat /proc/mdstat

Personalities : [raid1]
md0 : active raid1 sda1[1] sdb1[2]
      102336 blocks super 1.0 [2/2] [UU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md127 : active raid1 sda2[1] sdb2[2]
      976659008 blocks super 1.0 [2/2] [UU]
      bitmap: 1/8 pages [4KB], 65536KB chunk

unused devices: <none>

The raid1 looks ok

 LANG=C LC_ALL=C cat /etc/default/grub_installdevice
/dev/disk/by-id/ata-WDC_WD10EARX-00N0YB0_WD-WMC0S0321217
/dev/disk/by-id/ata-WDC_WD10EARS-00Y5B1_WD-WCAV5N180031
activate

This is wrong shold be

 LANG=C LC_ALL=C cat /etc/default/grub_installdevice
/dev/disk/by-id/ata-WDC_WD10EARX-00N0YB0_WD-WMC0S0321217
**/dev/disk/by-id/ata-T-CREATE_T253TA001T_112101040070018**
activate

thx for your help

arvidjaar · July 14, 2023, 12:31pm

Correct. Now edit /etc/default/grub_installdevice, replace the old HDD device name with the new SSD device name and run

update-bootloader --reinit

It will reinstall grub on both disks. Now try to reboot, enter BIOS setup and select SSD as boot disk. Does it boot now from SSD?

catarino · July 15, 2023, 8:52am

Hi,
Everything is working 100‰

After replace the hdd drives one by one and rebuild the grub

Many thanks to arvidjar, you are the best.