Grub problem after substitute failed disk in an mdadm Raid1

Hello all,

After substitute a disk into a Raid1 system, i am unable to fix grub from Yast. The system startup only manually, with the command line of Grub.

Into yast, i don’t see as root device /dev/md0 (wich is the root partition), but only /dev/md2 and /dev/md3. As result, yast can’t re-install grub correctly.

Here is my blkid:

/dev/sda1: UUID="2a4e6095-a9c2-39f0-0477-060e49a1bbbe" UUID_SUB="78a1dac3-cbb6-7d01-2b6f-69c87585d409" LABEL="linux:0" TYPE="linux_raid_member"
/dev/sda2: UUID="1d39a2e1-22b1-d95d-d727-cf11d4dd65de" UUID_SUB="83ff9767-2be3-f24b-1904-c0a33d4b32df" LABEL="linux:1" TYPE="linux_raid_member"
/dev/sda3: UUID="96aaf4c0-5364-150c-1304-df883950f929" UUID_SUB="2fe57595-dcf6-49d3-08a9-43c6f584ddff" LABEL="linux:2" TYPE="linux_raid_member"
/dev/sda4: UUID="8a38879b-1f60-f4a7-7a3c-2d6ca1306844" UUID_SUB="eee184c0-44e9-0077-034d-2ed7567d630f" LABEL="linux:3" TYPE="linux_raid_member"
/dev/sdb1: UUID="2a4e6095-a9c2-39f0-0477-060e49a1bbbe" UUID_SUB="64b4fb23-883f-ac43-3962-ae3c60c118a8" LABEL="linux:0" TYPE="linux_raid_member"
/dev/sdb2: UUID="1d39a2e1-22b1-d95d-d727-cf11d4dd65de" UUID_SUB="c876ca43-e5f0-5693-dc21-cc7a07d2f76f" LABEL="linux:1" TYPE="linux_raid_member"
/dev/sdb3: UUID="96aaf4c0-5364-150c-1304-df883950f929" UUID_SUB="3d564fa9-fad8-f46f-f87f-01b7230393ec" LABEL="linux:2" TYPE="linux_raid_member"
/dev/sdb4: UUID="8a38879b-1f60-f4a7-7a3c-2d6ca1306844" UUID_SUB="8cac3c53-50a4-6e56-ca30-993b9661b7b5" LABEL="linux:3" TYPE="linux_raid_member"
/dev/md0: UUID="099e775b-2df0-4de8-a08e-b4920beff9d7" TYPE="ext3" PTTYPE="dos"
/dev/md2: UUID="abe4d05b-581b-44f5-a2e1-0ec14393f0ab" TYPE="swap"
/dev/md3: UUID="54c755bb-39f1-45c8-9871-519aab91fcab" TYPE="ext3"
/dev/md1: UUID="9aa63ee5-7c49-4ae2-b388-e3716b478d68" TYPE="ext3" PTTYPE="dos"


Notice the PTTYPE=“dos” for /dev/md0 and /dev/md1. I think is that the problem, because in yast i see as root device only md2 and md3. But i don’t know how to remove that PTTYPE…

Here is my result of bootinfo

                  Boot Info Script 0.61      [1 April 2012]


============================= Boot Info Summary: ===============================

 => Grub Legacy (v0.97) is installed in the MBR of /dev/sda and looks on the
    same drive in partition #1 for /boot/grub/stage2 and /boot/grub/menu.lst.
 => Grub Legacy (v0.97) is installed in the MBR of /dev/sdb and looks on the
    same drive in partition #1 for /boot/grub/stage2 and /boot/grub/menu.lst.

sda1: __________________________________________________________________________

    File system:       linux_raid_member
    Boot sector type:  Grub Legacy
    Boot sector info:  Grub Legacy (v0.97) is installed in the boot sector
                       of sda1 and looks at sector 4212736 of the same hard
                       drive for the stage2 file.  A stage2 file is at this
                       location on /dev/sdb.  Stage2 looks on partition #1
                       for /boot/grub/menu.lst.

sda2: __________________________________________________________________________

    File system:       linux_raid_member
    Boot sector type:  Grub Legacy
    Boot sector info:  Grub Legacy (v0.97) is installed in the boot sector
                       of sda2 and looks at sector 4427368 of the same hard
                       drive for the stage2 file.  A stage2 file is at this
                       location on /dev/sdb.  Stage2 looks on partition #1
                       for /boot/grub/menu.lst.

sda3: __________________________________________________________________________

    File system:       linux_raid_member
    Boot sector type:  -
    Boot sector info:

sda4: __________________________________________________________________________

    File system:       linux_raid_member
    Boot sector type:  -
    Boot sector info:

sdb1: __________________________________________________________________________

    File system:       linux_raid_member
    Boot sector type:  Grub Legacy
    Boot sector info:  Grub Legacy (v0.97) is installed in the boot sector
                       of sdb1 and looks at sector 4212736 of the same hard
                       drive for the stage2 file.  A stage2 file is at this
                       location on /dev/sdb.  Stage2 looks on partition #1
                       for /boot/grub/menu.lst.

sdb2: __________________________________________________________________________

    File system:       linux_raid_member
    Boot sector type:  Grub Legacy
    Boot sector info:  Grub Legacy (v0.97) is installed in the boot sector
                       of sdb2 and looks at sector 4427368 of the same hard
                       drive for the stage2 file.  A stage2 file is at this
                       location on /dev/sdb.  Stage2 looks on partition #1
                       for /boot/grub/menu.lst.

sdb3: __________________________________________________________________________

    File system:       linux_raid_member
    Boot sector type:  -
    Boot sector info:

sdb4: __________________________________________________________________________

    File system:       linux_raid_member
    Boot sector type:  -
    Boot sector info:

md0: ___________________________________________________________________________

    File system:       ext3
    Boot sector type:  Grub Legacy
    Boot sector info:  Grub Legacy (v0.97) is installed in the boot sector
                       of md0 and looks at sector 4212736 of the same hard
                       drive for the stage2 file.  A stage2 file is at this
                       location on /dev/sdb.  Stage2 looks on partition #1
                       for /boot/grub/menu.lst.
    Operating System:  Welcome to openSUSE 13.1
                       "Bottle" - Kernel ().
    Boot files:        /boot/grub/menu.lst /etc/fstab

md2: ___________________________________________________________________________

    File system:       swap
    Boot sector type:  -
    Boot sector info:

md3: ___________________________________________________________________________

    File system:       ext3
    Boot sector type:  -
    Boot sector info:
    Operating System:
    Boot files:

md1: ___________________________________________________________________________

    File system:       ext3
    Boot sector type:  Grub Legacy
    Boot sector info:  Grub Legacy (v0.97) is installed in the boot sector
                       of md1 and looks at sector 4427368 of the same hard
                       drive for the stage2 file.  A stage2 file is at this
                       location on /dev/sdb.  Stage2 looks on partition #1
                       for /boot/grub/menu.lst.
    Operating System:
    Boot files:

============================ Drive/Partition Info: =============================

Drive: sda _____________________________________________________________________

Disk /dev/sda: 360.1 GB, 360080695296 bytes, 703282608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Partition  Boot  Start Sector    End Sector  # of Sectors  Id System

/dev/sda1    *          2,048    20,965,375    20,963,328  fd Linux raid autodetect
/dev/sda2          20,965,376    83,875,839    62,910,464  fd Linux raid autodetect
/dev/sda3          83,875,840    88,068,095     4,192,256  fd Linux raid autodetect
/dev/sda4          88,068,096   488,396,799   400,328,704  fd Linux raid autodetect


Drive: sdb _____________________________________________________________________

Disk /dev/sdb: 750.2 GB, 750156374016 bytes, 1465149168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Partition  Boot  Start Sector    End Sector  # of Sectors  Id System

/dev/sdb1    *          2,048    20,965,375    20,963,328  fd Linux raid autodetect
/dev/sdb2          20,965,376    83,875,839    62,910,464  fd Linux raid autodetect
/dev/sdb3          83,875,840    88,068,095     4,192,256  fd Linux raid autodetect
/dev/sdb4          88,068,096   488,396,799   400,328,704  fd Linux raid autodetect


"blkid" output: ________________________________________________________________

Device           UUID                                   TYPE       LABEL

/dev/md0         099e775b-2df0-4de8-a08e-b4920beff9d7   ext3
/dev/md1         9aa63ee5-7c49-4ae2-b388-e3716b478d68   ext3
/dev/md2         abe4d05b-581b-44f5-a2e1-0ec14393f0ab   swap
/dev/md3         54c755bb-39f1-45c8-9871-519aab91fcab   ext3
/dev/sda1        2a4e6095-a9c2-39f0-0477-060e49a1bbbe   linux_raid_member linux:0
/dev/sda2        1d39a2e1-22b1-d95d-d727-cf11d4dd65de   linux_raid_member linux:1
/dev/sda3        96aaf4c0-5364-150c-1304-df883950f929   linux_raid_member linux:2
/dev/sda4        8a38879b-1f60-f4a7-7a3c-2d6ca1306844   linux_raid_member linux:3
/dev/sdb1        2a4e6095-a9c2-39f0-0477-060e49a1bbbe   linux_raid_member linux:0
/dev/sdb2        1d39a2e1-22b1-d95d-d727-cf11d4dd65de   linux_raid_member linux:1
/dev/sdb3        96aaf4c0-5364-150c-1304-df883950f929   linux_raid_member linux:2
/dev/sdb4        8a38879b-1f60-f4a7-7a3c-2d6ca1306844   linux_raid_member linux:3

================================ Mount points: =================================

Device           Mount_Point              Type       Options

/dev/md0         /                        ext3       (rw,relatime,data=ordered)
/dev/md1         /srv                     ext3       (rw,relatime,data=ordered)
/dev/md3         /var                     ext3       (rw,relatime,data=ordered)


=========================== md0/boot/grub/menu.lst: ============================

--------------------------------------------------------------------------------
# Modified by YaST2. Last modification on Thu May 15 13:00:48 CEST 2014
# THIS FILE WILL BE PARTIALLY OVERWRITTEN by perl-Bootloader
# For the new kernel it try to figure out old parameters. In case we are not able to recognize it (e.g. change of flavor or strange install order ) it it use as fallback installation parameters from /etc/sysconfig/bootloader

default 0
timeout 8
##YaST - activate

###Don't change this comment - YaST2 identifier: Original name: linux###
title Kernel-3.11.6-4-desktop
    kernel  root=/dev/sda1 vga=0x317
    initrd
--------------------------------------------------------------------------------

================================ md0/etc/fstab: ================================

--------------------------------------------------------------------------------
/dev/disk/by-id/md-uuid-96aaf4c0:5364150c:1304df88:3950f929 swap                 swap       defaults              0 0
/dev/disk/by-id/md-uuid-2a4e6095:a9c239f0:0477060e:49a1bbbe /                    ext3       acl,user_xattr        1 1
/dev/disk/by-id/md-uuid-1d39a2e1:22b1d95d:d727cf11:d4dd65de /srv                 ext3       acl,user_xattr        1 2
/dev/disk/by-id/md-uuid-8a38879b:1f60f4a7:7a3c2d6c:a1306844 /var                 ext3       acl,user_xattr        1 2
proc                 /proc                proc       defaults              0 0
sysfs                /sys                 sysfs      noauto                0 0
debugfs              /sys/kernel/debug    debugfs    noauto                0 0
usbfs                /proc/bus/usb        usbfs      noauto                0 0
devpts               /dev/pts             devpts     mode=0620,gid=5       0 0
--------------------------------------------------------------------------------

==================== md0: Location of files loaded by Grub: ====================

           GiB - GB             File                                 Fragment(s)


=============================== StdErr Messages: ===============================

  No volume groups found
cat: /tmp/BootInfo-KA6R3oda/Tmp_Log: No such file or directory


Help is needed, i don’t know how is it appening and why, and most important, how to resolve it…

Claudio.

On 2014-05-15 13:56, Claudio75 wrote:

> Notice the PTTYPE=“dos” for /dev/md0 and /dev/md1. I think is that the
> problem, because in yast i see as root device only md2 and md3. But i
> don’t know how to remove that PTTYPE…

I think it means that there appears to be a partition table there.


minas-tirith:~ # blkid /dev/sda
/dev/sda: PTTYPE="dos"
minas-tirith:~ #

A partition can contain a partition table, but it is atypical.
That it says “dos” is expected.


Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))

Ok, it can be… but how i can fix my system to make it boot again automatically from grub ?

I think this can be a yast bug, because it isn’t able to see some of my md devices, and obviously isn’t able to re-install a new updated grub menu. In this situation Yast write always an incomplete menu.lst, and the system don’t boot automatically…

On 2014-05-15 21:16, Claudio75 wrote:

> Ok, it can be… but how i can fix my system to make it boot again
> automatically from grub ?

You can verify the situation with “fdisk -l /dev/md0”, it will tell you
if there is a partition table in there. There should not be, but I’m not
sure it is a problem.

How to solve the situation, I don’t know, I’m not an expert on raid
booting problems.

Years ago you needed non-raid, separate boot partitions to boot raids.
Grub can handle some raid types automatically, but then YaST does not
support all the possibilities, or doesn’t do it well. This I know
because it has been commented here.

Maybe you have to resort to configure grub without using YaST :-?


Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))

fdisk -l /dev/md0

Disk /dev/md0: 10.7 GB, 10733150208 bytes, 20963184 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00000000

    Device Boot      Start         End      Blocks   Id  System

No partition, as you can see. Only that “Disk label type: dos”…

Into the same system, another md:

fdisk -l /dev/md3

Disk /dev/md3: 205.0 GB, 204968222720 bytes, 400328560 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

No Disk label, and anything else. I don’t know how is possible, but all the partition are created with the same method, but differs…strange…

Yes, i have also thinked this. I give a try next week (is a production server, so Friday is not the right day for testing :wink: )

Meantime, if anyone want to give me an help, is welcome!

On 2014-05-16 13:56, Claudio75 wrote:

> No partition, as you can see. Only that “Disk label type: dos”…

Nope, there is a partition table, with no partitions. YaST sees that and
is confused, because what is used is the entire device (md0), not a
partition on it, which does not exist.

md3 has no partition table.

>> Maybe you have to resort to configure grub without using YaST :-?
>>
>
> Yes, i have also thinked this. I give a try next week (is a production
> server, so Friday is not the right day for testing :wink: )

Or… you can recreate md0 without a partition table. Back it up fully,
remove from raid, erase, say, the first 100 MB, and recreate raid and
filesystem.

That’s what I would attempt. I can not guarantee results, but I would
like to find out if that makes YaST happy, and easier life aftewards…


Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))

Which disk of two?

i am unable to fix grub from Yast.

I would not worry much about YaST at this point but about making your system bootable.

The system startup only manually, with the command line of Grub.

This indicates that it does not find menu.lst. What would be interesting - if it finds stage2 at all or stops at stage1.5. What commands are available at this point?

Here is my result of bootinfo

It could be bootinfoscript error, but it does not find anything on /dev/sda1 (stage2 or menu.lst). Did you replace sda? Please show output of “cat /proc/mdstat”, “mdadm --detail --scan --verbose” and “mdadm --examine /dev/sda1 /dev/sdb1”

I have changed /dev/sdb

I dubt it can’t find the menu.lst, because when try to boot, the menù is showed, but when selected the kernel it says “unable to mount partition”.

No, i have replaced /dev/sdb.

cat /proc/mdstat
Personalities : [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md1 : active raid1 sdb2[3] sda2[2]
      31455160 blocks super 1.0 [2/2] [UU]
      bitmap: 1/1 pages [4KB], 65536KB chunk

md3 : active raid1 sdb4[3] sda4[2]
      200164280 blocks super 1.0 [2/2] [UU]
      bitmap: 2/2 pages [8KB], 65536KB chunk

md2 : active raid1 sdb3[3] sda3[2]
      2096116 blocks super 1.0 [2/2] [UU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md0 : active raid1 sdb1[3] sda1[2]
      10481592 blocks super 1.0 [2/2] [UU]
      bitmap: 1/1 pages [4KB], 65536KB chunk


 mdadm --detail --scan --verbose
ARRAY /dev/md0 level=raid1 num-devices=2 metadata=1.0 name=linux:0 UUID=2a4e6095:a9c239f0:0477060e:49a1bbbe
   devices=/dev/sda1,/dev/sdb1
ARRAY /dev/md2 level=raid1 num-devices=2 metadata=1.0 name=linux:2 UUID=96aaf4c0:5364150c:1304df88:3950f929
   devices=/dev/sda3,/dev/sdb3
ARRAY /dev/md3 level=raid1 num-devices=2 metadata=1.0 name=linux:3 UUID=8a38879b:1f60f4a7:7a3c2d6c:a1306844
   devices=/dev/sda4,/dev/sdb4
ARRAY /dev/md1 level=raid1 num-devices=2 metadata=1.0 name=linux:1 UUID=1d39a2e1:22b1d95d:d727cf11:d4dd65de
   devices=/dev/sda2,/dev/sdb2
mdadm --examine /dev/sda1 /dev/sdb1
/dev/sda1:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 2a4e6095:a9c239f0:0477060e:49a1bbbe
           Name : linux:0
  Creation Time : Thu Jun  9 12:05:39 2011
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 20963184 (10.00 GiB 10.73 GB)
     Array Size : 10481592 (10.00 GiB 10.73 GB)
   Super Offset : 20963312 sectors
   Unused Space : before=0 sectors, after=120 sectors
          State : clean
    Device UUID : 78a1dac3:cbb67d01:2b6f69c8:7585d409

Internal Bitmap : -8 sectors from superblock
    Update Time : Mon May 19 09:57:57 2014
       Checksum : ad2763bb - correct
         Events : 1737


   Device Role : Active device 1
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdb1:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 2a4e6095:a9c239f0:0477060e:49a1bbbe
           Name : linux:0
  Creation Time : Thu Jun  9 12:05:39 2011
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 20963184 (10.00 GiB 10.73 GB)
     Array Size : 10481592 (10.00 GiB 10.73 GB)
   Super Offset : 20963312 sectors
   Unused Space : before=0 sectors, after=120 sectors
          State : clean
    Device UUID : 64b4fb23:883fac43:3962ae3c:60c118a8

Internal Bitmap : -8 sectors from superblock
    Update Time : Mon May 19 09:57:57 2014
  Bad Block Log : 512 entries available at offset -8 sectors
       Checksum : 62082e57 - correct
         Events : 1737


   Device Role : Active device 0
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)


Thank you.

OK, so I misunderstood your original message - I assumed system stops in GRUB command line.

In this case the obvious question - why do you have “root=/dev/sda1” in menu.lst when your root is on MD RAID?

That is the messed up grub after Yast changed it, here is my original menu.lst:

# Modified by YaST2. Last modification on Wed May 14 20:11:48 CEST 2014
# THIS FILE WILL BE PARTIALLY OVERWRITTEN by perl-Bootloader
# For the new kernel it try to figure out old parameters. In case we are not able to recognize it (e.g. change of flavor or strange install order ) it it use as fallback installation parameters from /etc/sysconfig/bootloader

default 0
timeout 8
##YaST - generic_mbr
##YaST - activate

###Don't change this comment - YaST2 identifier: Original name: linux###
title Desktop -- openSUSE - 3.11.6-4
    root (hd0,0)
    kernel /boot/vmlinuz-3.11.6-4-default root=/dev/disk/by-id/md-uuid-2a4e6095:a9c239f0:0477060e:49a1bbbe resume=/dev/disk/by-id/md-uuid-96aaf4c0:5364150c:1304df88:3950f929 splash=silent quiet showopts
    initrd /boot/initrd-3.11.6-4-desktop

###Don't change this comment - YaST2 identifier: Original name: failsafe###
title Failsafe -- openSUSE - 3.11.6-4
    root (hd0,0)
    kernel /boot/vmlinuz-3.11.6-4-desktop root=/dev/disk/by-id/md-uuid-2a4e6095:a9c239f0:0477060e:49a1bbbe video=1280x1024 resume=/dev/disk/by-id/md-uuid-96aaf4c0:5364150c:1304df88:3950f929 splash=silent quiet showopts vga=0x317
    initrd /boot/initrd-3.11.6-4-desktop


As you can see, the root is /dev/md0, the correct root device.

Another update:

I have tryied to reboot the machine, to make some tests (as a production server i can’t reboot when i want). With the following configuration Grub works and the system starts:

# Modified by YaST2. Last modification on Tue Apr  1 09:50:15 CEST 2014
# THIS FILE WILL BE PARTIALLY OVERWRITTEN by perl-Bootloader
# For the new kernel it try to figure out old parameters. In case we are not able to recognize it (e.g. change of flavor or strange install order ) it it use as fallback installation parameters from /etc/sysconfig/bootloader

default 0
timeout 8
##YaST - generic_mbr
##YaST - activate

###Don't change this comment - YaST2 identifier: Original name: linux###
title Desktop -- openSUSE - 3.11.6-4
    root (hd1,0)
    kernel /boot/vmlinuz-3.11.6-4-desktop root=/dev/disk/by-id/md-uuid-2a4e6095:a9c239f0:0477060e:49a1bbbe video=1280x1024 resume=/dev/disk/by-id/md-uuid-96aaf4c0:5364150c:1304df88:3950f929 splash=silent quiet showopts vga=0x317
    initrd /boot/initrd-3.11.6-4-desktop

###Don't change this comment - YaST2 identifier: Original name: failsafe###
title Failsafe -- openSUSE - 3.11.6-4
    root (hd1,0)
    kernel /boot/vmlinuz-3.11.6-4-desktop root=/dev/disk/by-id/md-uuid-2a4e6095:a9c239f0:0477060e:49a1bbbe video=1280x1024 resume=/dev/disk/by-id/md-uuid-96aaf4c0:5364150c:1304df88:3950f929 splash=silent quiet showopts vga=0x317
    initrd /boot/initrd-3.11.6-4-desktop


But… !! If i go under Yast - Boot Loader, it says to me “because of incorrect partitioning the bootloader can’t be installed”. If i try to edit something from Yast, it breaks again the menu.lst.

At this point, is this a Yast bug? How i can fix it ?

May be a yast bug if so report it. Until the developers fix it don’t use yast just edit the files if you need to modify grub.

On 2014-05-20 21:06, gogalthorp wrote:
>
> May be a yast bug if so report it. Until the developers fix it don’t use
> yast just edit the files if you need to modify grub.

IMO, YaST is apparently confused because the OP has two raid partitions,
md0 and md1, that contain an empty partition table.


Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))

Ok, but now my problem is: if i upgrade the kernel via zypper up or you, the new kernel can be broken because of this problem ? I think you or zypper use the same procedure to modify the grub and generate the initrd for a new kernel, so the result can be messed up ?