Page 2 of 3 FirstFirst 123 LastLast
Results 11 to 20 of 22

Thread: Upgrade destroys dmraid

  1. #11
    Join Date
    Sep 2012
    Posts
    4,944

    Default Re: Upgrade destroys dmraid

    Thanks. It looks like race condition + bug in detection of dmraid disks. Quoting interesting lines from your log.
    Code:
    Sep 10 22:08:48 linux-scdz systemd[1]: systemd 228 running in system mode. (+PAM -AUDIT +SELINUX -IMA +APPARMOR -SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT -GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID -ELFUTILS +KMOD -IDN)
    Sep 10 22:08:48 linux-scdz dracut-cmdline[100]: Using kernel command line parameters: rd.dm.uuid=nvidia_cbidcbcg resume=/dev/mapper/nvidia_cbidcbcg_part5 root=/dev/mapper/nvidia_cbidcbcg_part3 rootfstype=btrfs rootflags=rw,relatime,space_cache,subvolid=259,subvol=/@/.snapshots/1/snapshot,subvol=@/.snapshots/1/snapshot BOOT_IMAGE=/vmlinuz-4.4.79-18.26-default root=UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 resume=/dev/mapper/nvidia_cbidcbcg-part5 splash=silent quiet showopts
    initrd starts. It wants to use /dev/mapper/nvidia_cbidcbcg_part3 as root device.
    Code:
    Sep 10 22:08:50 linux-scdz kernel: BTRFS: device fsid 44aa9465-f432-40e4-81a1-1ab9b9ac2c27 devid 1 transid 1109 /dev/sdb3
    Still kernel gets /dev/sdb3 as btrfs device, most likely from udev scanning sdb3.
    Code:
    Sep 10 22:08:51 linux-scdz dracut-initqueue[260]: Scanning for dmraid devices nvidia_cbidcbcg
    Sep 10 22:08:51 linux-scdz dracut-initqueue[260]: Found dmraid sets:
    Sep 10 22:08:51 linux-scdz dracut-initqueue[260]: /dev/sdb: "pdc" and "nvidia" formats discovered (using nvidia)! nvidia_cbidcbcg
    Sep 10 22:08:51 linux-scdz dracut-initqueue[260]: Activating nvidia_cbidcbcg
    Sep 10 22:08:51 linux-scdz dracut-initqueue[260]: /dev/sdb: "pdc" and "nvidia" formats discovered (using nvidia)!
    Sep 10 22:08:51 linux-scdz dracut-initqueue[260]: RAID set "nvidia_cbidcbcg" was activated
    Sep 10 22:08:51 linux-scdz dracut-initqueue[260]: RAID set "nvidia_cbidcbcg" was not activated
    dmraid itself works as intended and activates array.
    Code:
    Sep 10 22:08:52 linux-scdz systemd[1]: Found device /dev/mapper/nvidia_cbidcbcg-part5.
    Sep 10 22:08:52 linux-scdz systemd[1]: Starting Resume from hibernation using device /dev/mapper/nvidia_cbidcbcg-part5...
    We see that systemd correctly tries resume from swap on dmraid device ("correctly" meaning - it does what was configured on kernel command line).
    Code:
    Sep 10 22:08:52 linux-scdz systemd[1]: Starting File System Check on /dev/disk/by-uuid/44aa9465-f432-40e4-81a1-1ab9b9ac2c27...
    Sep 10 22:08:52 linux-scdz systemd[1]: Started File System Check on /dev/disk/by-uuid/44aa9465-f432-40e4-81a1-1ab9b9ac2c27.
    Sep 10 22:08:52 linux-scdz systemd[1]: Mounting /sysroot...
    Sep 10 22:08:52 linux-scdz kernel: BTRFS info (device dm-3): disk space caching is enabled
    Sep 10 22:08:52 linux-scdz kernel: BTRFS info (device dm-3): has skinny extents
    Sep 10 22:08:52 linux-scdz systemd[1]: Mounted /sysroot.
    Here unfortunately we do not have enough information, but I think that systemd mounts dmraid device because it says dm-3 and not sdb3.
    Code:
    Sep 10 22:08:56 linux-scdz kernel: systemd: 18 output lines suppressed due to ratelimiting
    Not directly related but as general comment - it makes sense to always run with printk.devkmsg=on kernel parameter to restore previous kernel behavior. Otherwise some early log messages tend to get lost.
    Code:
    ep 10 22:08:54 linux-scdz systemd[1]: systemd 228 running in system mode. (+PAM -AUDIT +SELINUX -IMA +APPARMOR -SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT -GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID -ELFUTILS +KMOD -IDN)
    Main system is started from initrd.
    Code:
    Sep 10 22:08:59 linux-scdz systemd[1]: Mounting /var/spool...
    Sep 10 22:08:59 linux-scdz systemd[1]: Mounting /var/lib/mariadb...
    Sep 10 22:08:59 linux-scdz systemd[1]: Mounting /var/lib/mysql...
    Sep 10 22:08:59 linux-scdz systemd[1]: Mounting /usr/local...
    Sep 10 22:08:59 linux-scdz mount[1016]: mount: /dev/sdb3 is already mounted or /var/spool busy
    Oops. Although we have seen before that root was probably mounted via dmraid device, systemd now attempts to mount subvolumes via individual disk which fails as it should. And it will end up in emergency mode
    Code:
    Sep 10 22:09:00 linux-scdz systemd[1]: Started Emergency Shell.
    So my educated guess is following:

    1. During scanning of pysical disk it (incorrecly in this case) identifies it as containing btrfs filesystem so links /dev/disk/by-uuid (and others) point to physical disk.
    2. initrd is using configured device name (/dev/mapper/nvidia_cbidcbcg_part3) so effectively ignores these links.
    3. /etc/fstab is configured with UUID=xxx device for subvolumes, so when systemd tries to mount them it resolves UUID to /dev/disk/by-uuid links which points to physical device.


    To get more information at least device links (or simply "ls -lR /dev") when it fails, and /etc/fstab is needed. If my guess is right, you can workaround it by replacing UUID entries with /dev/mapper/nvidia_cbidcbcg_part3 in /etc/fstab.

  2. #12

    Default Re: Upgrade destroys dmraid

    Here is the output of "ls -lR /dev":

    https://paste.opensuse.org/46414425

    Here is fstab:

    Code:
    UUID=8124ab12-a079-425a-a111-a88a0d4e7f68 swap swap defaults 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 / btrfs defaults 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /home btrfs subvol=@/home 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /opt btrfs subvol=@/opt 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /srv btrfs subvol=@/srv 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /tmp btrfs subvol=@/tmp 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /usr/local btrfs subvol=@/usr/local 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /var/cache btrfs subvol=@/var/cache 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /var/crash btrfs subvol=@/var/crash 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /var/lib/libvirt/images btrfs subvol=@/var/lib/libvirt/images 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /var/lib/machines btrfs subvol=@/var/lib/machines 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /var/lib/mailman btrfs subvol=@/var/lib/mailman 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /var/lib/mariadb btrfs subvol=@/var/lib/mariadb 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /var/lib/mysql btrfs subvol=@/var/lib/mysql 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /var/lib/named btrfs subvol=@/var/lib/named 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /var/lib/pgsql btrfs subvol=@/var/lib/pgsql 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /var/log btrfs subvol=@/var/log 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /var/opt btrfs subvol=@/var/opt 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /var/spool btrfs subvol=@/var/spool 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /var/tmp btrfs subvol=@/var/tmp 0 0
    UUID=44aa9465-f432-40e4-81a1-1ab9b9ac2c27 /.snapshots btrfs subvol=@/.snapshots 0 0
    UUID=b892a114-7ba8-44b6-8460-17e33fda874a /boot                ext4       acl,user_xattr        1 2

  3. #13
    Join Date
    Sep 2012
    Posts
    4,944

    Default Re: Upgrade destroys dmraid

    Quote Originally Posted by amadrits View Post
    Here is the output of "ls -lR /dev":
    It shows correct links. As you were able to boot at the end, it could have changed after initial failure though. Next time when it stops in emergency mode, could you capture output of "systemctl show '*'", this could give more insight in what state systemd was at the point of failure.

  4. #14

    Default Re: Upgrade destroys dmraid

    Here is the output of "systemctl show '*'":

    https://paste.opensuse.org/81168739

  5. #15
    Join Date
    Sep 2012
    Posts
    4,944

    Default Re: Upgrade destroys dmraid

    Quote Originally Posted by amadrits View Post
    Here is the output of "systemctl show '*'":

    https://paste.opensuse.org/81168739
    It looks like output is truncated. Could you check if this is full content; if not probably try to compress file.

  6. #16

    Default Re: Upgrade destroys dmraid

    I compared directly with the output on the screen in the dracut emergency shell and it is complete.

    https://paste.opensuse.org/99793386

  7. #17

    Default Re: Upgrade destroys dmraid

    Hello

    I wanted to inquire how this is going on.

    Do I have to bring some other logs or do I have to make some other tests?

    Will this be fixed in openSUSE 15?

    When will openSUSE 15 be released?

  8. #18
    Join Date
    Jun 2008
    Location
    Groningen, Netherlands
    Posts
    19,586
    Blog Entries
    14

    Default Re: Upgrade destroys dmraid

    Quote Originally Posted by amadrits View Post
    Hello

    I wanted to inquire how this is going on.

    Do I have to bring some other logs or do I have to make some other tests?

    Will this be fixed in openSUSE 15?

    When will openSUSE 15 be released?
    Did you file a bug report at bugzilla.opensuse.org ? Most of the devs/packagers aren't forums oriented, they need bug reports.

    We don't know what will be fixed and what won't in openSUSE 15 yet. It's too far away from release ( next year )
    ° Appreciate my reply? Click the star and let me know why.

    ° Perfection is not gonna happen. No way.

    https://en.opensuse.org/openSUSE:Board#Members
    http://en.opensuse.org/User:Knurpht
    http://nl.opensuse.org/Gebruiker:Knurpht

  9. #19

    Default Re: Upgrade destroys dmraid

    Replacing UUID entries with /dev/mapper/nvidia_cbidcbcg_part3 in /etc/fstab seems to work, but has to be checked every time Yast2 updated /etc/fstab, although settings in Yast2 are configured to use the device names to mount.

    I could make an ext4 boot partition out of my swap partition, but i could not delete the subvolumes /boot/grub2/i386-pc and /boot/grub2/x86_64-efi. How do i delete this subvolumes?

  10. #20

    Default Re: Upgrade destroys dmraid

    Sorry for this question. Subvolumes can be managed in Yast2 by editing the partition and klick "Subvolume Handling".

    For the race condition i filed a bug report (bug 1060551).

Page 2 of 3 FirstFirst 123 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •