Results 1 to 8 of 8

Thread: How do I get openSUSE to recognize/allocate LVM2 logical volumes on "hotplug"

  1. #1
    Join Date
    Dec 2008
    Location
    Buffalo, MN USA
    Posts
    82

    Question How do I get openSUSE to recognize/allocate LVM2 logical volumes on "hotplug"

    I've got a machine that's used as a local file server (Samba & NFS) which I wanted to upgrade the OS (it's running Kubuntu 10.04) and increase the size of the primary drive 1TB -> 2TB. I also wanted to switch to LVM partitioning because I've been using it on another machine since openSUSE 11.3 (it's now up to openSUSE 12.3) and I like the flexibility. Doing the LVM setup in YAST was so simple that I thought I'd do the disk partitioning and LVM setup in YAST before installing the OS. That way I could copy all of the non-changing files from the old drive to the new one and minimize the down-time of the file sharing system.

    So I plugged the drive into my USB 2.0-to-SATA adapter attached to the openSUSE machine, went into the Yast partitioner and created a 500 MB primary partition formatted as ext4 and an unformatted extended partition that occupied the rest of the drive. I created a volume group that encompassed that entire partition and created a number of ext4 formatted logical volumes.

    When YAST had completed, all of those new logical volumes showed up in /dev/disk/by-label/, so I mounted one of them at /mnt and proceeded to copy everything from the NFS-mounted volume on the other machine (cpio -p). When the copy was complete, I unmounted the volume at /mnt and tried to do an e2fsck of the just unmounted volume. I was greeted with an error about a zero-length superblock, and got the same error when trying to do an e2fsck on any of the newly-created (but never mounted or used) logical volumes.

    So I assumed that I needed to format the extended partition before creating the volume group and proceeded to do the entire process over, but this time formatting the extended partition as ext4 before going back into the YAST partitioner to create the volume group and logical volumes. Unfortunately that didn't help. Exactly the same error message.

    It seems that there's some issue (udev rules?) with udev and dm for drives that aren't present at boot time. For example:
    Code:
    # ls -l /dev/disk/by-label/
    total 0
    lrwxrwxrwx 1 root root 10 Dec 29 16:33 FB_boot -> ../../sdd1
    lrwxrwxrwx 1 root root 10 Dec 29 10:19 FreeBSD -> ../../sdb3
    lrwxrwxrwx 1 root root  9 Dec 29 10:21 openSUSE\x2013.1\x20KDE\x20Live -> ../../sr0
    lrwxrwxrwx 1 root root 10 Dec 29 10:19 oS_boot -> ../../sda1
    lrwxrwxrwx 1 root root 10 Dec 29 10:19 oS_home -> ../../dm-0
    lrwxrwxrwx 1 root root 10 Dec 29 10:19 oS_opt -> ../../dm-1
    lrwxrwxrwx 1 root root 10 Dec 29 10:19 oS_root -> ../../dm-2
    lrwxrwxrwx 1 root root 10 Dec 29 10:19 oS_shared -> ../../dm-3
    lrwxrwxrwx 1 root root 10 Dec 29 10:19 oS_srv -> ../../dm-4
    lrwxrwxrwx 1 root root 10 Dec 29 10:19 oS_swap -> ../../dm-5
    lrwxrwxrwx 1 root root 10 Dec 29 10:19 oS_tmp -> ../../dm-6
    lrwxrwxrwx 1 root root 10 Dec 29 10:19 oS_usr -> ../../dm-7
    lrwxrwxrwx 1 root root 10 Dec 29 10:19 oS_var -> ../../dm-8
    lrwxrwxrwx 1 root root 10 Dec 29 10:19 System -> ../../sdb1
    lrwxrwxrwx 1 root root 10 Dec 29 10:19 V-WXPpro-32 -> ../../sdb2
    notice that a number of nodes have been created at /dev/dm-?
    These are the logical volumes that are in use by the running system (openSUSE 12.3 x86_64). Notice also the /dev/disk/by-label/FB_boot is the first (only) primary partition on /dev/sdd. So udev finds the "hotplugged" USB drive, but not the logical volumes on that drive.

    see
    Code:
    # pvscan
      PV /dev/sdd5   VG freebee   lvm2 [1.82 TiB / 0    free]
      PV /dev/sda2   VG system    lvm2 [465.62 GiB / 301.62 GiB free]
      Total: 2 [2.27 TiB] / in use: 2 [2.27 TiB] / in no VG: 0 [0   ]
    now see
    Code:
    # vgscan
      Reading all physical volumes.  This may take a while...
      Found volume group "freebee" using metadata type lvm2
      Found volume group "system" using metadata type lvm2
    and
    Code:
    # lvs
      LV     VG      Attr      LSize   Pool Origin Data%  Move Log Copy%  Convert
      backup freebee -wi------ 966.51g                                           
      home   freebee -wi------ 250.00g                                           
      opt    freebee -wi------   3.00g                                           
      root   freebee -wi------   1.00g                                           
      shared freebee -wi------ 600.00g                                           
      srv    freebee -wi------   3.00g                                           
      swap   freebee -wi------   4.00g                                           
      tmp    freebee -wi------  10.00g                                           
      usr    freebee -wi------  10.00g                                           
      var    freebee -wi------  15.00g                                           
      home   system  -wi-ao---  25.00g                                           
      opt    system  -wi-ao---   3.00g                                           
      root   system  -wi-ao---   1.00g                                           
      shared system  -wi-ao--- 100.00g                                           
      srv    system  -wi-ao---   2.00g                                           
      swap   system  -wi-ao---   3.00g                                           
      tmp    system  -wi-ao---  10.00g                                           
      usr    system  -wi-ao---  10.00g                                           
      var    system  -wi-ao---  10.00g
    so lvm is seeing all of the logical volumes in both volume groups but nodes are only created for those actually present at boot. If I have the USB adapter and drive powered on at bootup, the necessary nodes ARE created.
    Code:
    # ls -l /dev/mapper/
    total 0
    crw------- 1 root root 10, 236 Dec 29 10:19 control
    lrwxrwxrwx 1 root root       7 Dec 29 10:19 system-home -> ../dm-0
    lrwxrwxrwx 1 root root       7 Dec 29 10:19 system-opt -> ../dm-1
    lrwxrwxrwx 1 root root       7 Dec 29 10:19 system-root -> ../dm-2
    lrwxrwxrwx 1 root root       7 Dec 29 10:19 system-shared -> ../dm-3
    lrwxrwxrwx 1 root root       7 Dec 29 10:19 system-srv -> ../dm-4
    lrwxrwxrwx 1 root root       7 Dec 29 10:19 system-swap -> ../dm-5
    lrwxrwxrwx 1 root root       7 Dec 29 10:19 system-tmp -> ../dm-6
    lrwxrwxrwx 1 root root       7 Dec 29 10:19 system-usr -> ../dm-7
    lrwxrwxrwx 1 root root       7 Dec 29 10:19 system-var -> ../dm-8
    It turns out that there's no problem with the logical volumes created by YAST. I plugged the USB adapter into the Kubuntu machine and after installing the lvm2 package all of the logical volumes appeared and I was able to successfully run fsck on all of them. I can also power down the USB adapter and when I power it back up all of the logical volumes reappear.

    So I suspect that this is an issue with udev rules not requiring udev to invoke the device mapper except at boot. I also supect that those same udev rules are why I'm having an issue with demonstrably working eSATA hardware not working in "hotplug" situations.

    Any guidance you can provide will be greatly apreciated.

    Regards,
    ron

  2. #2
    Join Date
    Dec 2008
    Location
    Buffalo, MN USA
    Posts
    82

    Default Re: How do I get openSUSE to recognize/allocate LVM2 logical volumes on "hotplug"

    Thanks to a suggestion on a different problem I'm having (https://forums.opensuse.org/english/...86_64-kde.html), here's a little more data which may be helpful:
    Code:
    ~> dmesg | tail -20
    [46224.606589] usb 2-2: new high-speed USB device number 4 using ehci_hcd
    [46224.722941] usb 2-2: New USB device found, idVendor=152d, idProduct=2336
    [46224.722952] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=5
    [46224.722960] usb 2-2: Product: JM20336 SATA, USB Combo
    [46224.722966] usb 2-2: Manufacturer: JMicron
    [46224.722970] usb 2-2: SerialNumber: 1F1688888888
    [46224.724619] scsi10 : usb-storage 2-2:1.0
    [46225.726685] scsi 10:0:0:0: Direct-Access     ST2000DM 001-9YN164            PQ: 0 ANSI: 2 CCS
    [46225.727243] sd 10:0:0:0: Attached scsi generic sg4 type 0
    [46225.728413] sd 10:0:0:0: [sdd] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
    [46225.729425] sd 10:0:0:0: [sdd] Write Protect is off
    [46225.729435] sd 10:0:0:0: [sdd] Mode Sense: 00 38 00 00
    [46225.730543] sd 10:0:0:0: [sdd] Asking for cache data failed
    [46225.730546] sd 10:0:0:0: [sdd] Assuming drive cache: write through
    [46225.733544] sd 10:0:0:0: [sdd] Asking for cache data failed
    [46225.733555] sd 10:0:0:0: [sdd] Assuming drive cache: write through
    [46225.746659]  sdd: sdd1 sdd2 < sdd5 >
    [46225.749790] sd 10:0:0:0: [sdd] Asking for cache data failed
    [46225.749800] sd 10:0:0:0: [sdd] Assuming drive cache: write through
    [46225.749808] sd 10:0:0:0: [sdd] Attached SCSI disk
    So the kernel found my USB-attached drive, here's what udev did with it:
    Code:
    # udevadm monitor
    monitor will print the received events for:
    UDEV - the event which udev sends out after rule processing
    KERNEL - the kernel uevent
    
    KERNEL[46257.808996] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2 (usb)
    KERNEL[46257.809961] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0 (usb)
    KERNEL[46257.810421] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10 (scsi)
    KERNEL[46257.810471] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/scsi_host/host10 (scsi_host)
    UDEV  [46257.819571] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2 (usb)
    UDEV  [46257.820766] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0 (usb)
    UDEV  [46257.821583] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10 (scsi)
    UDEV  [46257.822094] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/scsi_host/host10 (scsi_host)
    KERNEL[46258.813206] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0 (scsi)
    KERNEL[46258.813332] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0 (scsi)
    KERNEL[46258.813386] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/scsi_disk/10:0:0:0 (scsi_disk)
    KERNEL[46258.813477] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/scsi_device/10:0:0:0 (scsi_device)
    KERNEL[46258.813615] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/scsi_generic/sg4 (scsi_generic)
    KERNEL[46258.813754] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/bsg/10:0:0:0 (bsg)
    UDEV  [46258.814250] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0 (scsi)
    UDEV  [46258.814713] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0 (scsi)
    UDEV  [46258.815966] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/scsi_device/10:0:0:0 (scsi_device)
    UDEV  [46258.816315] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/scsi_disk/10:0:0:0 (scsi_disk)
    KERNEL[46258.817131] add      /devices/virtual/bdi/8:48 (bdi)
    UDEV  [46258.818232] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/scsi_generic/sg4 (scsi_generic)
    UDEV  [46258.818615] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/bsg/10:0:0:0 (bsg)
    UDEV  [46258.818665] add      /devices/virtual/bdi/8:48 (bdi)
    KERNEL[46258.833285] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/block/sdd (block)
    KERNEL[46258.833347] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/block/sdd/sdd1 (block)
    KERNEL[46258.833383] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/block/sdd/sdd2 (block)
    KERNEL[46258.833416] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/block/sdd/sdd5 (block)
    UDEV  [46259.001652] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/block/sdd (block)
    UDEV  [46259.011588] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/block/sdd/sdd2 (block)
    UDEV  [46259.071041] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/block/sdd/sdd5 (block)
    UDEV  [46259.127678] add      /devices/pci0000:00/0000:00:13.2/usb2/2-2/2-2:1.0/host10/target10:0:0/10:0:0:0/block/sdd/sdd1 (block)
    So it looks like udev found /dev/sdd1 and /dev/sdd5, but didn't call device mapper to bring up the logical volumes in /dev/sdd5.

    Any suggestions for what to do next?

    Thanks,
    ron

  3. #3
    Join Date
    Sep 2012
    Posts
    7,092

    Cool Re: How do I get openSUSE to recognize/allocate LVM2 logical volumes on "hotplug"

    Quote Originally Posted by r_widell View Post
    So I assumed that I needed to format the extended partition before creating the volume group
    No, it is not needed.
    It turns out that there's no problem with the logical volumes created by YAST. I plugged the USB adapter into the Kubuntu machine and after installing the lvm2 package all of the logical volumes appeared and I was able to successfully run fsck on all of them. I can also power down the USB adapter and when I power it back up all of the logical volumes reappear.
    So you say that you can
    - boot your system with external disk unplugged
    - plug in external hard disk
    - get all LVM volumes on it automatically recognized?

    Do you have any link to Ubuntu implementation?

    Any suggestions for what to do next?
    To my best knowledge, there is no support for incremental assembly of LVM volumes. It is relatively simple to run script to scan disk for LVM and activate it - the problem is, it will not work in general case when volumes could span multiple devices. So your best bet is to find out how Ubuntu does it and reuse implementation. You could also ask upstream

  4. #4
    Join Date
    Dec 2008
    Location
    Buffalo, MN USA
    Posts
    82

    Default Re: How do I get openSUSE to recognize/allocate LVM2 logical volumes on "hotplug"

    Quote Originally Posted by arvidjaar View Post
    No, it is not needed.
    I didn't think it should be, which is why my first attempt did NOT format partition 5

    Quote Originally Posted by arvidjaar View Post
    So you say that you can
    - boot your system with external disk unplugged
    - plug in external hard disk
    - get all LVM volumes on it automatically recognized?
    Not exactly, I didn't reboot the Kubuntu system (it only gets rebooted for kernel or libc6 updates-- or power failures).

    I plugged the external disk into the Kubuntu system and powered it on and was notified of new media, but it was only the ext4 primary partition. So I checked for the status of lvm2 and saw that it wasn't installed on that machine, so I installed it (it automatically also installed the dependencies). Once the install was complete, I was notified that new media was available. That new media was all of the logical volumes I had defined in partition 5 of the external drive.

    I can power down the drive (after ensuring that nothing's mounted) and completely detach it from the system. When I reattach the drive to that already running Kubuntu system and apply power to the external drive I am notified of new media. That new media consists of ALL formatted file systems on the drive, the ext4 primary partition and all of the logical volumes on partition 5.

    Quote Originally Posted by arvidjaar View Post
    Do you have any link to Ubuntu implementation?
    I'm not sure I understand the request, so please bear with me if this isn't what you wanted.
    Code:
    ~$ aptitude show lvm2
    Package: lvm2
    State: not installed
    Version: 2.02.54-1ubuntu4.1
    Priority: optional
    Section: admin
    Maintainer: Ubuntu Core Developers <ubuntu-devel-discuss@lists.ubuntu.com>
    Uncompressed Size: 1,114k
    Depends: libc6 (>= 2.7), libdevmapper-event1.02.1 (>= 2:1.02.20),
             libdevmapper1.02.1 (>= 2:1.02.39), libreadline6 (>= 6.0), lsb-base,
             dmsetup (> 2:1.02.36), watershed (>= 2)
    Conflicts: lvm-common
    Replaces: lvm-common
    Description: The Linux Logical Volume Manager
     This is LVM2, the rewrite of The Linux Logical Volume Manager.  LVM supports
     enterprise level volume management of disk and disk subsystems by grouping
     arbitrary disks into volume groups. The total capacity of volume groups can be
     allocated to logical volumes, which are accessed as regular block devices.
    Homepage: http://sources.redhat.com/lvm2/
    This shows the version of the lvm2 package that was installed (ignore the "not installed" status, this command was run on a different machine). It also shows all of the dependencies and their version requirements.
    http://us.archive.ubuntu.com/ubuntu/...tu4.1_i386.deb
    http://us.archive.ubuntu.com/ubuntu/...untu4_i386.deb
    http://us.archive.ubuntu.com/ubuntu/pool/main/l/lvm2/libdevmapper1.02.1-udeb_1.02.39-1ubuntu4.1_i386.udeb
    http://us.archive.ubuntu.com/ubuntu/...tu4.1_i386.deb
    were the files installed. All of them and the sources can be found at Index of /ubuntu/pool/main/l/lvm2. A description of the watershed package can be found at https://launchpad.net/ubuntu/+source/watershed

    The sources are in the form of:
    PkgNameVersion.{orig,debian}.tar.gz and PkgNameVersion-{debian,ubuntu}Revision.diff.gz. There will also be a .dsc file for each package which is a text file describing all of the requirements for building a .deb package.

    If that's not what you wanted, please let me know so I can try again.

    As an aside, in the Debian APT system all versions of all packages are kept in a large pool. To determine which of those packages are in a particular release you need to bore down through the dists/release subdirectories of the Index of /ubuntu archive 'til you get to a compressed text file called Packages. eg. http://us.archive.ubuntu.com/ubuntu/...6/Packages.bz2 is the bzip2-compressed file which identifies which versions of which packages are in the main-i386 repository for the lucid (10.04) release of ubuntu. I know it can be confusing at first.

    Quote Originally Posted by arvidjaar View Post
    To my best knowledge, there is no support for incremental assembly of LVM volumes. It is relatively simple to run script to scan disk for LVM and activate it - the problem is, it will not work in general case when volumes could span multiple devices. So your best bet is to find out how Ubuntu does it and reuse implementation. You could also ask upstream
    But that's probably the crux of my problem: I haven't figured out how to activate a logical volume from the command line. None of the LVM commands I've looked at seem appropriate. I can create/remove/resize physical volumes/volume groups/logical volumes but I can't figure out how to activate a logical volume. I thought that's what devmapper was for, but I haven't successfully grok'd dmsetup to get all of the symlinks created.

    I can understand how it might be problematic to create symlinks for an incomplete logical volume (not all physical volumes are present), but it seems to me that for those cases dm should just emit an error message (syslog?) and do the volumes that are complete.

    Thanks,
    ron

  5. #5
    Join Date
    Nov 2009
    Location
    West Virginia Sector 13
    Posts
    16,285

    Default Re: How do I get openSUSE to recognize/allocate LVM2 logical volumes on "hotplug"

    Have you simply tried using the installer to create the LVM? That normally works. You certainly don't need to pre-create a partition. You can't simply copy the OS from one environment to another without at lest editing /etc/fstab to the new IDs and running grub2-mkconfig

    To by honest I'm a little lost in you explanations. And not sure exactly what you want to accomplish

    But guessing What I would do is install fresh to the new drive then copy at least home over If you have a lot installed maybe copy all but /etc /proc /boot and /dev to new drive from old using a cd/dvd/usb boot IMO it would be easiest to just reinstall any apps you want. All important stuff should be in /home that is the data you really want brought over.

    In any case walk before you run get it running fist then worry about the bells and whistles

  6. #6
    Join Date
    Dec 2008
    Location
    Buffalo, MN USA
    Posts
    82

    Default Re: How do I get openSUSE to recognize/allocate LVM2 logical volumes on "hotplug"

    Quote Originally Posted by gogalthorp View Post
    Have you simply tried using the installer to create the LVM? That normally works. You certainly don't need to pre-create a partition. You can't simply copy the OS from one environment to another without at lest editing /etc/fstab to the new IDs and running grub2-mkconfig
    Agreed that the installer has no problem creating LVM partitions, however:

    Quote Originally Posted by gogalthorp View Post
    To by honest I'm a little lost in you explanations. And not sure exactly what you want to accomplish
    What I'm trying to accomplish is to minimize downtime on this file server by pre-replicating ~700GiB of data. I don't want to take the system down to swap drives and install, then configure a new OS and then have to wait several additional hours to replicate ~700 GiB of data before the system is actually up and useable. I have no intention of replicating any of the system files, just the data in the NFS and Samba shares.

    And I can do that now, after installing lvm2 on the kubuntu machine. This discussion is because I'd like to see the same functionality wrt hotplugging drives with lvm in openSUSE as already exists in ubuntu, and incidentally, Fedora. I just tried it on a Fedora machine and dm automatically created all of the /dev/dm-? nodes and symlinks when I powered up the external drive.

    Regards,
    ron

  7. #7
    Join Date
    Sep 2012
    Posts
    7,092

    Default Re: How do I get openSUSE to recognize/allocate LVM2 logical volumes on "hotplug"

    Quote Originally Posted by r_widell View Post
    But that's probably the crux of my problem: I haven't figured out how to activate a logical volume from the command line.
    Plug in disk; run "vgscan; vgchange -ay". Did you try it?

  8. #8
    Join Date
    Dec 2008
    Location
    Buffalo, MN USA
    Posts
    82

    Lightbulb Re: How do I get openSUSE to recognize/allocate LVM2 logical volumes on "hotplug"

    Quote Originally Posted by arvidjaar View Post
    Plug in disk; run "vgscan; vgchange -ay". Did you try it?
    No I hadn't, but I have now and that's the magical incantation I needed.
    Reminder to self: Brush up on your reading comprehension!

    Thanks arvidjaar. Although it would be nice if it happened automatically like on some other distros, I can work with this.

    Regards,
    ron

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •