Adding Physical volumes to an existing LUKS encrypted LVM to expand size?

Hello this is probably a somewhat advanced topic. I hope someone knows the answer.

I opted to go with the LUKS encrypted LVM setup using the Opensuse installer. I have:

LVG called system

/home (encrypted on system logical volume) dev-system-home (part of ‘system’ lvg)
/ (encrypted on system logical volume) dev-system-root (part of ‘system’ lvg)
swap (encrypted on system logical volume) dev-system-swap (part of ‘system’ lvg)
/boot (non encrypted boot partition) /dev/sdd1

The above are all on one 500 GB disk.

But I also have 3 other 80 GB disks which are unallocated. (actually I created another separate encrypted logical volume called data but let’s pretend I haven’t done that yet as I have no data on it and I can easily remove it – I’d rather have it all added to the existed encrypted logical volume if this is possible without a lot of hassle)

Is it possible to add these three disks and the new space to my existing encrypted logical volume? So that instead of ~500GB in my /home I would have ~700GB in /home?

I know that if it were simply an unencrypted LVM I could do this. I also know that if it was all on one disk I could easily do it because I did something like it before: https://forums.opensuse.org/english/get-technical-help-here/install-boot-login/472194-sketchy-lvm-encryption-seeking-documentation-pointers.html but now what about adding on three 80 GB disks like this? :slight_smile:


  --- Logical volume ---
  LV Name                /dev/system/home
  VG Name                system
  LV UUID                90bJD8-UKCV-ngHp-X4jU-e3xX-DSjh-kjomH8
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                425.00 GiB
  Current LE             108800
  Segments               2
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:1
   
  --- Logical volume ---
  LV Name                /dev/system/root
  VG Name                system
  LV UUID                y7iFhf-aWWt-fMu9-8nSu-PEV3-re3u-ACVnCr
  LV Write Access        read/write
  LV Status              available                                                                                                                                          
  # open                 1                                                                                                                                                  
  LV Size                20.00 GiB                                                                                                                                          
  Current LE             5120                                                                                                                                               
  Segments               1                                                                                                                                                  
  Allocation             inherit                                                                                                                                            
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:2
   
  --- Logical volume ---
  LV Name                /dev/system/swap
  VG Name                system
  LV UUID                BuGeKj-2wvA-zsEf-Tnl0-5922-T672-LLLytw
  LV Write Access        read/write
  LV Status              available
  # open                 2
  LV Size                2.00 GiB
  Current LE             512
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     1024
  Block device           253:3


So basically I would like to take disks /dev/sda + /dev/sdb + /dev/sdc (currently on a separate “data” LVG) and add them in the ‘system’ lvg instead and expand the ‘/dev/system/home’ encrypted LV.


  --- Physical volume ---
  PV Name               /dev/sda1
  VG Name               Data
  PV Size               74.56 GiB / not usable 2.00 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              19087
  Free PE               0
  Allocated PE          19087
  PV UUID               QOvRgJ-P7Kb-g6RI-OLpM-Wxfa-AGz5-AW3CyN
   
  --- Physical volume ---
  PV Name               /dev/sdb1
  VG Name               Data
  PV Size               76.33 GiB / not usable 2.00 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              19541
  Free PE               0
  Allocated PE          19541
  PV UUID               JjE1zY-u0dp-jz6T-VOic-c2UB-BO5O-BQA0wp
   
  --- Physical volume ---
  PV Name               /dev/sdc1
  VG Name               Data
  PV Size               74.53 GiB / not usable 2.00 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              19079
  Free PE               3
  Allocated PE          19076
  PV UUID               5nynBe-Jini-SmVf-02MW-xeGf-HSf2-6v7T1T
   
  --- Physical volume ---
  PV Name               /dev/mapper/cr_sdd2
  VG Name               system
  PV Size               465.61 GiB / not usable 0   
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              119195
  Free PE               4763
  Allocated PE          114432
  PV UUID               0kwAo1-9hu4-IpGz-ibYI-izio-ZDlU-vY09jS


Documentation on this scenario is light. Most suggestions seem to be that it isn’t possible and that it appears I am stuck either backing up and then repartitioning or just creating a separate LVM and mounting the extra space somewhere and encrypting it (as opposed to simply expanding my existing encrypted LVM to include the additional disks)

I hope I am not too confusing…

I don’t actually have the answer, and have never tried that.

As far as I know, LUKS is for a single partition (or a single file, if done that way).

When using an encrypted LVM, you first have to setup the LUKS partition, then unlock it with cryptsetup (which creates a virtual device), and then build the LVM on the virtual device.

To do that over several disks, it seems to me that you would have to independently create LUKS partitions on each disk (they could all use the same key), and then you could build a multi-device LVM out of those.

My best guess is that you can follow that method for what you are trying. Setup the LUKS on each of the disks. Make entries in “/etc/crypttab” for them. And then expand your LVM by adding the LUKS virtual devices.

The downside of this, is that you might be prompted 3 times for the encryption key (once for each component LUKS partition). If this is done in opensuse 12.2, and if the volume is opened during bootup, then the plymouth splash software should be asking for the key and will try the same key for each LUKS partition, so you should only be asked once. With an older opensuse version, you can force the LUKS setup to the initrd, and in that case you will again probably only be asked once for the key.

Again, I have never tried this. I’m just making untested suggestions.

I was able to create a separate lvg (logical volume group) called ‘data’ out of the 3 disks and then I created a single logical volume from them called data1 and in the YaST partitioner I simply checked the encrypted box for the new logical volume. This seems to work great and interestingly as long as the password is the same as my other lvg called ‘system’ it seems to only ask for the password once and is able to mount things fine.

But I’d rather delete this new lvg ‘data’ which I created (out of the 3 separate 80 GB disks) and add that on the existing lvg ‘system’ and then add it to the logical volume ‘/dev/system/home’ instead. This way I would have it all available on one volume. The trouble is that I’m not sure it can be done. I guess there is something like an “encrypted container” with which LUKS operates on. Once decrypted then it is a regular LVM LVG and works as normal from what I understand. My worry is that if I were to add on the existing LVG ‘system’ as I normally would if it were unencrypted (by first creating new physical volumes on the 3 disks, then allocating the space the lvg ‘system’, finally then growing the /dev/system/home’ partition as I did last time in this post: https://forums.opensuse.org/english/get-technical-help-here/install-boot-login/472194-sketchy-lvm-encryption-seeking-documentation-pointers.html) the “encrypted container” would no longer be valid to LUKS and it wouldn’t know what to do because suddenly now it is ~700GB whereas before it was ~500GB. That is my worry. That I will lose all data will no longer be able to decrypt.

I guess there is an advantage in creating a separate encrypted lvg as I did so far though. I could always backup important things to both encrypted LVGs so if one fails I still have access to the other. Int heory from what I understand if I use these four disks to form 1 encrypted LVG (assuming this is even possible to do encrypted, i know it is possible with LVM2 alone though) then if any of the disks fails I will lose all data unless I have other backups. Still it would be interesting to know if adding on to an existing LUKS encrypted LVG by using additional disks is possible. :\ :slight_smile:

I suppose I should learn more about how LUKS actually works with LVM. If it is flexible enough where we can add on additional disks on the fly then that is excellent and a great feature.

While it might be an oversimplification, I am inclined to say that LUKS does not know anything about LVM, and LVM does not know anything about LUKS. Both are sufficiently modular, that we can put them together. That they are independent of one another is part of what makes this all work.

I think this might help to better show what I have:

http://i.imgur.com/ZHHO2l.png](http://imgur.com/ZHHO2)

http://i.imgur.com/drSHql.png](http://imgur.com/drSHq)

You’ll notice that there are two LVG groups, ‘system’ and ‘Data’. ‘system’ is the one I had before on the 500 GB drive. ‘Data’ is the one I just created out of the three drives. From the first screenshot you’ll notice the encrypted icon that sort of looks like a padlock. For the first one, the ‘system’ LVG it is on the device /dev/sdd2 Type “Linux LVM” and on the second one ‘data’ I created it is on ‘/dev/Data/data1’ type LV.

So it looks to me like the way LUKS encrypted the first encrypted volume I made at install time, ‘system’ was by just encrypting the entire partition ‘/dev/sdd2’. Then when that is decrypted it is seen as a LVG ‘system’ with the /dev/system/root, /dev/system/swap, and /dev/system/home logical volumes on it all on one disk.

Based on that it looks to me like I cannot simply try to add on disks to the ‘system’ lvg because the way LUKS was setup at install time was to work with the partition /dev/sdd2. Because if I try to add in the other disks to the lvm ‘system’ it would no longer all be housed in /dev/sdd2. Is this correct?

But I’m thinking had openSUSE (and I the user) set up LUKS+LVM2 in the way I did with my new ‘data’ lvg (actually specifically /dev/Data/data1) you could potentially expand the LUKS encrypted lvm with additional disks since LUKS is looking for the logical volume ‘/dev/Data/data1’ as opposed to just a partition such as ‘/dev/sdd2’. Is this right? If so, I wonder why it doesn’t always do this so as to provide the most flexibility? I guess if we encrypted each logical volume separately it would be more management hassle (even though if they are all one password it seems to mount with one prompt) Just curious. I’m probably off on something. Just trying to learn. :slight_smile:

That makes sense. I posted some screenshots above and I think I understand more now too. Unfortunately it appears that with the way I initially setup my system (I think openSUSE suggested this for LUKS+LVM but I am not sure) it won’t work to add in additional drives to ‘system’ because LUKS is only encrypting/decrypting ‘/dev/sdd2’. But had it been set up where each logical volume were separately encrypted I theorize that I would be able to resize the logical volume by adding on additional disks since it is only looking for a logical volume rather than one specific partition or disk. I might be wrong though.

…I guess another question is whether LUKS expects the “container” to always be the same size. Would it balk if suddenly the logical volume tripled in size?

It’s pretty complex, hopefully someone actually knows. :slight_smile:

It is apparently possible to grow a LUKS container - or at least some web pages suggest so.

Here’s how I created the encrypted LVM that I am using on this system:

I booted a live KDE disk. Then running from the live system, I generated the partitioning I wanted (I think I used “fdisk” for that), and I set the partition type to LVM.

Next, I setup LUKS encryption. I don’t remember whether I used “cryptsetup” at the command line, or whether I used Yast for that. Not that it matters.

So now, I had a LUKS partition. I opened it with “cryptsetup luksOpen /dev/sda2 cr_lvm”.

That gave me a virtual device “/dev/mapper/cr_lvm”.

I thought about using command line lvm commands to create the lvm on that. But I’m inexperienced with it, so I instead used Yast partitioner to create the LVM, and create root, home and swap volumes in the LVM).

I presume you might be able to do something similar. Do a cryptsetup to open your second LVM as “/dev/mapper/something” (whatever you want “something” to be). And then see if you can add that virtual device to your existing LVM. It would probably add as raw unformatted space, and you might have to work at the command line.

Having said all of that, I’m going to wonder whether what you are currently doing isn’t better. A single larger LVM now becomes more likely to fail because there are two disks in it to fail. And the effects of the failure would be harder to isolate. So maybe your current setup is actually more robust anyway.

The next time I’m starting fresh (or when I have amble backup storage to mirror everything) I’m definitely going to experiment with these things to see exactly what’s possible and what is not in regards to LUKS and LVM2. I notice that there is a bit of mystery almost everywhere about the way it all works. We could definitely use a definitive and comprehensive easy to understand, up to date guide with most scenarios laid out assuming one isn’t already out there.

Having said all of that, I’m going to wonder whether what you are currently doing isn’t better. A single larger LVM now becomes more likely to fail because there are two disks in it to fail. And the effects of the failure would be harder to isolate. So maybe your current setup is actually more robust anyway.

That’s what I’m thinking as well. The three 80 GB drives are quite old. Probably 5+ years at least. So it might be asking for trouble to rely on them that much.

Thanks for your help in helping me understand all this. :slight_smile: