virt-clone create non-sparse volumes

Hello all

I am using KVM/Qemu for testing purpose, and so the possibility to have dynamic allocations for VMs storage is very useful to me, because i create many, I want to return back to most later, I don’t need them to be extremely fast and reliable, but I need them not to take too much space

By default, virt-clone is supposed to create such dynamically allocated volumes in qcow2 format, and there is a --nonsparse option to do the opposite when needed. The problem is on my freshly installed Leap 15.1, every time I clone a disk, no matter it is with virt-manager or with virt-clone and the original disk is dynamically allocated, I always end with a fully allocated disk files that I can’t keep because of the huge amount of space it wastes on my HD, and there is no --sparse option in virt-clone to return to what should be the default behaviour

Actually, I work around the issue by creating disk files by hand with qemu-img or on another libvirt from another distribution, and when I need a clone on OpenSuse, I make a copy of the original file, and then I create a new VM using it, but it would be nicer to be able to use the provided tools instead

Does anyone knows how to return back to the normal behaviour ? I looked in various configuration files in /etc, but didn’t find anything useful…

Thanks in advance, if you have an idea

Hi and welcome to the Forum :slight_smile:
AFAIK the convert command should recompress the image, for example;


cp image1.qcow2 image1.qcow2_backup
qemu-img convert -O qcow2 image1.qcow2_backup image2.qcow2

Yes,
But you should also know how to compress disks properly…
After emptying the trash and anything else that retains deleted files,

You have to zero all the empty space on your disk to remove artifacts, file fragments, data “ghosts” and so on…
This is because a fundamental requirement of compression is that the compressed space must contain some pattern that can be represented by something much smaller, and the simplest “pattern” is a null.
Then, you can further improve your compression by defragging your data, making the data all contiguous so that your free space is also contiguous.

Only then should you compress your disk.

Depending on your objectives, the above can also be recommended whenever you clone your disks.

TSU

Thank you very much malcomlewis. It is good to know, and perhaps Clone + Compress may be less work than Copy + New VM. I hope there is a way to have VM disks sparsed by default as it was supposed to be, by a way or another though. It would be much easier and natural to work with

Thank you tsu2. Of course, cleaning as much as possible of the filesystem can make the result way more efficient, but just starting with a clean sparse volume do 50-75% of the job in my case, which is a lot already. I could think of you advice for keeping a VM in a safe place for a long time, but compressed in LZMA or something close, then…

A Clone operation as always is a byte by byte copy, without regard to what that byte is or represents.
Therefor, the copy is exactly identical to the original, and if the original is not compressed already the copy will not be any different.

TSU

Hi
Um, sure run it on the original, seems prudent to me to create a backup and use that… but whatever…

The convert process does compress the image.

Use overlays, or backing images as qemu calls them. That is you create a base image with Opensuse and once you are happy with it you put that away and only use new overlay images that reference your base image as backing image. Here is the idea


# my fine and huge image
# opensuse.qcow2
#
mv opensuse.qcow2 opensuse-base.qcow2
qemu-img create -b opensuse-base.qcow2 -f qcow2 opensuse.qcow2

# you do not even have to change the VM specs as it will happily use the overlay image
# cloning will also clone the overlay, not the base (I think, have not tried recently)
# just never fire up the base image in a vm as that might break the overlays
# you can commit (merge) an overlay into the base, which obviously breaks other overlays, so merge copies and give them new names

So, I make a habit of never firing up a base image in a VM. I always create overlay images and place those in VMs. That might help you with your cloning/space probs.

Now, if I could only get terraform-libvirt to create overlays (which it can do) without cloning the base too…