Results 1 to 7 of 7

Thread: virt-clone create non-sparse volumes

  1. #1

    Default virt-clone create non-sparse volumes

    Hello all

    I am using KVM/Qemu for testing purpose, and so the possibility to have dynamic allocations for VMs storage is very useful to me, because i create many, I want to return back to most later, I don't need them to be extremely fast and reliable, but I need them not to take too much space

    By default, virt-clone is supposed to create such dynamically allocated volumes in qcow2 format, and there is a --nonsparse option to do the opposite when needed. The problem is on my freshly installed Leap 15.1, every time I clone a disk, no matter it is with virt-manager or with virt-clone and the original disk is dynamically allocated, I always end with a fully allocated disk files that I can't keep because of the huge amount of space it wastes on my HD, and there is no --sparse option in virt-clone to return to what should be the default behaviour

    Actually, I work around the issue by creating disk files by hand with qemu-img or on another libvirt from another distribution, and when I need a clone on OpenSuse, I make a copy of the original file, and then I create a new VM using it, but it would be nicer to be able to use the provided tools instead

    Does anyone knows how to return back to the normal behaviour ? I looked in various configuration files in /etc, but didn't find anything useful...

    Thanks in advance, if you have an idea

  2. #2
    Join Date
    Jun 2008
    Location
    Podunk
    Posts
    26,499
    Blog Entries
    15

    Default Re: virt-clone create non-sparse volumes

    Hi and welcome to the Forum
    AFAIK the convert command should recompress the image, for example;

    Code:
    cp image1.qcow2 image1.qcow2_backup
    qemu-img convert -O qcow2 image1.qcow2_backup image2.qcow2
    Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
    SUSE SLE, openSUSE Leap/Tumbleweed (x86_64) | GNOME DE
    If you find this post helpful and are logged into the web interface,
    please show your appreciation and click on the star below... Thanks!

  3. #3
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    10,923
    Blog Entries
    2

    Default Re: virt-clone create non-sparse volumes

    Yes,
    But you should also know how to compress disks properly...
    After emptying the trash and anything else that retains deleted files,

    You have to zero all the empty space on your disk to remove artifacts, file fragments, data "ghosts" and so on...
    This is because a fundamental requirement of compression is that the compressed space must contain some pattern that can be represented by something much smaller, and the simplest "pattern" is a null.
    Then, you can further improve your compression by defragging your data, making the data all contiguous so that your free space is also contiguous.

    Only then should you compress your disk.

    Depending on your objectives, the above can also be recommended whenever you clone your disks.

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  4. #4

    Default Re: virt-clone create non-sparse volumes

    Quote Originally Posted by malcolmlewis View Post
    Hi and welcome to the Forum
    AFAIK the convert command should recompress the image, for example;

    Code:
    cp image1.qcow2 image1.qcow2_backup
    qemu-img convert -O qcow2 image1.qcow2_backup image2.qcow2
    Thank you very much malcomlewis. It is good to know, and perhaps Clone + Compress may be less work than Copy + New VM. I hope there is a way to have VM disks sparsed by default as it was supposed to be, by a way or another though. It would be much easier and natural to work with

    Quote Originally Posted by tsu2
    Yes,
    But you should also know how to compress disks properly...
    After emptying the trash and anything else that retains deleted files,
    Thank you tsu2. Of course, cleaning as much as possible of the filesystem can make the result way more efficient, but just starting with a clean sparse volume do 50-75% of the job in my case, which is a lot already. I could think of you advice for keeping a VM in a safe place for a long time, but compressed in LZMA or something close, then...

  5. #5
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    10,923
    Blog Entries
    2

    Default Re: virt-clone create non-sparse volumes

    Quote Originally Posted by Arkem View Post
    Thank you very much malcomlewis. It is good to know, and perhaps Clone + Compress may be less work than Copy + New VM. I hope there is a way to have VM disks sparsed by default as it was supposed to be, by a way or another though. It would be much easier and natural to work with
    A Clone operation as always is a byte by byte copy, without regard to what that byte is or represents.
    Therefor, the copy is exactly identical to the original, and if the original is not compressed already the copy will not be any different.

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  6. #6
    Join Date
    Jun 2008
    Location
    Podunk
    Posts
    26,499
    Blog Entries
    15

    Default Re: virt-clone create non-sparse volumes

    Quote Originally Posted by tsu2 View Post
    A Clone operation as always is a byte by byte copy, without regard to what that byte is or represents.
    Therefor, the copy is exactly identical to the original, and if the original is not compressed already the copy will not be any different.

    TSU
    Hi
    Um, sure run it on the original, seems prudent to me to create a backup and use that... but whatever...

    The convert process does compress the image.
    Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
    SUSE SLE, openSUSE Leap/Tumbleweed (x86_64) | GNOME DE
    If you find this post helpful and are logged into the web interface,
    please show your appreciation and click on the star below... Thanks!

  7. #7

    Default Re: virt-clone create non-sparse volumes

    Use overlays, or backing images as qemu calls them. That is you create a base image with Opensuse and once you are happy with it you put that away and only use new overlay images that reference your base image as backing image. Here is the idea

    Code:
    # my fine and huge image
    # opensuse.qcow2
    #
    mv opensuse.qcow2 opensuse-base.qcow2
    qemu-img create -b opensuse-base.qcow2 -f qcow2 opensuse.qcow2
    
    # you do not even have to change the VM specs as it will happily use the overlay image
    # cloning will also clone the overlay, not the base (I think, have not tried recently)
    # just never fire up the base image in a vm as that might break the overlays
    # you can commit (merge) an overlay into the base, which obviously breaks other overlays, so merge copies and give them new names
    So, I make a habit of never firing up a base image in a VM. I always create overlay images and place those in VMs. That might help you with your cloning/space probs.

    Now, if I could only get terraform-libvirt to create overlays (which it can do) without cloning the base too...

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •