Page 1 of 2 12 LastLast
Results 1 to 10 of 18

Thread: Backing up KVMs

  1. #1

    Default Backing up KVMs

    Reviewing some past threads I read one common way to backup a VM is just copying the entire VM file; similar to VirtualBox except that here one copies an entire directory instead of a single file.

    Thing is, for almost obvious reasons, I created the VM in a different storage pool than the default one: a directory called "VirtualMachines" just inside home directory. Of course I manually created the directory beforehand.
    So inside is the VM file, but it was created with permissions "root:root -rw------". So by default it doesn't allow to copy the VM as normal user.

    1) Are such ownership and permissions certainly default behavior of virt-manager?

    2) What would be a "best practice" to be able to backup VM? Or is it that the only one supposed to do any KVM related stuff is root indeed?
    What if I need to copy directly to a USB stick or external hard disk? Wouldn't it be *highly insecure* to access any external storage as root?
    And for some reason I feel modifying ownerships and/or permissions would somewhat break virt-manager (not to mention some security flaws?)

    Thanks beforehand.

  2. #2

    Default Re: Backing up KVMs

    Excuse me, any help?
    Sorry for this, but I'm in kind of a hurry actually...

  3. #3
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    13,295
    Blog Entries
    2

    Default Re: Backing up KVMs

    Same applies to all virtualization technologies, there is no difference for example between VBox and KVM... The directory will contain <all> the necessary files plus possibly some unnecessary to run. Copying only a diskfile is possible, but did you copy all the diskfiles if there is one or more snapshots? BTW - Is one reason to merge snapshots ASAP when they're no longer needed... many times snapshots won't be recoverable or read in a recovery of any type.

    At is most basic you have
    The all-important disk file(s).
    The configuration file

    Then you have all those other files which may or may not be important like memory files, lock files, more.

    You can also
    - Clone a machine. When you do this, you create a copy of your machine but with different settings which can run simultaneously with the original on the same network. But, of course since it will have its own identity, dependent applications won't run the same or need modification.

    - You can do ordinary backups within the Guest
    - You can do ordinary backups on the HostOS pointing at the Guests.

    Lots and lots of choices, and they all work.
    The important question isn't what is the Best Backup method, but what backup methods should be chosen that are consistent with your Backup Strategy, and <that> should be defined by your understanding of what <is> a recommended Backup Strategy according to best practices.

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  4. #4

    Default Re: Backing up KVMs

    It would seem as if you only read exclusively the "best practices" part of my first post. Did you even read question 1) ? Because question 2) had *direct* base on first one... at least that's how I meant

    How do you even make backups "from guest itself", "from host to guest" or whatever? What are you talking about!?

    All virtualization technologies being the same? WTH? That would sound illogical.
    For example, I currently have just one KVM guest, with no snapshots, and inside the KVM storage pool directory I only see one file: the qcow2 one itself. Probably all VM settings are stored within this file as well? If not, is it that settings get stored somewhere else? In both cases KVM is NOT the same as VirtualBox, which creates different separate files for storage disk, settings, xml, etc, all under a same folder!

    Regarding snapshots,
    https://forums.opensuse.org/showthre...53#post2848753
    From what I read in other sites, internal snapshots *don't* ever create any separate files from the original virtual disk file, not even with qcow2. All snapshots are stored within same qcow2 file itself. I would certainly prefer the separate files though...

  5. #5
    Join Date
    Jul 2008
    Location
    Seattle, WA
    Posts
    17,317

    Default Re: Backing up KVMs

    On Fri, 22 Dec 2017 22:56:02 +0000, F style wrote:

    > It would seem as if you only read exclusively the "best practices" part
    > of my first post. Did you even read question 1) ? Because question 2)
    > had *direct* base on first one... at least that's how I meant
    >
    > How do you even make backups "from guest itself", "from host to guest"
    > or whatever? What are you talking about!?
    >
    > All virtualization technologies being the same? WTH? That would sound
    > illogical.
    > For example, I currently have just one KVM guest, with no snapshots, and
    > inside the KVM storage pool directory I only see one file: the qcow2 one
    > itself. Probably all VM settings are stored within this file as well? If
    > not, is it that settings get stored somewhere else? In both cases KVM is
    > NOT the same as VirtualBox, which creates different separate files for
    > storage disk, settings, xml, etc, all under a same folder!
    >
    > Regarding snapshots,
    > http://tinyurl.com/yadwossw From what I read in other sites, internal
    > snapshots *don't* ever create any separate files from the original
    > virtual disk file, not even with qcow2. All snapshots are stored within
    > same qcow2 file itself.


    Let's not be argumentative with someone who's trying to help you. Relax.

    Jim
    --
    Jim Henderson
    openSUSE Forums Administrator
    Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

  6. #6
    Join Date
    Jun 2008
    Location
    Groningen, Netherlands
    Posts
    20,925
    Blog Entries
    14

    Default Re: Backing up KVMs

    Quote Originally Posted by F_style View Post
    Excuse me, any help?
    Sorry for this, but I'm in kind of a hurry actually...
    Please be aware that this is not some corporate business customer support centre, but a 100% volunteer community. Which means people try to help you and others in their own spare time. They deserve a better approach than you're showing here.

    If you want 24/7 support, that's availble for SUSE Linux Enterprise, in various SLA's.
    ° Appreciate my reply? Click the star and let me know why.

    ° Perfection is not gonna happen. No way.

    http://en.opensuse.org/User:Knurpht
    http://nl.opensuse.org/Gebruiker:Knurpht

  7. #7
    Join Date
    May 2012
    Location
    Finland
    Posts
    2,188

    Default Re: Backing up KVMs

    One way to backup KVM;
    - Snapshot
    - Backup the original file
    - Reintegrate the snapshot into the original file

    One that I know works;
    https://mangolassi.it/topic/12537/kv...ckup-script/31

    Another one:
    https://github.com/michaltrzcinski/kvm-snapshot-backup

    And like the other guys said, we're kinda sorta "help when we have time" kind of forum and demanding stuff from people here will usually get you blacklisted by 'those who know stuff' or just otherwise shunned.
    .: miuku @ #opensuse @ irc.libera.chat

  8. #8
    Join Date
    Jun 2008
    Location
    San Diego, Ca, USA
    Posts
    13,295
    Blog Entries
    2

    Default Re: Backing up KVMs

    Quote Originally Posted by F_style View Post
    It would seem as if you only read exclusively the "best practices" part of my first post. Did you even read question 1) ? Because question 2) had *direct* base on first one... at least that's how I meant

    How do you even make backups "from guest itself", "from host to guest" or whatever? What are you talking about!?

    All virtualization technologies being the same? WTH? That would sound illogical.
    For example, I currently have just one KVM guest, with no snapshots, and inside the KVM storage pool directory I only see one file: the qcow2 one itself. Probably all VM settings are stored within this file as well? If not, is it that settings get stored somewhere else? In both cases KVM is NOT the same as VirtualBox, which creates different separate files for storage disk, settings, xml, etc, all under a same folder!

    Regarding snapshots,
    https://forums.opensuse.org/showthre...53#post2848753
    From what I read in other sites, internal snapshots *don't* ever create any separate files from the original virtual disk file, not even with qcow2. All snapshots are stored within same qcow2 file itself. I would certainly prefer the separate files though...
    From within a running Guest, you can run a backup application.
    Not within a Guest, the HostOS has access to the locations where your Guest files are stored, so you can run a backup application in the HostOS pointing to the overall or any part of your storage location to back up those files.

    Regarding your observation that you don't see more than the diskfile in KVM, it's not because those files don't exist... They're just stored elsewhere. Libvirt gathers up all the necessary files to run a Guest when necessary. Think about it a second... When you inspect a Guest's configuration in virt manager, those settings have to be stored somewhere, and it's not in the disk file.

    As for snapshots and disk files, I'd advise you to experiment and see for yourself what happens.

    The following does not apply to Xen, it stands alone in its unique architecture.
    But for everything else, read on...

    For those that might want to take the time to learn about more than one virtualization technology, and actually use them you will find that generally all virtualization are somewhat similar, particularly since all 64-bit virtualization today requires CPU extensions (hardware assist). That by itself means that the biggest part of how virtualization works today is using common functionality. Each virtualization will try to tweak a few things to do things better, but those are modifications around the edges. This means that no matter what virtualization technology you choose (except for Xen), the overall architecture is largely the same and you can also expect similar performance.

    You should know that we are in the middle of an evolution of I/O virtualization (including graphics). One day it's envisioned that I/O virtualization will become standardized similar to how CPU virtualization is, but today it's still one of those things that can be different between virtualization technologies. There can be notable differences, like a recent thread on VMware graphical performance using a WebGL benchmark.

    So, at least today when you compare all non-Xen virtualization, you should focus on other differentiators, primarily
    - Usability. Although the virtualization is generally same, every virt technology provides a different set of User mode tools which directly affects your management and maintenance experience.
    - Features. Although related to Usability, going down the feature list of each virtualization clarifies in detail why you might prefer one over the other.
    - Support. Some provide special support, others don't. Some will patch, maintain and improve their User mode tools better, others won't. Some are backed only by community, others involve sponsors or are proprietary.
    - Cost. If you can find something at no cost that works fine for you, great. If you want the assurance of paid support, then you can pay for whatever might be worthwhile to you.

    Hope this clarifies what I mean that depending on what you're looking at, there might be little or no difference what virtualization technology you choose depending on what you're looking at.

    TSU
    Beginner Wiki Quickstart - https://en.opensuse.org/User:Tsu2/Quickstart_Wiki
    Solved a problem recently? Create a wiki page for future personal reference!
    Learn something new?
    Attended a computing event?
    Post and Share!

  9. #9

    Default Re: Backing up KVMs

    Ok, thanks everyone and sorry.

    But I still wonder if you could help with the permissions "issue" please.
    Let me try to rewrite it again in attempt for a bit better clarity.

    I created the VM in a different storage pool than the default one: a directory called "VirtualMachines" just inside home directory.
    So inside is the VM disk file, but it was created with permissions "root:root -rw------". So by default it doesn't allow to copy the VM as normal user (being copying entire VM the preferred way of backup for now since I was taught so back when using VirtualBox).

    1) Are such ownership and permissions certainly default behavior of virt-manager? Is libvirt in general supposed to be used mostly as root just like virsh?

    2) What if I need to copy directly to a USB stick or external hard disk? Wouldn't it be *highly insecure* to access any external storage with a root shell?

    3) Would modifying VM file's ownerships and/or permissions somewhat break VM itself or virt-manager (not to mention some security flaws?) I.e., I would like to add the VM file only read permissions for group and others just to be able to copy as normal user.

    Thanks again.

  10. #10
    Join Date
    Jun 2008
    Location
    Podunk
    Posts
    32,357
    Blog Entries
    15

    Default Re: Backing up KVMs

    On Tue 26 Dec 2017 08:06:02 PM CST, F style wrote:

    Ok, thanks everyone and sorry.

    But I still wonder if you could help with the permissions "issue"
    please.
    Let me try to rewrite it again in attempt for a bit better clarity.

    I created the VM in a different storage pool than the default one: a
    directory called "VirtualMachines" just inside home directory.
    So inside is the VM disk file, but it was created with permissions
    "root:root -rw------". So by default it doesn't allow to copy the VM as
    normal user (being copying entire VM the preferred way of backup for
    now since I was taught so back when using VirtualBox).

    1) Are such ownership and permissions certainly default behavior of
    virt-manager? Is libvirt in general supposed to be used mostly as root
    just like virsh?

    2) What if I need to copy directly to a USB stick or external hard disk?
    Wouldn't it be *highly insecure* to access any external storage with a
    root shell?

    3) Would modifying VM file's ownerships and/or permissions somewhat
    break VM itself or virt-manager (not to mention some security flaws?)
    I.e., I would like to add the VM file only read permissions for group
    and others just to be able to copy as normal user.

    Thanks again.
    Hi
    1) Yes, iso images in the storage pool/volume are owned by qemu:qemu,
    the qcow2 images are owned by root:root. The virt-manager and virsh are
    expected to be used by root.... To connect to the vm use virt-viewer as
    your user....

    For example, for my remote kvm system (running Tumbleweed) I use (all as
    my user);
    Code:
    virsh -c qemu+ssh://root@oscar.homelinux.org/system list --all
    Id    Name                           State
    ----------------------------------------------------
    -     openSUSE_Leap_42.3             shut off
    -     openSUSE_Tumbleweed            shut off
    -     Solaris_11_3                   shut off
    -     Windows_10                     shut off
    -     Windows_7_SP1                  shut off
    -     Windows_8.1                    shut off
    
    virsh -c qemu+ssh://root@oscar.homelinux.org/system start Solaris_11_3
    Domain Solaris_11_3 started
    
    virt-viewer -c qemu+ssh://oscar.homelinux.org/system Solaris_11_3


    2) Just copy -ar /rsync/sftp etc to backup, as root user, ext4 or xfs
    for your storage medium, depends on your requirements, encrypt,
    password protect etc, but just the copy function is fine to you backup
    medium. As always, make a second backup and keep off-site.

    3) Not for a backup, just need to sure the images (qcow2) are correct if
    copied back to the machine, root:root and 0600 permissions.

    But you should be backing up the whole /var/lib/libvirt directories
    as well for snapshot xml files etc. These won't be covered if you just
    backup your other added storage pools. I always run /var/lib/libvirt on
    it's own disk makes it easy to backup, replace etc.

    --
    Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
    openSUSE Leap 42.3|GNOME 3.20.2|4.4.103-36-default
    If you find this post helpful and are logged into the web interface,
    please show your appreciation and click on the star below... Thanks!


Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •