Reviewing some past threads I read one common way to backup a VM is just copying the entire VM file; similar to VirtualBox except that here one copies an entire directory instead of a single file.
Thing is, for almost obvious reasons, I created the VM in a different storage pool than the default one: a directory called “VirtualMachines” just inside home directory. Of course I manually created the directory beforehand.
So inside is the VM file, but it was created with permissions “root:root -rw------”. So by default it doesn’t allow to copy the VM as normal user.
Are such ownership and permissions certainly default behavior of virt-manager?
What would be a “best practice” to be able to backup VM? Or is it that the only one supposed to do any KVM related stuff is root indeed?
What if I need to copy directly to a USB stick or external hard disk? Wouldn’t it be highly insecure to access any external storage as root?
And for some reason I feel modifying ownerships and/or permissions would somewhat break virt-manager (not to mention some security flaws?)
Same applies to all virtualization technologies, there is no difference for example between VBox and KVM… The directory will contain <all> the necessary files plus possibly some unnecessary to run. Copying only a diskfile is possible, but did you copy all the diskfiles if there is one or more snapshots? BTW - Is one reason to merge snapshots ASAP when they’re no longer needed… many times snapshots won’t be recoverable or read in a recovery of any type.
At is most basic you have
The all-important disk file(s).
The configuration file
Then you have all those other files which may or may not be important like memory files, lock files, more.
You can also
Clone a machine. When you do this, you create a copy of your machine but with different settings which can run simultaneously with the original on the same network. But, of course since it will have its own identity, dependent applications won’t run the same or need modification.
You can do ordinary backups within the Guest
You can do ordinary backups on the HostOS pointing at the Guests.
Lots and lots of choices, and they all work.
The important question isn’t what is the Best Backup method, but what backup methods should be chosen that are consistent with your Backup Strategy, and <that> should be defined by your understanding of what <is> a recommended Backup Strategy according to best practices.
It would seem as if you only read exclusively the “best practices” part of my first post. Did you even read question 1) ? Because question 2) had direct base on first one… at least that’s how I meant
How do you even make backups “from guest itself”, “from host to guest” or whatever? What are you talking about!?
All virtualization technologies being the same? WTH? That would sound illogical.
For example, I currently have just one KVM guest, with no snapshots, and inside the KVM storage pool directory I only see one file: the qcow2 one itself. Probably all VM settings are stored within this file as well? If not, is it that settings get stored somewhere else? In both cases KVM is NOT the same as VirtualBox, which creates different separate files for storage disk, settings, xml, etc, all under a same folder!
On Fri, 22 Dec 2017 22:56:02 +0000, F style wrote:
> It would seem as if you only read exclusively the “best practices” part
> of my first post. Did you even read question 1) ? Because question 2)
> had direct base on first one… at least that’s how I meant
>
> How do you even make backups “from guest itself”, “from host to guest”
> or whatever? What are you talking about!?
>
> All virtualization technologies being the same? WTH? That would sound
> illogical.
> For example, I currently have just one KVM guest, with no snapshots, and
> inside the KVM storage pool directory I only see one file: the qcow2 one
> itself. Probably all VM settings are stored within this file as well? If
> not, is it that settings get stored somewhere else? In both cases KVM is
> NOT the same as VirtualBox, which creates different separate files for
> storage disk, settings, xml, etc, all under a same folder!
>
> Regarding snapshots,
> http://tinyurl.com/yadwossw From what I read in other sites, internal
> snapshots don’t ever create any separate files from the original
> virtual disk file, not even with qcow2. All snapshots are stored within
> same qcow2 file itself.
Let’s not be argumentative with someone who’s trying to help you. Relax.
Please be aware that this is not some corporate business customer support centre, but a 100% volunteer community. Which means people try to help you and others in their own spare time. They deserve a better approach than you’re showing here.
If you want 24/7 support, that’s availble for SUSE Linux Enterprise, in various SLA’s.
And like the other guys said, we’re kinda sorta “help when we have time” kind of forum and demanding stuff from people here will usually get you blacklisted by ‘those who know stuff’ or just otherwise shunned.
From within a running Guest, you can run a backup application.
Not within a Guest, the HostOS has access to the locations where your Guest files are stored, so you can run a backup application in the HostOS pointing to the overall or any part of your storage location to back up those files.
Regarding your observation that you don’t see more than the diskfile in KVM, it’s not because those files don’t exist… They’re just stored elsewhere. Libvirt gathers up all the necessary files to run a Guest when necessary. Think about it a second… When you inspect a Guest’s configuration in virt manager, those settings have to be stored somewhere, and it’s not in the disk file.
As for snapshots and disk files, I’d advise you to experiment and see for yourself what happens.
The following does not apply to Xen, it stands alone in its unique architecture.
But for everything else, read on…
For those that might want to take the time to learn about more than one virtualization technology, and actually use them you will find that generally all virtualization are somewhat similar, particularly since all 64-bit virtualization today requires CPU extensions (hardware assist). That by itself means that the biggest part of how virtualization works today is using common functionality. Each virtualization will try to tweak a few things to do things better, but those are modifications around the edges. This means that no matter what virtualization technology you choose (except for Xen), the overall architecture is largely the same and you can also expect similar performance.
You should know that we are in the middle of an evolution of I/O virtualization (including graphics). One day it’s envisioned that I/O virtualization will become standardized similar to how CPU virtualization is, but today it’s still one of those things that can be different between virtualization technologies. There can be notable differences, like a recent thread on VMware graphical performance using a WebGL benchmark.
So, at least today when you compare all non-Xen virtualization, you should focus on other differentiators, primarily
Usability. Although the virtualization is generally same, every virt technology provides a different set of User mode tools which directly affects your management and maintenance experience.
Features. Although related to Usability, going down the feature list of each virtualization clarifies in detail why you might prefer one over the other.
Support. Some provide special support, others don’t. Some will patch, maintain and improve their User mode tools better, others won’t. Some are backed only by community, others involve sponsors or are proprietary.
Cost. If you can find something at no cost that works fine for you, great. If you want the assurance of paid support, then you can pay for whatever might be worthwhile to you.
Hope this clarifies what I mean that depending on what you’re looking at, there might be little or no difference what virtualization technology you choose depending on what you’re looking at.
But I still wonder if you could help with the permissions “issue” please.
Let me try to rewrite it again in attempt for a bit better clarity.
I created the VM in a different storage pool than the default one: a directory called “VirtualMachines” just inside home directory.
So inside is the VM disk file, but it was created with permissions “root:root -rw------”. So by default it doesn’t allow to copy the VM as normal user (being copying entire VM the preferred way of backup for now since I was taught so back when using VirtualBox).
Are such ownership and permissions certainly default behavior of virt-manager? Is libvirt in general supposed to be used mostly as root just like virsh?
What if I need to copy directly to a USB stick or external hard disk? Wouldn’t it be highly insecure to access any external storage with a root shell?
Would modifying VM file’s ownerships and/or permissions somewhat break VM itself or virt-manager (not to mention some security flaws?) I.e., I would like to add the VM file only read permissions for group and others just to be able to copy as normal user.
On Tue 26 Dec 2017 08:06:02 PM CST, F style wrote:
Ok, thanks everyone and sorry.
But I still wonder if you could help with the permissions “issue”
please.
Let me try to rewrite it again in attempt for a bit better clarity.
I created the VM in a different storage pool than the default one: a
directory called “VirtualMachines” just inside home directory.
So inside is the VM disk file, but it was created with permissions
“root:root -rw------”. So by default it doesn’t allow to copy the VM as
normal user (being copying entire VM the preferred way of backup for
now since I was taught so back when using VirtualBox).
Are such ownership and permissions certainly default behavior of
virt-manager? Is libvirt in general supposed to be used mostly as root
just like virsh?
What if I need to copy directly to a USB stick or external hard disk?
Wouldn’t it be highly insecure to access any external storage with a
root shell?
Would modifying VM file’s ownerships and/or permissions somewhat
break VM itself or virt-manager (not to mention some security flaws?)
I.e., I would like to add the VM file only read permissions for group
and others just to be able to copy as normal user.
Thanks again.
Hi
Yes, iso images in the storage pool/volume are owned by qemu:qemu,
the qcow2 images are owned by root:root. The virt-manager and virsh are
expected to be used by root… To connect to the vm use virt-viewer as
your user…
For example, for my remote kvm system (running Tumbleweed) I use (all as
my user);
virsh -c qemu+ssh://root@oscar.homelinux.org/system list --all
Id Name State
----------------------------------------------------
- openSUSE_Leap_42.3 shut off
- openSUSE_Tumbleweed shut off
- Solaris_11_3 shut off
- Windows_10 shut off
- Windows_7_SP1 shut off
- Windows_8.1 shut off
virsh -c qemu+ssh://root@oscar.homelinux.org/system start Solaris_11_3
Domain Solaris_11_3 started
virt-viewer -c qemu+ssh://oscar.homelinux.org/system Solaris_11_3
Just copy -ar /rsync/sftp etc to backup, as root user, ext4 or xfs
for your storage medium, depends on your requirements, encrypt,
password protect etc, but just the copy function is fine to you backup
medium. As always, make a second backup and keep off-site.
Not for a backup, just need to sure the images (qcow2) are correct if
copied back to the machine, root:root and 0600 permissions.
But you should be backing up the whole /var/lib/libvirt directories
as well for snapshot xml files etc. These won’t be covered if you just
backup your other added storage pools. I always run /var/lib/libvirt on
it’s own disk makes it easy to backup, replace etc.
–
Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
openSUSE Leap 42.3|GNOME 3.20.2|4.4.103-36-default
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!
I think this is totally related to a previous post of mine, even though it was answered couple of days later. Virt-manager alone can be started as normal user, but in order to “connect” to the VMs I need to enter root password. I use this GUI way for now because I’m still beginning with KVM and still need to learn how to use command line in virtualization.
Now, from your example, does it mean virsh can actually be run as normal user as well, instead of just by root? Or does it work just for remote connections? Mine is not a remote connection, it’s local KVM/qemu!
I was really fearing what tsu2 mentioned in that same thread was true: KVM/qemu/libvirt is designed to run as a server rather than an application, and only managed by root… So this is why they just launched a Linux version of VirtualBox for normal people?
But isn’t it insecure to access any external storage with a root shell?
So if I first change permissions (even if just adding read permissions) to original VM disk image and files inside /var/lib/libvirt in order to create backup by copying as normal user, do I indeed break stuff? Does it fix by just setting all permissions back to how exactly they were?
Hi
As you user use virt-viewer (no need to enter root password), as long as the vm instance has been started as root (virsh or virt-manager), then you can leave running in the background and connect (as your user) with virt-viewer any time.
Virtualbox still needs maintenance as root user, rebuilding kernel modules, waiting for fixes when it doesn’t work with a new kernel etc…
Why/how? How does that differ from the current storage that is attached? Leave ownerships as they are, sure you can change if you want to on the backup medium, but it will serve no purpose, since you will need to change back if the backup is needed.
You need to be root user esp in /var/lib/libvirt/images, backup as root and press on…
Wait, I think I found where the misunderstanding is, or rather, I missed some info. Just as I tried to imply in this other post, I always start the VM, I never ever leave any VM to autostart! This is a laptop, for use as a normal desktop user, not a server!
The only directories with stuff inside are dnsmasq and qemu. The dnsmasq one has some dns, host and virbr stuff… perhaps all virtual networking stuff is stored here?
The qemu directory I was able to access only as root, guess libvirt does not put normal user inside qemu group by default…
Snapshots seem to go to /var/lib/libvirt/qemu/snapshots, but how is it possible? I had read libvirt currently supported only “internal” snapshots, which integrate the snapshot files within VM disk file itself instead of making files apart…
Hi
If you backup the full structure and files in /var/lib/libvirt then it’s easy to get things back rather than rely on specific directories or files in case you miss one with your backup.
Then carry on running with virt-manager then, just if you start/stop it will require you root password… server/laptop it doesn’t matter I just script it all to make life easier to start/shutdown/destroy the instance either on the local machine of remote(+ssh)…
If you want to make a backup that can be restored either on the local or another machine, minimally all that is required is the disk file(s). The catch is that to restore the <full> machine, you really do need to include the entire set of disk files, and unmerged snapshots can be a problem.
So, general rule based on KISS, is to minimize the “set” of disk files, with the ultimate solution being a single disk file only.
Another basic rule is that if you don’t include the “other” files (particularly the configuration files) in your backup, then you have to re-create them. Re-creating configurations are considered non-critical, and if you are re-deploying on a machine with different resources, then could be necessary anyway (older settings may not be valid). But, if you’re restoring to the same machine, then it can be a convenience not to have to re-create (typically create a new machine using an existing disk file(s)).
It should also be noted that if you try to copy an active (running) VM’s disk files, you could run into internal integrity issues. So, only disk files of shut down machines should be copied.
From what has been posted in this thread, I suspect that the desired objective is to create a backup which can be restored to the same machine with fewest complications. For this, I’d suggest not re-inventing the wheel and doing as little manually as possible.
So, bottom line is that although copying disk files can be done, beware the issues involved.
To avoid issues involved with crude disk file copying,
I recommend you consider any of the many scripts and/or solutions you’d get if you Google “libvirt backup”
Or, as I originally mentioned you can typically use regular backup applications.
If it’s worth clarifying, objective was always certainly to backup based on KISS rules: merging/deleting all snapshots and shutting down VM before making backup. Backing up config files would certainly depend on the case, whether same PC or another different one.
But, a doubt that I have always had, was KVM/qemu/libvirt really generally designed to run as a server rather than an application (thus only managed by root), despite nevertheless being able to be used as a normal user application as well?
My general impression is that libvirt with KVM is designed to support Enterprise Server deployment fully.
This means that unlike for instance Virtualbox,
In a Production “Business” deployment, you should consider installing a dedicated, single purpose server with minimal attack surface which should serve you well.
Again comparing to Virtualbox, it should not be difficult to run KVM as a server service, and secure by default (but don’t take anything for granted. Inspect everything and modify as you think fit).
