I am not sure if this is the right place to ask these questions. But there are no “cafe corner” on this forum and this is the closest subcategory of relevance :).
I currently have a server on 16.04 running LXC’s, Dockers, KVM’s and native services. The packages available on 16.04 are now getting so old I cant keep up with the current development requirements. I use arch on my Desktop / Laptops, but as you know arch have small breaks here and there and (in my opinion!) I dont want to much management on my Server. I have been experimenting with openSuse Thumbleweed for a few days now and I am impressed. The package managers seem solid and the availability of packages are good.
I was surprised at first when the installer wanted to install btrfs as default. I used to have btrfs on my drives until I experienced two separate unrecoverable crashes of btrfs partitions. This was just plain volumes without sub-volumes and not in RAID. However, reading some more I discovred Snapper. Now this is starting to sound to good to be true. So here’s the questions:
After installing a BASE system, lets say server with Docker, LXC and KVM. I can basically make a snapshot of this? And is there some sort of compression on these snapshots or can this be 7zipped.
With the plugins for zypper / Yast2 / Grub its possible to do a roll-back after an upgrade and reboot?
Can this snapshots be autodeleted / rotated?
Does snapper support incremental snapshots?
How does restoring snapshots work after a drive failure?
What I am looking for is a system that requires a minimum of administration. My plan is to script an update/backup every week, with service monitoring through systemD making the system self-managing and auto-updating every week. SystemD would make it easy to control if everything is running after an update.
Any experiences around these subjects are welcome.
I assume you’re saying you’re now deployed on Ubuntu LTS 16.04.
You should understand that TW is not anywhere close to a LTS, quite contrarily it’s a leading/bleeding edge release that is completely re-imaged with each new release cycle (more or less every 2 weeks).
If you want stability and reliability, you should opt for the LEAP release (currently 42.3) which has a release cycle approx 18 mths, with continuously released <updates>, not massive re-imaging the entire system.
If you’re interested in thumbing through documentation regarding the installation, configuration and maintenance on openSUSE (both TW and LEAP),
It starts with using YaST to install the default recommended setup (supports installing LXC, KVM and/or Xen all managed by libvirt) which is described in the openSUSE documentation, then primarily relying on the SUSE 11 SP4 documentation for general information, configuration and management practices, and then possibly the SUSE 12 SP3 documentation for the various things in it not mentioned in the other documentation.
There are other types of virtualization as well, so if you find you need to support others, then there may be additional documentation or you can ask. Perhaps others before you have already asked your question and is searchable in this Forum.
Re: LXC and libvirt, I don’t know how Unbuntu manages its LXC, if it’s not by libvirt you may need to run special migration/conversion to import into libvirt management. If you can’t find the documentation to do this, post here.
Until the various issues I’ve posted about in the past have been fully addressed either by fixing or publicly declaring I’m wrong about those issues, I can only suggest using all these documentation in the way I’ve described. And, of course if anyone has specific questions about anything that’s unclear or surprising, you’re welcome to post in this Forum.
Regarding BTRFS as your storage file system,
There is a recent/current Forum thread which should of interest, the following post in that thread fully describes a lot about what to be aware of includes a link to the SUSE Enterprise Storage reference
After installing a BASE system, lets say server with Docker, LXC and KVM. I can basically make a snapshot of this? And is there some sort of compression on these snapshots or can this be 7zipped.
Snapshots are actually metadata so take less space than you might expect, and when snapshots are enabled a snapshot is automatically taken at known crucial times like initial install and before/after libzypp is invoked to install or remove any packages… and on boot. I haven’t looked into whether compression is implemented but there are arguments about compressing critical files because although rare it can also cause corruption.
With the plugins for zypper / Yast2 / Grub its possible to do a roll-back after an upgrade and reboot?
Yes. But let’s be clear that the openSUSE definition of an “upgrade” is a major re-imaging so has certain limitations whereas when you do an “update” only relatively minor changes are made and easily rolled back. Another good reason to install on LEAP instead of TW.
Can this snapshots be autodeleted / rotated?
That’s done automatically in the Snapper configuration. You can modify the configuration if you want to change policy.
Does snapper support incremental snapshots?
That’s already how snapshotting works but it’s all hidden so that the User doesn’t have to know the mechanics of what is happening.
How does restoring snapshots work after a drive failure?
Probably depends on what happened, and whether it’s in a RAID array. As of this wring, there is a well known corrupted parity bit due to a dirty shutdown bug regarding BTRFS and RAID 5/6. A patch has been submitted upstream but until it’s been approved and distributed, BTRFS should be used only in RAID arrays of 0, 1 and some combination thereof. Undeleting any corrupted files <might> work but YMMV in this case. Snapshot metadata today is always stored on the same volume being protected, so as long as the array can be recovered, snapshots should still work without a hitch.
Don’t know what your current storage file system on Ubuntu is, but ZFS is really the only other option if you don’t use BTRFS which has comparable automatic self-healing features. EXT4, XFS and other fs have their own features but not the same self-healing.
SUSE/openSUSE use of libvirt to manage your virtualization is pretty reliable with established history.
Figured I should clarify the RAID5/6 bug I mentioned…
It applies only to <software RAID> and shouldn’t apply to hardware RAID.
If anything is still uncertain, find the bug description (Should be easy to find) and read the details for yourself it you think it might apply to you (And in the meantime, be sure you’re running on a really good UPS).
First off, thank you for writing such a thorough answer.
You are correct in assuming I am currently running Ubuntu 16.04 on my server. I do understand Thumbleweed is not an LTS, thats not what I am looking for either. As I mentioned the packages are now “behind” and I am lacking the environment I need. I dont know if this is the case with LEAP, but I will have a look and se where its at. I fear i will run in to the same issue as I currently have with 16.04 (aging packages). That being said, I dont like the idea of a system that needs “major” attention every 18 months… But maybe OpenSuse handles this differently as the problem in Ubuntu is often breakage of PPA’s.
I dont use a manager for LXC. I manage them… You can use libvirt to manage LXC on Ubuntu as well, but I find virt-manager lacking in its futures for LXC. I do use virt-manager for KVM. Ubuntu has a LXC “manager” called LXD. Manager being a bit of an understatement.
My current storrage system is a combination of EXT4 for the system, ZFS for data and XFS in (HW)RAID5 for storrage. A bit of a jungle but it works. btrfs has burned me twice in the past, so I am not thrilled about that. But with snapshots to a ZFS drive I can live with it. (My previous burns was not swRAID)
The project if you will is to try and make a self-managing server and leave it alone for as long as possible. Most of which I am hoping to control with systemd and scripts/python.
The idea was thumbleweed is “more stable” than arch, and has more managing tools, as I am happy doing everything once on a laptop/desktop I would really like a simpler life when it comes to my server. The salespitch for me was when i read about snapper.
Thanks for your Documentation links, I have lots to read!
Well, i was wrong. The packages in LEAP are not “lagging”. OpenSuse must have a completely different philosophy than Ubuntu. LEAP is definitively on the table…
All these years I’ve never looked inn to OpenSuse… Thanks again for your great answers.
