BTRFS as the file system where your Virtual Machines are stored

English Virtualization
1

Partly as a reminder because for many virtualization technologies this is not new, but likely needs to be raised anew particularly because the imminent release of 15.1 will format the entire system with BTRFS by default and Virtualbox as always defaults to storing virtual machine files in /home,

With 15.1 approaching, I took a look around for any recent guidance, recommendations or documentation regarding BTRFS as the file system underlying virtual machine (Guest) files and found… nothing. This could be partially because awhile back RHEL dropped support for BTRFS in favor of ZFS leaving SUSE/openSUSE perhaps the only distro that has supported BTRFS unflinchingly.

with 15.1, openSUSE takes another major step encouraging the use of BTRFS so depending on your situation may require attention on your part.

There are the usual known benefits of BTRFS you can look forward to experiencing

  • File system auto-repair should mean less corruption that requires manual repair
  • Snapshots, able to rollback to erase changes and recover
  • Probably less well known than Snapshots, you can restore individual files. You’re not obligated to an entire volume “all or nothing” rollback

AFAIK there are only a few possible issues to consider, and these might not even exist if they have been resolved (I am not close enough to BTRFS Dev to know for sure what may be true today), I am hoping someone with better knowledge than myself can opine or post progress as warranted.

  • There have been issues in the past about “larger” disk arrays. Then I heard those issues were resolved. But I still see some postings, so don’t know if such issues still exist
  • RAID5 - Stay away from this one type of RAID
  • There may be performance issues related to internal fragmentation of very large files, which primarily affects database files and virtual machine files. I’ve only seen complaints, but no word whether application defragmentation tools can address this issue.

On general principles, you can probably expect that if you run virtual machines actively, you will be creating plenty of changes tracked by BTRFS creating a lot of metadata and larger snapshots, and if you run multiple virtual machines that can increase exponentially. One recommendation I’ve read that makes sense is to disable snapshots on the specified folder location and instead depend on your normal virtual machine backups.

An observation, the YaST Volume Manager module interestingly can be used to manage both LVM and BTRFS volumes maybe even transparently although at this time I think that it’s important to differentiate the two types of volumes.

As sparse as reliable information about deploying BTRFS is,
Here are a few sources I’d recommend reading. If anyone can recommend other sources, I’d encourage them to share those links as well.

The SUSE 12 Storage Administration Guide
https://www.suse.com/documentation/sles-12/singlehtml/stor_admin/stor_admin.html

A kernel.org page by someone who just wanted to post some <possible> issues. With the disclaimers, he also is saying what is on this page is uncertain, but things to be wary about
https://btrfs.wiki.kernel.org/index.php/Gotchas

As usual, Wikipedia can provide lots of info in summary form
https://en.wikipedia.org/wiki/Btrfs

As usual, the Arch Wiki provides lots of detailed info on everything from creating the file system manually to enabling/disabling features to troubleshooting and known issues. Since this is a live document, visit this page frequently for latest info
https://wiki.archlinux.org/index.php/Btrfs

HTH,
TSU

2

The SDB BTRFS has some info on this topic
https://en.opensuse.org/SDB:BTRFS

Although there doesn’t seem to be much documentation and guidance,

It should be noted that
The default locations for machine images managed by libvirt is excluded from snapshotting.
The /home sub-volume in 15.1 is excluded from snapshotting, which of course will automatically address Virtualbox which by default stores machines in a sub-directory of /home.

I assume if someone really wanted to set up snapshotting machine images as a recovery strategy, you’d want to deploy in a completely separate volume from / so that any system rollbacks wouldn’t also rollback your virtual machines. But, you’d also want to create a special Snapper policy that doesn’t already exist or manually execute snapshots. There could also be other issues I haven’t thought about.

TSU

3

The only issue I have observed with BTRFS was a pacemaker log file of 7GB (no log rotation was deployed begfore that), which was highly fragmented (use filefrag to check this) > 1000000 fragments and of course that caused severe problems to the storage during backups (10 machines were backed up in the same time).

So , you have to keep an eye on the fragmentation.
Ofcourse this can be overcome in KVM with creating a snapshot , backup the old disk and then ’ virsh blockcommit’ it back.
Sadly I’m not a VirtualBox user , but in case you have no other options - you can always add a second disk to your machine and make btrfs migrate all data over it.That approach is unsuitable if you have alot of VMs.

4

In order to avoid things like fragmentation or I/O congestion, extremely active logfiles should be in their own partition, or even better on their own RAID arrays optimized for fast writes, like RAID 0.

Your reference to a pacemaker suggests that your system is a Highly Available Compute cluster…

TSU