Wanted: Script to save BTRFS subvolumes and restore them

In another thread it was noted that there seems to be no disk/partition cloning/imaging software tool that can really handle BTRFS volumes besides doing a 1:1 sector copy which has lots of drawbacks. “Really” here means that all subvolumes are being re-created when restoring.

So I am looking for a script which accepts a single parameter which is a mount point (e.g. \ or \mnt\dir).

Let’s name the script subvolbk. It will basically consider the output of
sudo btrfs subvolume list <mount point>

It may throw an error if mount point does not match a top-level subvolume. Otherwise it should generate a list of btrfs subvolume create commands to recreate the subvolume structure on an empty top-level BTRFS volume.

So for example, i could do something like this:

Backup:

# mount /dev/sda2 /mnt/src
# findmnt -no FSTYPE /mnt/src
btrfs
# subvolbk.sh /mnt/src > /tmp/sda2.btrfs.sh

Restore:

# mkfs.btrfs -L "Clone" /dev/sdb2
# mount /dev/sdb2 /mnt/dst
# sh /tmp/sda2.btrfs.sh /mnt/dst

The output of btrfs subvolume list /mnt/dst should then be quite similar to that of btrfs subvolume list /mnt/src.

I hope this is understandable. Pointers to existing and already tested scripts are welcome and highly appreciated. Of course it doesn’t have to be a shell script, Python, Perl, whatever is fine.

Ouch :rofl:

Yup :sweat_smile:

This could be a starting point.

@duise Have a read here https://jwb-systems.com/btrfs-based-remote-backups-on-suse/

Infamous host and its sibling run btrbk: GitHub - digint/btrbk: Tool for creating snapshots and remote backups of btrfs subvolumes

It’s lightweight, robust and easy to use. Performed some 1000 full backups of root and /home:

erlangen:~ # btrbk -c /etc/btrbk/btrbk-home.conf stats
SOURCE_SUBVOLUME  SNAPSHOT_SUBVOLUME             TARGET_SUBVOLUME                SNAPSHOTS  BACKUPS
/home             /Btrbk/btrbk_snapshots/home.*  /Backup/btrbk_snapshots/home.*          1      504
/                 /Btrbk/btrbk_snapshots/ROOT.*  /Backup/btrbk_snapshots/ROOT.*          1      503

Total:
   2  snapshots  
1007  backups    (2 correlated)
erlangen:~ # 

Creating a full backup of 500GB /home on an external HDD takes less than a minute:

erlangen:/HDD # btrbk -c /etc/btrbk/btrbk-HDD.conf usage
MOUNT_SOURCE    PATH                      SIZE     USED       FREE
/dev/nvme1n1p2  /                         1.82TiB  620.34GiB  1.21TiB
/dev/sdb1       /HDD/btrbk_snapshots/HDD  7.28TiB    3.19TiB  4.09TiB
/dev/nvme1n1p2  /home                     1.82TiB  620.34GiB  1.21TiB
erlangen:/HDD # 
erlangen:/HDD # btrbk -c /etc/btrbk/btrbk-HDD.conf extents /HDD/btrbk_snapshots/HDD/home.20231021T1855/
TOTAL       EXCLUSIVE   SUBVOL
476.45 GiB  476.45 GiB  /HDD/btrbk_snapshots/HDD/home.20231021T1855

Union (1 subvolumes):  476.45 GiB
erlangen:/HDD # 
erlangen:/HDD # journalctl -b -u btrbk-HDD.service 
Oct 21 18:55:11 erlangen systemd[1]: Starting btrbk backup of /home to HDD...
Oct 21 18:55:49 erlangen btrbk[6566]: --------------------------------------------------------------------------------
Oct 21 18:55:49 erlangen btrbk[6566]: Backup Summary (btrbk command line client, version 0.32.6)
Oct 21 18:55:49 erlangen btrbk[6566]:     Date:   Sat Oct 21 18:55:11 2023
Oct 21 18:55:49 erlangen btrbk[6566]:     Config: /etc/btrbk/btrbk-HDD.conf
Oct 21 18:55:49 erlangen btrbk[6566]: Legend:
Oct 21 18:55:49 erlangen btrbk[6566]:     ===  up-to-date subvolume (source snapshot)
Oct 21 18:55:49 erlangen btrbk[6566]:     +++  created subvolume (source snapshot)
Oct 21 18:55:49 erlangen btrbk[6566]:     ---  deleted subvolume
Oct 21 18:55:49 erlangen btrbk[6566]:     ***  received subvolume (non-incremental)
Oct 21 18:55:49 erlangen btrbk[6566]:     >>>  received subvolume (incremental)
Oct 21 18:55:49 erlangen btrbk[6566]: --------------------------------------------------------------------------------
Oct 21 18:55:49 erlangen btrbk[6566]: /
Oct 21 18:55:49 erlangen btrbk[6566]: +++ /Btrbk/btrbk_snapshots/HDD/ROOT.20231021T1855
Oct 21 18:55:49 erlangen btrbk[6566]: >>> /HDD/btrbk_snapshots/HDD/ROOT.20231021T1855
Oct 21 18:55:49 erlangen btrbk[6566]: /home
Oct 21 18:55:49 erlangen btrbk[6566]: +++ /Btrbk/btrbk_snapshots/HDD/home.20231021T1855
Oct 21 18:55:49 erlangen btrbk[6566]: >>> /HDD/btrbk_snapshots/HDD/home.20231021T1855
Oct 21 18:55:49 erlangen systemd[1]: btrbk-HDD.service: Deactivated successfully.
Oct 21 18:55:49 erlangen systemd[1]: Finished btrbk backup of /home to HDD.
Oct 21 18:55:49 erlangen systemd[1]: btrbk-HDD.service: Consumed 8.144s CPU time.
erlangen:/HDD # 

Something I’ve always done when working with backup / imaging software I’ve not worked with:

a. Back it up
b. Restore it to a spare drive
c. Boot test the restored drive

Works? Yes - great. No - move on to the next thing.

Personally, I have /home on a separate XFS partition that gets backed up constantly.

The OS partition, (I use BTRFS for / ) I’m not concerned about - if it flounders, and can’t boot to a past restore point, I simply reinstall. Doing so is actually not very time consuming.

1 Like

Actually I looked at btrbk when doing my research because it was mentionend in one of the threads that aggie posted. But I am surprised to see that it shall re-create subvolumes because in the README it says:

“Btrbk does not provide any mechanism to restore your backups, this has to be done manually.”

You may read this first: System Backup with btrfs send / receive. Manual restore requires running a single command, basically as easy as copying a single file, e.g.:

btrfs send /mnt/backup/267/snapshot | btrfs receive /.snapshots/267/

When restoring a system subvolume managed by snapper a description helps, e.g.:

 erlangen:~ # cat /.snapshots/2768/info.xml 
<?xml version="1.0"?>
<snapshot>
  <type>single</type>
  <num>2768</num>
  <date>2023-10-20 21:12:24</date>
  <description>writable copy of Backup</description>
</snapshot>
erlangen:~ # 

After creating the above snapshot I made it the default subvolume and successfully booted infamous host erlangen.

erlangen:~ # snapper list |grep 2768
2768  | single |       | Fri Oct 20 23:12:24 2023 | root |         | writable copy of Backup |              
erlangen:~ #