How to do a full system backup (and restore) on opensuse 42.2 with btrfs file systems?

How to do a full system backup on opensuse 42.2 with btrfs file systems? And of course, after that a (easy, simple, fast) system restore on e.g. a new harddisk, which also creates the partitions? (note: I am not asking how to backup my data – I use tar & rsync for that).

When using ext4 I could easily boot a rescue system and tar-ing every partition, but with btrfs I have much more partitions which are also “nested”, so this seems to be much more complicated (e.g. a subpartition /var with subsub partitions /var/cache, /var/crash, etc).

Thanks!

For copying the whole of a disk bytewise (which will then include inddeed all of the disk, that is the MBR with the partition table, all partitions and all that is in those partitions, ranging from rubbish to btrfs), you can look into a tool like

dd

I also think that Clonezilla might be used (I never did).

Remember that doing this the disk must be unused. Thus when it contains a system, that system must not be running (use a life/rescue system or one from another disk) and when it does not contain a system, all file systems on it must be unmounted. Else your copy wil most probably not be consistent.

Thanks. I think this will work, however probably you need to restore on a disk of the same size?
And it is not possible to get e.g. the /etc tree from the image file without restoring the whole disk?
Any other tips?

Yes, preferably a disk of the same size (or larger). But that is included in your description of what you want as I have read it. Copying the disk (including partioning) and restoring the same.

You can of course also use dd (or the like) to copy just each partition themselve. That would allow you to copy them them one by one (but every one on a partition of the same size) on the one or more disks.

And as this is a copy on some medium, it depends on the medium, but if that is some form of mass-storage, you can mount the file systems that are on there and then have all in those file systems available, including /etc.

BTW, I read that you already backup user data using rsync, etc. IMHO doing the same to /etc will give you the system configurations at hand in case of system disaster and reinstallation (not for a blind copying back, but for comparing with what I had). I do not make clones of disks or partitions, but copies for back up of /home, /etc, /boot, /root and /srv. But hat is all to be adapted to your needs.

Thanks.

Any other suggestions from other forum visitors?

I have been using http://www.terabyteunlimited.com/image-for-linux.htm
for backing up my systems here now for years. While this is a
product for purchase I can highly recommend it. I use it
to backup my windows and Linux systems. You can backup
to just about any media available today, DVD, USBPen/disks
etc. There is nothing easier to use that I have come across
and for the very reasonable pricing you cannot beat it.

They have versions for DOS, Windows and Linux.as
well as some other tools and utilities including a
great partitioning tool.

… same size or larger.

I use Clonezilla, have for many years, love it, has not failed me. But, I do not use btrfs file system, instead sticking with ext4, and I recall someone who ran into a serious problem trying to do just what you want to do with btrfs.

You might also like to reference this post:
https://forums.opensuse.org/showthread.php/521277-LEAP-42-2-btrfs-root-filesystem-subvolume-structure

If your btrfs root filesystem has snapshots enabled you might start with using the btrfs send receive commands to copy a snapshot to a replica on another media. You could then use rsync for the twenty odd subvolumes not subject to snapshots.

I recently spent considerable time researching how I might keep a replica of a openSUSE btrfs rootfs. This included figuring out how to accurately replica a rootfs and its subvolumes and snapshot capability. I posted my findings a forum entry.

In the end I found it easier to stay with ext4.

Thanks, I tried clonezilla (the most recent version, “alternative stable - 20161121-yakkety”) and it failed on my btrfs partition with no clear error message. Maybe opensuse 42.2 has some changes somewhere in in btrfs which are not supported by this version yet?

Thanks, this is really useful. I have wondered for a long time how and why the “opensuse standard” subvolume structure was chosen and finally ended up with thinking that it probably is “best practice”. I ended up in choosing btrfs instead of ext4 because it “probably is the future” (and seems to have some advantages above ext4). I also liked the replication mechanism but it doesn’t replicate everything in the root filesystem tree, so then its use seems limited to me. But this might be caused by that I do not really understand the pro’s and con’s of other approaches — that is why I still wonder about a good backup mechanism of the root filesystem.

What I do now to backup the root filesystem is as follows:

  • booting using the opensuse rescue disc
  • run a script that mounts all of the root filesystems (according to what is in my /etc/fstab) on e.g. /mnt
  • do a “tar --format=pax --xattrs --acls --sparse --create --file [some file name].tar /mnt”

If I do the following to restore, would that be a sensible and workable solution?

  • I boot using the opensuse rescue disc
  • I mount /dev/sdxx to /mnt
  • I use your script in your forum entryto restore the subvolume structure on the /mnt (I first adapt it so that it is conforming my /etc/fstab)
  • to restore I do a “tar --format=pax --xattrs --acls --sparse --extract --file [some file name].tar /mnt”

Would this work and restore the right files to the right subvolumes?

Note: with “your script” I refer to the script starting like this:

    # Mount subvolid=0 subvolume
    mkdir -p /mnt/tmp_rootsv
    mount /dev/sdb2 -o subvolid=0 /mnt/tmp_rootsv

    # Create the main snapshotted subvolume of the root filesystem 
    btrfs subvolume create /mnt/tmp_rootsv/@

    # Create the non-snapshotted subvolumes
    mkdir -p /mnt/tmp_rootsv/@ && btrfs subvolume create /mnt/tmp_rootsv/@/.snapshots
    mkdir -p /mnt/tmp_rootsv/@/boot/grub2 && btrfs subvolume create /mnt/tmp_rootsv/@/boot/grub2/i386-pc
    mkdir -p /mnt/tmp_rootsv/@/boot/grub2 && btrfs subvolume create /mnt/tmp_rootsv/@/boot/grub2/x86_64-efi
    ...etc...]

I hope you are aware of the fact that clonezilla “clones” partitions (or whole disks) by copying disk bl;ocks. It is not aware of the meaning of what data is in those blocks. In other words, Clonezilla has no idea if other software (or you) think the contents of the partition is a file system of any type. The fact that it is a Btrfs file system is of no importance at all.

I think that should work: get the new root built to the same structure, then mount all its subvolumes in the right places within it, and then untar over the mounted structure. I’m a little uncertain as to how to treat the snapshots. If I were using tar I would likely not backup the snapshots, and then after a restore I might do snapshot to give me a new baseline, then set that to be the snapshot to be mounted by default on next boot. Alternatively you could just not setup any snapshots and leave the default boot to be the @ volume, then figure out what to do about snapshots later. I haven’t tried any of this this, I suggest you experiment on a VirtualBox.

I was intending to use rsync to a live replica for an online backup on a separate drive. That’s what I do for my ext4 root - that way I can recover from failure of the root media or corruption of root quite quickly by using a rescue disk to tweak the online backup. I also use the same rsync scripts to update other media which gets rotated offline and out of the house. I use rsync to speed up the transfer times and reduce writes.

If I were to persist with btrfs I would write some scripts that thoroughly trap errors for both the replication of the rootfs and the mounting of all the subvolumes. Then I would need to test things thoroughly using VirtualBoxes. That seems like many days of work, so I decided to pass and continue using ext4, which required next to no effort.

As mentioned in my notes, I’m not happy with openSUSE’s choice to have so many subvolumes. Especially with no straight forward supporting tools to help with doing the kind of backup and restores described above.

On a higher plane, it seems to me the btrfs root has it guts hanging out all over the place: the subvolumes, the .snapshots folder, obscure default subvolume mounting business, funny @ names, the need for special versions of df and du, space disappearing into snapshots. I wonder if that means that the filesystem is layering in too many concepts/abstractions into one layer?

The reason I wrote up my notes was that I was surprised at how much googling I had to do the find explanations of how each piece of the puzzle fitted together.

BTW - I would have though clonezilla or dd would have no trouble with any kind of partition - I think at one point I did use dd to copy a btrfs rootfs with no issues.

Thanks. What I don’t understand, if I read Clonezilla’s website (http://clonezilla.org/), it starts with “Many File systems are supported: … a long list follows, among which btrfs …] For these file systems, only used blocks in partition are saved and restored. For unsupported file system, sector-to-sector copy is done by dd in Clonezilla.”

Maybe my run on clonezilla failed because I choose the defaults and it tried to save only used blocks and apparently needed to mount the btrfs tree for that. Are you suggesting to use the sector-to-sector copy by dd in clonezilla? I can try that, thanks again.

Thanks again. Yes, I also tried googling for a good backup method for the root file system but didn’t find something which I could get to work. Maybe, some time in the future the opensuse 42.x distribution will include an easy backup tool…

So it seems that Clonezilla tries to be intelligent and to interprete the contens of the partition.

For me that is contrary to cloning, I only would use dd (itself not through clonezilla) to be sure it does what I want it to do.

When I do not want to clone a partition (or disk), I use other tools that work at the file level (rsync, tar, cp).

But that is all personal preference.

Actually, I have seen at least one forum member run into that failure in 42.1, as well, the one I mentioned in my post. It left me dashedly embarrassed!:shame:

Although I am still using Clonezilla to backup my 13.1 machines, I have also returned to using dd for backups (I prefer to make more than one backup with different utilities, just to add extra precaution.).