(not sure this is the right forum…) So, it looks like BtrFS really wants to create logical volumes. So this is the question I’ve been g**gling…
“…and then hard drives (HDs) were created for personal computers, and these HD’s had “filesystems” which existed in dedicated on-disk “partitions.” The physical disks had up to several “master recordkeeping areas” (MRAs). Then, a limited number of Master Partitions and an Extended Partition (containing “logical drives”) could be created, with the information stored in an on-drive MRA…”
Now, enter LVM* and BtrFS Logical Volumes. How do these relate to logical drives - I assume they’re essentially identical except they’ve been unencumbered by restrictions posed by previous modern filesystems. So, basically, I assume BtrFS has in its structure a place equivalent to what I naively called “MRAs”…
Is that filesystem-config data stored in the MBR/GPT system somewhere or is it in a dedicated BtrFS data-storage area?
What is the main drawback to thinking of a BtrFS ‘Logical Volume’ as essentially equivalent to a “logical drive?” (…except LV’s don’t get “mounted” or edited in Partitioner/GParted? Oops - I take that back, they do get mounted… I see them in /etc/fstab looking like separate partitions)
Of course, there’s a TON of technical detail left out of that… but my problem seems to be seeing the “forest for the trees” buried in all the complex online discussions. I basically need to be sure of how to change my backup strategies and applications (dd, clonezilla, etc.).
Thanks for the great link. I’m reading about bitrot and it makes me want to backup my data and reinstall Opensuse with btrfs to get transparent checksumming. I didn’t realize BtrFS had that.
So it looks like it’s “OK” to think of “volumes” in BtrFS as “partitions” that we’ve dealt with since forever.
Quick question - back in the days of Lindows, when I first started on Linux - the mantra was “backup your data” - which meant something like: tar -czvf backup.tgz /home/user.
So, what does “backup your data” mean now with BtrFS? It can be really difficult to deal with a giant tar file, or even a backup manager application, and that difficulty spawns consequential errors for non-gurus. Is “snapshots” (or something else) useful and uncomplicated?
Please be aware, snapshots are no substitute for backup! Snapshots are very helpful if you run into trouble with your system. I once ran into a no-space-left problem during the distro upgrade with zypper dup. Seing all the failure messages from zypper I would normally have panicked. But then, I just stopped it, made a rollback to a safe snapshot and freed enough space on the disk. (That included deleting some snapshots, but they weren’t the culprit.)
However, any “real” trouble, i.e. hardware issue and you’re lost. If your drive is damaged your snapshots are gone with it. That’s why I still keep /home on separate partitions - usually even separate drives. Whatever problem you have with you root system, even if you need a fresh install, change to another filesystem or even hardware, just mount your /home during install. And /home you should backup, of course.
I have to admit I never used tar for backups. (Not sure, does it compress the archives?). Nowadays disk space isn’t the main issue, I’d say. Many people use rsync. There’s also dedicated backup software. I’m personally a friend of unison for syncing to my homeserver and from there backing up to other drives including an external hdd.
I really do like that idea of a snapshot-before-upgrade sort of thing. I also like error correction of bitrot, but that seems to require a RAID setup - I don’t know if a snapshot could be used for that but it seems like it would be possible. I think I saw some web pages where people make snapshots (somehow, FULL snapshots) that are moved to another drive (like a USB HD). I’m not sure how that works. BtrFS seems to be changing on an annual bases. So if you could do system snapshots to a USB HD/SSD that would help home computers a lot, since USB3 is quite fast.
Tar has the option for compression, yes. The “z” in tar -cxvf… uses gzip. Now, the problem I have is that it’s difficult to extract single files from a tar file. In fact, I’ve never succeeded in it due to my misunderstandings of the syntax. So I keep a separate .txt file with the verbose output of tar. That sort of helps. My problem with backup software is its complexity (maybe I’m dyslexic) and difficulty to get rollbacks in the event of a bad system crash.
Have you ever used dd for a backup? I think you use rsync because it will do incremental backups? People are also talking about dup for correcting errors, although I don’t understand how it does that.
Sorry, I don’t have enough knowledge to comment on your question about dup + Btrfs. However, to most issues I prefer a pragmatic approach. Therefore I usually (try to) ask myself, “Is it worth the bother?”. If it comes to backup the question is, what do you want to achieve? I am concerned about bit rot, too. That mostly considers my digital “valuables”, memories etc. But a large part of the snapshots taken by Btrfs is just the operating system. The snapshots are supposed to make it easier to recover. So, does it make sense to make too big a fuss of conserving my OS 100%? It might be easier to backup the relevant settings. If the “normal” rollback doesn’t help anymore, just go for the fresh install and restore your settings. Most should be in /etc/ aside from /home/user. There may be more in /usr/… . For a lot of things, just taking notes of your setup may yet be easier than trying to restore it from backup. IMHO, everybody needs to define his / her “breakeven point” of backup efforts vs. backup benefits.
As for your question: In order to use dd you have to unmount the drive in question (AFAIK). For regular backups I think it’s not very handy. It won’t bother about snapshots, filesystem or anything but just clone everything. So it’s a very thorough way if not some kind of overkill.
I think you use rsync because it will do incremental backups?
I do like the incremental backup but I revert to unison rather than rsync - being pragmatic. It has a GUI which - if needed - helps you getting started with huge backups (or rather syncs). You can easily check what will be changed **before **hitting “go”. But once your backup / sync roots are established, you can let it run in the background, text mode, batch mode which is very fast - similar to rsync. It can use rsync mode as well, BTW. You can put it in a wrapper script and read out the return value if sync was ok. If not, the log files is very detailed.
And the documentation is quite good once you found it in /usr/share/doc/packages/unison/unison-manual.html after installing.
…a large part of the snapshots taken by Btrfs is just the operating system.
Sorry, forgot to add: That’s assuming your /home is a logical volume of Btrfs. If it is a separate partition it will have it’s own file system and the snapshots will be 100% operating system.
I would agree that backing up the OS is not usually necessary - but backing up the /home/user directory can be really important. I think I had some fairly serious bitrot on my system, or something that has a similar effect… I had upgraded from 42.1, stepwise, but when the time came for the 15.1 -> 15.2 upgrade, there were some issues, and my /home partition had been around a LONG time. A big one seemed to be associated with Firefox. I was forced to go back to backup to get ff going again. But there were issues with that, too.
I do tend to keep things around as long as possible, so I do care a lot about phenomena such as bitrot - I don’t know of any others specifically - but hd crashes and software crashes, OS hangs, etc. tend to pile up in their effects on data and apps, especially big, complex ones like KDE and ff.
So BtrFS can be set up to “dup” your data, and make it more or less immune to bitrot - but it cuts your drive size in half - BUT there is also lzo/zlib compression, so that may partially compensate. Also there are snapshots - I am not sure how these really work yet - I think they also consume drive space, eventually, significant drive space. So BtrFS still has a way to go.
Yeah, I remember you Firefox problem. I have just posted some update in that thread.
I will now check your link about Btrfs and DUP. I am going to replace both HDD in the raid on my homeserver and now I’m curious. I have never considered using Btrfs for pure data drives without OS. Would anybody recommend that?
Except, apparently, for SSD’s. This apparently doesn’t work with SSD’s because they deduplicate files/metadata to save storage space? So my laptop has room for two hard drives - maybe I could use one SSD as the BtrFS backup for the other? I think this wouldn’t work (but I don’t know) because BtrFS needs a defined RAID to do backup - and dual-HD laptops don’t usually have RAID capability.
Well, BtrFS is a default FS in many big Linux distros now. Since it can auto-repair (and compress, etc.) bit errors, I would go for it. But I only use a laptop for my main computer, so I don’t know if I can do this.
Thanks for the ff update. I just gave up. :shame: I should maybe move to some less trouble-prone browser. I think my problem(s) with ff may have related to undetected bit-rot or just accumulations of bit-write-errors. My /home directory has been literally the same partition since 42.1 or thereabouts. That’s a long time. So that’s a strong argument for BtrFS for folks like me.
Does anyone know enough about BtrFS to know if it can be used with dual hard drives (SSD’s) to get auto bit-error-correction? Do you just tell BtrFS that both drives are part of the same FS, but tell BtrFS that one drive is the backup data/metadata for the other?
(this sounds complex and I don’t know if it would work)
I think I saw some web pages where people make snapshots (somehow, FULL snapshots) that are moved to another drive (like a USB HD). I’m not sure how that works. BtrFS seems to be changing on an annual bases. So if you could do system snapshots to a USB HD/SSD that would help home computers a lot, since USB3 is quite fast.
