Page 3 of 4 FirstFirst 1234 LastLast
Results 21 to 30 of 34

Thread: partition/volume

  1. #21

    Default Re: partition/volume

    Quote Originally Posted by karlmistelberger View Post
    If you know any other wide-ranging (but useful) BtrFS discussions/examples, let us know! My current question concerns using two separate SSDs in a single system to get bitrot/bit-error protection. I guess they could be set up as a single BtrFS filesystem somehow - but BtrFS would have to somehow know that the second drive is the "mirror" of the first (for bit redundancy). That would likely also be a question for non-SSD dual-drive systems, also. But I haven't found a discussion/example of this online yet.

    EDIT: I take that back... (I guess I found the right search-terms) The last part of this is the issue with SSDs.
    https://btrfs.wiki.kernel.org/index....ltiple_Devices
    "A Btrfs filesystem can be created on top of many devices, and more devices can be added after the FS has been created.
    By default, metadata will be mirrored across two devices and data will be striped across all of the devices present. This is equivalent to mkfs.btrfs -m raid1 -d raid0.
    If only one device is present, metadata will be duplicated on that one device. For HDD mkfs.btrfs -m dup -d single, for SSD (or non-rotational device) mkfs.btrfs -m single -d single."

    (It seems like YaST Partitioner should be aware of these things, and support options, since BtrFS is a default.)

    And here's a question - suppose one of the two devices fails (given that BtrFS striped the meta/data). How would that be recoverable, if at all, other from backup?

  2. #22
    Join Date
    Jun 2011
    Location
    Germany
    Posts
    297

    Default Re: partition/volume

    From what I have seen on the Btrfs website, it's not that easy:

    https://btrfs.wiki.kernel.org/index.php/FAQ

    For parity there is reference made to RAID 5/6 which has the issues Svyatko pointed out. Otherwise it's the dup option, which would cost 50% space. I have to admit I didn't check further. 50% of two drives would be ok. 50% of the raid1 I am currently using is not. However, my HDD's are rotating 24/7 since app. 10 years, same as my system SSD I purchased in Taiwan 11 years ago. I have already been facing errors in dmesg and hangs during boot I could only overcome with sys-req keys. So I'm being a coward. The system SSD is already replaced thanks to the Leap live system and dd. The first hdd I have replaced by a new SSD letting BIOS rebuild the raid. The second may follow over night.

    I'm still interested if there is a positive reply to your question:

    Does anyone know enough about BtrFS to know if it can be used with dual hard drives (SSD's) to get auto bit-error-correction? Do you just tell BtrFS that both drives are part of the same FS, but tell BtrFS that one drive is the backup data/metadata for the other?
    Since it does support RAID 0 as well dup, I am quite positive.
    I might give it a try, too. Just later on...

    Edit: Ah, I was typing too slowly...

  3. #23

    Default Re: partition/volume

    It looks like "default" for a multiple-drive BtrFS partition is that it creates a combination of "mirror" (RAID1?) and "striped" (RAID5) if two devices are present. But also you can control it from the command-line when creating the BtrFS. Apparently this requires that you create the BtrFS *before* running the Opensuse installer?

    This "implies" to me redundancy so that bit-rot is automatically corrected - but I'm not perfectly sure.

    I will test with this when I set up a dual-SSD system:
    (although it's not exactly perfect - only checking for "half space" on the resulting dual-SSD "BtrFS-raid")
    https://www.linux.com/training-tutor...-linux-part-1/

    And this is why... (from https://btrfs.wiki.kernel.org/index....ltiple_Devices )
    (I don't really understand the "-m raid1 -d raid0" part yet...)
    " Multiple devices
    A Btrfs filesystem can be created on top of many devices, and more devices can be added after the FS has been created.
    By default, metadata will be mirrored across two devices and data will be striped across all of the devices present. This is equivalent to mkfs.btrfs -m raid1 -d raid0.
    If only one device is present, metadata will be duplicated on that one device. For HDD mkfs.btrfs -m dup -d single, for SSD (or non-rotational device) mkfs.btrfs -m single -d single."

    In particular, I'm wondering what dup-ing just the metadata does for you? (besides saving a ton of space)



  4. #24

    Default Re: partition/volume

    Multiple devices
    A Btrfs filesystem can be created on top of many devices, and more devices can be added after the FS has been created.
    By default, metadata will be mirrored across two devices and data will be striped across all of the devices present. This is equivalent to mkfs.btrfs -m raid1 -d raid0.
    If only one device is present, metadata will be duplicated on that one device. For HDD mkfs.btrfs -m dup -d single, for SSD (or non-rotational device) mkfs.btrfs -m single -d single."
    (https://btrfs.wiki.kernel.org/index....ltiple_Devices)

    So, mkfs.btrfs -m raid1 -d raid0 means stripe the metadata and mirror the data (across the devices). Apparently the definitions of various RAID levels have changed since the last time I was heavily into RAID! (Or else I never really understood them.)
    https://en.wikipedia.org/wiki/Standard_RAID_levels

    Why raid0 the data: no redundancy or fault-tolerance in striped data (wikipedia)?? Raid1-ing the metadata gives redundancy (automatic bit-repair) for the metadata. I guess you can't use raid5 with only two SSD's, so that means raid1 is the only option. (i.e., mkfs.btrfs -m raid1 -d raid1) ...or, maybe BtrFS' version of raid0 includes redundancy?

  5. #25
    Join Date
    Feb 2010
    Location
    Germany
    Posts
    4,061

    Default Re: partition/volume

    Quote Originally Posted by pattiM View Post
    My current question concerns using two separate SSDs in a single system to get bitrot/bit-error protection.
    In other words, RAID …
    • The built-in Btrfs RAID versus the current Linux RAID …


  6. #26

    Default Re: partition/volume

    Quote Originally Posted by dcurtisfra View Post
    In other words, RAID …
    • The built-in Btrfs RAID versus the current Linux RAID …

    Thanks for the links!! I will read them to see if they clear up the confusion. Speed isn't really important. My question is BtrFS-redundancy for data and metadata (to enable automatic self-correction of bit-rot-like phenomena) on SSD's contained in a BtrFS raid. Using RAID-like terminology in BtrFS options is confusing because, for instance, raid0 works "without parity information, redundancy, or fault tolerance" - yet with BtrFS seems to inherit these things when you set up -m raid0 -d raid1 - as here...

    Code:
    # Stripe the data without mirroring, metadata are mirrored
    mkfs.btrfs -d raid0 /dev/sdb /dev/sdc
    The real, basic, underlying problem is that Wizards, Gurus, Hackers, and the like, have large amounts of tacit/implicit knowledge in their brains which enable them to fairly-easily navigate the jargon of new, complex systems, like BtrFS. Noobs like me are relegated to asking questions and hoping some of that knowledge rubs off on us.

    So, the plan is to take two SSDs in a laptop and create a BtrFS across both devices - but I'm not exactly sure what "raid-like-level" to set for data and metadata in order to insure maximal redundancy and self-correction. "Striping" seems most appropriate, but again - that's standard-raid-think, and BtrFS raid-like-levels appear to be not exactly like standard-definition raid-levels. Also, raid5 (striping with parity, etc. according to standard-raid-defs) is newer in BtrFS and may therefore be more error-prone.

    "...Realistically, raid1 mode is the most well tested of the various multi-device configurations provided by BTRFS, by virtue of the facts that it's one of the oldest supported configurations, and it's also one of the most widely used. " https://unix.stackexchange.com/questions/480391/btrfs-raid-1-vs-mdadm-raid-1

    (Maybe I'm being retentive here, but I just don't want to make a sorry mistake.) The idea that's forming in my mind is that there appear to be "three raids" possible in BtrFS - standard raid, metadata raid, and data raid..

    I don't believe the Opensuse installer covers the setup I have in mind, so I'll create the BtrFS using the OS live distro first. The issue of snapshoting this setup safely still is an open question in my mind, but at least bitrot-like-phenomena (such as write errors) are precluded.

  7. #27

    Default Re: partition/volume

    Or is this the optimal setup?

    Multiple devices
    A Btrfs filesystem can be created on top of many devices, and more devices can be added after the FS has been created.
    By default, metadata will be mirrored across two devices and data will be striped across all of the devices present. This is equivalent to mkfs.btrfs -m raid1 -d raid0.
    If only one device is present, metadata will be duplicated on that one device. For HDD mkfs.btrfs -m dup -d single, for SSD (or non-rotational device) mkfs.btrfs -m single -d single.

    This looks like the "metadata" is mirrored (does metadata include everything, except the redundant data, that's needed to repair bit-errors?). OK. But the data itself is striped. That seems like it would be faster, at least on rotating drives... or is there some data-safety advantage to "mirror" (raid1) the "data" also? This will be two SSD's, so "intrinsic SSD de-duping" shouldn't be a problem as long as data/metadata copies never reside on the same device?
    So this last consideration would instead seem to suggest using raid1 for both data and metadata for the BtrFS that will span these two SSDs, correct?

  8. #28
    Join Date
    Feb 2010
    Location
    Germany
    Posts
    4,061

    Default Re: partition/volume

    Quote Originally Posted by pattiM View Post
    So this last consideration would instead seem to suggest using raid1 for both data and metadata for the BtrFS that will span these two SSDs, correct?
    Correct – that's RAID Btrfs-style – RAID always stripes user-files across the devices – when a device fails, no problem – simply swap it out and re-establish the RAID with the new member – the (complete) user data (nothing is lost) held on the other RAID devices will be created on the new device and then the striping will continue, across the RAID.

    The Btrfs Metadata is however another issue – it's relevant only to the Btrfs filesystem on a specific device – not across the RAID – each RAID device has to maintain it's own Metadata for the Btrfs filesystem on each device …

    BTW – please be aware that, RAID doesn't protect against data corruption due to a malicious (Virus) attack – the corruption performed by the intruder will be faithfully striped across the RAID. Therefore, in the current world, a RAID still needs regular backups – in the early days, when viruses were not an issue, that wasn't the case but, today, it is …

  9. #29

    Default Re: partition/volume

    OK, a little confused on the word "device" here. I'm using two SSD's (two devices) which will constitute a single BtrFS file system. I have the choice of BtrFS' raid0 or raid1 for both metadata and data.

    Quote Originally Posted by dcurtisfra View Post
    Correct – that's RAID Btrfs-style – RAID always stripes user-files across the devices – when a device fails, no problem – simply swap it out and re-establish the RAID with the new member – the (complete) user data (nothing is lost) held on the other RAID devices will be created on the new device and then the striping will continue, across the RAID.

    The Btrfs Metadata is however another issue – it's relevant only to the Btrfs filesystem on a specific device – not across the RAID – each RAID device has to maintain it's own Metadata for the Btrfs filesystem on each device …
    ...so you mean "...it's relevant only to the part of the BtrFS filesystem on a particular device..." ?? (there aren't two filesystems in my proposed setup - or are there?)

    Quote Originally Posted by dcurtisfra View Post
    BTW – please be aware that, RAID doesn't protect against data corruption due to a malicious (Virus) attack – the corruption performed by the intruder will be faithfully striped across the RAID. Therefore, in the current world, a RAID still needs regular backups – in the early days, when viruses were not an issue, that wasn't the case but, today, it is …
    Sure - Btrfs raid is not a backup!!

  10. #30
    Join Date
    Feb 2010
    Location
    Germany
    Posts
    4,061

    Default Re: partition/volume

    Quote Originally Posted by pattiM View Post
    ...so you mean "...it's relevant only to the part of the BtrFS filesystem on a particular device..." ?? (there aren't two filesystems in my proposed setup - or are there?)
    Correct – the Btrfs RAID will contained striped user files (only one copy visible to the users) but, the Btrfs Metadata will held per SSD (one copy per SSD).

Page 3 of 4 FirstFirst 1234 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •