Decision to use BTRFS in 13.2?

English Install/Boot/Login
1

This is the earliest I’ve ever taken a look at a distro in development, openSUSE 13.2 milestone 0.

Installed into VMware Workstation 10.

Amused to see the mixture of 12.3 graphics and 13.1 documentation, but I understand those are minor issues at this point.

Am most interested in finding BTRFS is the default FS, is BTFRS considered sufficiently stable today for likely implementation or is it mainly offered for wide spread testing at this point?

As soon as I saw BTRFS, I started looking up recent information about it…

Am also curious about implementation of the tmpfs mount points (7 of them! plus the swap and physical disk), they are 999mb each although I only allocated 8GB physical disk and 2GB RAM total to the VM. Assuming those tmpfs partitions are maximum and not fully allocated values (how could they be?) and not knowing why each of those mount points were created, I wonder if fewer mount points might have some benefits? So, for instance on a physical disk drive creating so many partitions causes inefficient use of space and potential out of space issues… I don’t know if that would be true for tmpfs. Similarly, I’d be curious about the reason for creating so many mount points, on a physical drive reasons could include isolation (isolating errors to what is in that partition), controlling fragmentation, minimizing maintenance issues (data on different partitions managed differently).

In fact, the errors and overall scenario described in the article written a few weeks I referenced above might have been controlled by better partitioning.

I’ve also played around a little bit with the ZFS Linux kernel extensions for similar benefits BTFRS provides, ie volumes across multiple devices, RAID volumes, snapshots.

TSU

2

The installer wanted to use “btrfs” for 13.2M0, but I told it to use “ext4” instead. I’m guessing that I will have to pay attention during installs.

I tried “btrfs” during the pre-release testing of 13.1, but decided that it was not for me. I might consider trying again for further pre-release testing of 13.2 - still waiting for Milestone1 here. My plan is for “ext4” on the final release.

Concerns that I saw with “btrfs” and 13.1 milestones:

  • a lot of space is tied up with snapshots;
  • it seemed to interfere with grub (probing for other linux systems did not work properly);
  • there were no obvious benefits from using “btrfs” - it seemed that I would have to learn quite a bit to take advantage of the snapshots.
3

BTRFS looks to be the new default. I think it is a bad idea. It does appear to be stable now and the recovery utilities are improving. But the problem will be that new users who do not understand the ramification of snapshots are going to be bitten with not having enough space allocated. At this point it is unclear how much space would be needed since snapshot size is dependent on the amount of changes you make to the files. IMO snapshots should be .set off by default and if a user wants them they can turn them on presumably knowing the ramifications. If default is on we will be very busy here on the board about 4-6 weeks after the release when people start running out of space and standard tools say they have plenty. Perhaps a warning at install may medicate the problems but I doubt it toaster users generally don’t read :’(

BTRFS looks to be good for complex installation like servers but I really doubt the utility for a simple Desktop.

Tmpfs memory is shared I believe and when the physical is used up it will pop into swap. I suspect it may be problematic in very low memory systems. Great if you have 8+ gig not so much if only 1

4

I agree.

And with the concerns of tsu2 and nrickert.

And with this:

Perhaps a warning at install may (mitigate) the problems but I doubt it toaster users generally don’t read

5

Maybe the installer should check whether the machine is amd64 and the machine should have at least 4GB RAM and at least 200GB free HD space to default for BTRFS otherwise it defaults to use ext4?

6

While I am looking forward to having a more in depth play with BTRFS, I think that you are exactly right here. Newbs will select BTRFS (or, more exactly, not do anything to deselect BTRFS), not really be concious of anything unusual having happened and then be very surprised when it goes pear shaped some time later.

One question, though: are any of the existing tools being re-written to give more information in the case of snapshots being active (or new utils being written)? One thing that I don’t know is whether there will be new versions of things like du and df that will either go to ‘extra information mode’ if snapshots are present, or show zero space for snapshots, if snapshots aren’t active. Its probably potentially a bit of a mess either way, with either backward compatibility or deceptive information being issues. And doesn’t that part of the issue exist for LVM already?

Another thing to get ahead might be a wiki page for the ‘Why Haven’t I Got the Free Disk Space That I Think I Should’ info; at least then, the question could be answered with a simple link. I doubt it will do much to reduce the number of questions, but there will be less writing in each answer. But still, some medication could be required…(but then, I thought that about systemd, and that wasn’t as bad as expected).

7

On 2014-05-24 02:06, gogalthorp wrote:

> BTRFS looks to be good for complex installation like servers but I
> really doubt the utility for a simple Desktop.

It could be interesting for external backup media, because you
automatically get current and old versions of each file (ok, you need to
trigger one snapshot in between backups).

Another interesting point is that it supports compression (not stable
yet, but not sure about it). This is a feature that Windows has had for
more than a decade, natively, in ntfs. Ext2/3/4 had provision of it, but
it was never implemented.

A read/write compressed filesystem is interesting for storage of things
that can be compressed a lot, like mail, or logs. And backups, of
course, for use with rsync or simimilar, instead of tar.gz.

–
Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))

8

On 2014-05-24 09:06, FurciferPardalis wrote:
>
> Maybe the installer should check whether the machine is amd64 and the
> machine should have at least 4GB RAM and at least 200GB free HD space to
> default for BTRFS otherwise it defaults to use ext4?

That’s an interesting idea.

–
Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))

9

Been using btrfs with Snapper on logical root partion for Tumbleweed (12.3 through 13.1) where the updates are large, frequent, and done with zypper dup. Kept it down around 20GB partition, after reducing the default snapshot retention by >50%. Monitoring used space with btrfs command-line tools is not difficult. Not had any btrfs stability issues, not doing compression or anything fancy.

No problems with Grub2 in my multi-boot configuration. Grub2 is in a separate /boot, logical partition (512MB) belonging to Tumbleweed. No problems with Grub2 management, although the other partitions have all been current and older openSUSE releases.

Noticed some extra overhead on my notebook wrt disk i/o activity, at boot time and immediately after updates.

Also concerned about btrfs as default. More concerned about Snapper as default for btrfs on root partition, especially if the current snapshot retention levels are retained for 13.2 and Snapper is an “opt-out” feature. It should be an “opt-in”, especially for the default desktop system.

10

I think that is a good suggestion.

But, I still think BTRFS should not be the default choice, in any case. The seasoned users can always make a conscious choice to use it when they install, if they feel it is superior for their needs. And, it is the seasoned users who will stop to consider such things when installing.

11

All bells and whistles should be turned off if the BTRFS is made the default. Some kind of note letting users know that BTRFS is active and it has great features if you want to use them and a pointer to a GUI (Yast maybe) where you can turn things on and fine tune.

Cramming new features down new users throats is too MS for me. So far I have seen no real way of estimating disk usage and I don’t see normal Linux command line tools being fixed. There are a set of tools for BTRFS and they work IF you know about them.

12

For those interested in performance,
Here are some marks using the Phoronix Test Suite done in Aug 2013 on a single disk system.
http://www.phoronix.com/scan.php?page=article&item=zfs_linux_062&num=1

May not be too surprising to some, compared to some other older published reports BTRFS and ZFS both made significant improvements trying to catch up to EXT4, but in practically every test in every published report I’ve read EXT4 has always been first in every test.

That said, I believe on <multi-disk> systems there are some really interesting features that both BTRFS and ZFS provide.

Also, if someone installs one of these next-gen fs for their versioning/snapshotting capability but want the stability and reputation of ext4, you can get undelete capability (not quite the same as long term versioning) with extundelete
http://extundelete.sourceforge.net/

It works as an application that tracks file changes on ext fs, the metatdata and file snapshots are generated as a cron job (not instantaneously, but there is usually only an imperceptible penalty to running the cron job <very> frequently). Last I looked at this, it works fantastic on a single disk system, there was a bug pointing to any other but the first disk.

Perfect for anyone who wants a “better” trashcan preserving undelete capability no matter what the desktop or other applications running.
Very easy to setup with very reasonable defaults, I recommend it highly.

TSU

13

Just my point of view: ext4 should be the default fs: it’s stable, well tested, easy to handle.
Expert users can switch to btrfs during the setup.

14

On Fri, 23 May 2014 23:26:01 +0000, tsu2 wrote:

> Am most interested in finding BTRFS is the default FS, is BTFRS
> considered sufficiently stable today for likely implementation or is it
> mainly offered for wide spread testing at this point?

Yes, it is considered stable enough with the selected features selected.
This was a topic of discussion when 13.1 came out, because rather late in
the game, it was suggested that it was ready to be the default at that
point (except the decision had already been made).

There are some who will say that they know of ways to corrupt btrfs and
have bugs (Carlos, notably, has a reproducible issue) - but in general,
it has proven itself to be stable enough.

If I move to 13.2 myself (I usually skip every other version on my main
systems since I use them for work, and like to keep things stable for 2
release cycles), I’ll probably migrate my partitions to btrfs so I am
running it myself.

That said, as others have mentioned, snapshots themselves can be
problematic if the user is unaware of them on the root partition - and
there had been some talk about fine-tuning the defaults for root
partitions in order to avoid running out of space on undersized root
partitions.

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

15

On 2014-05-27 01:33, Jim Henderson wrote:

> There are some who will say that they know of ways to corrupt btrfs and
> have bugs (Carlos, notably, has a reproducible issue) - but in general,
> it has proven itself to be stable enough.

LOL - I do!

Just for kicks, I recently found a new one on reiserfs. It locks the
kernel dead, randomly, after using dd with “conv=fdatasync”, which is
not a thing normally done, I agree. Took me a month to hunt it down.


> [18892.976044] BUG: unable to handle kernel paging request at ffffc90012825250
> [18892.977001] IP: <ffffffff8105e7a9>] get_next_timer_interrupt+0xa9/0x270
> [18892.977001] PGD 23f027067 PUD 23f028067 PMD 22b52e067 PTE 0
> [18892.977001] Oops: 0000 #1] PREEMPT SMP

I just mention this because, no filesystem we have is totally safe.
Years ago, I also managed to fully crash XFS, too. That one took years
to find out why and solve it. And before that, at the heyday of
reiserfs, I remember we found also a crash problem with it: just
creating two files with a particular pattern of names would do it. It
thought the names were the same and clashed, when obviously they were
not - because it used some clever name storage method that had still
problems.

Sigh… I hate finding those problems. O:-)

And… btrfs has some features I would love. I want to use it! But this
time I’ll stay behind while you people iron it out for me :wink:

–
Cheers / Saludos,

Carlos E. R.
(from 13.1 x86_64 “Bottle” at Telcontar)

16

But neither of those file systems are the default file system. :smiley:

Since the current default file system has been ext4 for a few years now, have you found any serious software corruptions with it?

17

On 05/27/2014 08:26 AM, consused pecked at the keyboard and wrote:
> robin_listas;2645665 Wrote:
>> Just for kicks, I recently found a new one on reiserfs. …
>>
>> I just mention this because, no filesystem we have is totally safe.
>> Years ago, I also managed to fully crash XFS, too…
> But neither of those file systems are the default file system. :smiley:
>
> Since the current default file system has been ext4 for a few years now,
> have you found any serious software corruptions with it?
>
>

Apparently some people haven’t heard the phrase:

“If it ain’t broke don’t fix it”

and instead are of the mindset:

“If it ain’t broke lets break it”

or use the excuse that Microsoft has had the ability for years.
Microsoft has needed the ability for rollback/recovery for years because
the OS blows up at times for no apparent reason and needs to be able to
recover by rolling back changes. Has linux now become so unstable that a
way to “rollback/recover” is becoming a necessity? I hope not.

Ken

18

On Tue 27 May 2014 01:06:08 PM CDT, Ken Schneider wrote:

On 05/27/2014 08:26 AM, consused pecked at the keyboard and wrote:
> robin_listas;2645665 Wrote:
>> Just for kicks, I recently found a new one on reiserfs. …
>>
>> I just mention this because, no filesystem we have is totally safe.
>> Years ago, I also managed to fully crash XFS, too…
> But neither of those file systems are the default file system. :smiley:
>
> Since the current default file system has been ext4 for a few years
> now, have you found any serious software corruptions with it?
>
>

Apparently some people haven’t heard the phrase:

“If it ain’t broke don’t fix it”

and instead are of the mindset:

“If it ain’t broke lets break it”

or use the excuse that Microsoft has had the ability for years.
Microsoft has needed the ability for rollback/recovery for years
because the OS blows up at times for no apparent reason and needs to be
able to recover by rolling back changes. Has linux now become so
unstable that a way to “rollback/recover” is becoming a necessity? I
hope not.

Ken

Hi
That’s why btrfs has snapshots :wink: But this has been lacking (rug had
rollback for package install) for years, btrfs also has the ability to
boot from a snapshot.

But like you say, “If it ain’t broke, lets break it” to me are corner
cases and can also send the the developers, bug handlers, packagers
etc to IMHO a basically time wasting exercise which does no one any
good.

Having said that, if the concerned user provides solid details in their
use case (rather than breaking because they can) in normal daily tasks
that will (could?) benefit other users, then for sure spend time on it.

I will soon have four systems all running btrfs along with UEFI/Secure
boot, systemd etc running and have yet to have any filesystem crashes
or issues, aside from the default snapper config, which is an easy fix.

–
Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
openSUSE 13.1 (Bottle) (x86_64) GNOME 3.10.1 Kernel 3.11.10-11-desktop
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!

19

If free space is a paramount <known> issue and recovery is either immensely difficult or impossible without major loss (eg deleting snapshots which might be required to recover),

IMO BTRFS <must> be required to implement a preventative measure. Many possible options and methods can be considered possibly including

  • A calculated “reserve” space(adjustable and mandatory quotas?) that permits the system to still boot into at least text or recovery mode
  • A notification system about lack of free space which informs even a non-technical User a critical event is imminent
  • Link(s) to resources to resolve the issue (likely a wiki which can support evolving information)

Surely the three items I just listed cannot be difficult to implement since they build on known and proven existing utilities, they just need someone’s time. Just integrate this functionality so it’s guaranteed to be running and not an optional and possibly overlooked add-on or method. And, after some testing, maybe mandatory controls should be implemented to ensure scenarios do not occur.

From what I’ve read, the plus is that BTRFS snapshots and self-healing can avoid a number of problems.
The minus is that if problems should occur, usual utilities either do not work or do not exist.
That puts a premium on implementation and avoiding problems over fixing problems after they occur.

And until this approach is fully implemented, I don’t know that a system can be considered ready for Prime Time because every known problem scenario should be either avoided or reasonably fixable (and it should not matter how “corner” the problem is).

TSU

20

On 2014-05-27 14:26, consused wrote:
>
> robin_listas;2645665 Wrote:
>>
>> Just for kicks, I recently found a new one on reiserfs. …
>>
>> I just mention this because, no filesystem we have is totally safe.
>> Years ago, I also managed to fully crash XFS, too…
> But neither of those file systems are the default file system. :smiley:

Reiserfs was! At the time I crashed it. :slight_smile:

> Since the current default file system has been ext4 for a few years now,
> have you found any serious software corruptions with it?

Mmm. Yes, I once fully corrupted an ext3 filesystem, beyond repair.
Ext4… I don’t remember. I heard things, but I don’t remember suffering
those problems myself. That is, I don’t clearly remember what I had on
ext3 or 4.

These filesystems are all “smart”. They keep many things in ram. A crash
at the wrong moment, say power failure, graphic hardware or driver
seizeup… and the potential damage to the filesystems can be tremendous.

–
Cheers / Saludos,

Carlos E. R.
(from 13.1 x86_64 “Bottle” at Telcontar)