Want to put an END TO BTRFS VOLUME FULL PROBLEMS

While trying to install openSUSE I found that the partitioner was suggesting partition scheme at pen drive itself. It was due to a bug on part of windoze >:)
thread describing the problem (It’s worth breaking head for the problem as it’s a bug on windoze side and can’t do anything about it. I’m just posting it for reference)

But that’s not why I’m posting here.

I’m posting here because I was disturbed to see that the partitioner was proposing a btrfs for a partition size of ~10GB. It’s extremely small size of for BTRFS

Also there has been a alot of threads describing volume full problem due to snapshots.

To overcome that I propose we make a openFATE feature request

  1. To not to propose BTRFS partition if partition size is less than a certain threshold limit - say 40 GB (if you feel the partition size is less or more - comment).
  2. We can also propose to turn off the snapshot feature by default.

Any Comments? Need more teeth to the given feature?

Sorry I was following Btrfs-Why Thread but I had nothing to contribute as I had never used BTRFS. :smiley:

On 2015-05-06 12:36, vish 99 wrote:
>
> While trying to install openSUSE I found that the partitioner was
> suggesting partition scheme at pen drive itself. It was due to a bug on
> part of windoze >:)
> ‘thread describing the problem’ (http://tinyurl.com/ox54f9l) (It’s worth
> breaking head for the problem as it’s a bug on windoze side and can’t do
> anything about it. I’m just posting it for reference)

No bug; intentional Windows feature >:-)

It is the same thing as if in Linux you install using LVM: Windows can
not see the volumes inside, even if they are plain FAT. LVM is the Linux
equivalent to Windows dynamic partitions.

> But that’s not why I’m posting here.
>
> I’m posting here because I was disturbed to see that the partitioner was
> proposing a btrfs for a partition size of ~10GB. It’s extremely small
> size of for BTRFS

That one IS an installation bug, so it should be reported in Bugzilla.
Perhaps on usability, dunno.

> - We can also propose to turn off the snapshot feature by default.

Well, that’s an intentional feature the developers want. It is one of
the main reasons for deciding to use btrfs as default. It permits such
things as roll back a faulty update, for instance. It may even be a
point in the official release announcement.

It would make sense, though, to turn them off if the partition is too
small for them, because it can’t work, anyway.

Me, I would consider having them on for the documents directory (not
possible), because I would get a version-backup feature which is
interesting for office work.


Cheers / Saludos,

Carlos E. R.
(from 13.1 x86_64 “Bottle” at Telcontar)

That’s what I meant to say. At installation, partitioner should disable snapper if the partition size is small. But, what size would you define small?

No, not a bug. It is possible to install 13.2/KDE in a 10GB partition with Btrfs, I have one installed for testing in a VM and kept regularly updated. It’s useful for comparison if an issue crops up on my Tumbleweed system, and the partition is only 9 GiB.

Of course it wouldn’t last long if I hadn’t drastically reduced the pre-post snapshot retention, and stopped time-line snapshots altogether.

The default Snapper configuration (/etc/napper/configs/root) remains the all important issue, whether one thinks Btrfs should be the default file system or not. It has recently been strongly discussed on the Mailing List anyway, and is a well-known issue.

On Fri, 08 May 2015 14:36:02 +0000, vish 99 wrote:

> That’s what I meant to say. At installation, partitioner should disable
> snapper if the partition size is small. But, what size would you define
> small?

It depends.

For some systems, 10 GB is just fine and btrfs won’t create a problem for
you.

For other systems, 10 GB is way, way too small and btrfs will create a
problem for you.

The installer lacks something: Knowledge about how you intend to use
(and ultimately end up using - plans don’t always go to plan) the
system. That’s only something the user can plan for at installation time
based on their needs.

I would far rather have the installer not try to guess what I intend when
I build a system.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

What I’m to propose is that if, Partition Scheme suggested by installer has root size less than 40 GB then it it should turn off snapper.

And if the user is proposing advance partition scheme then it should give the option (in the edit option of the partition) to user whether he wants the snapper function or not.

NOTE: This feature is meant for noobs who rely on automatic partition scheme.

Assumption - Average user would install 2-3 DEs and have 2-3 games.

AFAIK it does turn off snapper automatically below a certain threshold.
At least it did when I last did a fresh install in VirtualBox with a ~8GiB root partition.

I don’t know what that threshold is though, maybe it should be changed.

And if the user is proposing advance partition scheme then it should give the option (in the edit option of the partition) to user whether he wants the snapper function or not.

If you edit the partition scheme, there should already be a checkbox whether snapshots should be enabled or not, if btrfs is selected.

NOTE: This feature is meant for noobs who rely on automatic partition scheme.

Assumption - Average user would install 2-3 DEs and have 2-3 games.

Well, I’m not so sure whether the average user would install 2-3 DEs…
But anyway.
IMHO, the main problem is with installing a lot of updates. (a snapshot is created before and after installing updates, and especially updates modify the/a lot of files on the root partition)

Probably there is no problem with the default snapper settings if you don’t use any additional repos at all.
But then, most users probably do need Packman, and particularly there can be a lot of (big) updates…

On Mon, 11 May 2015 07:06:02 +0000, vish 99 wrote:

> What I’m to propose is that if, Partition Scheme suggested by installer
> has root size less than 40 GB then it it should turn off snapper.

Some systems would be fine with not disabling it. “It depends” applies
here.

> And if the user is proposing advance partition scheme then it should
> give the option (in the edit option of the partition) to user whether he
> wants the snapper function or not.
>
> NOTE: This feature is meant for noobs who rely on automatic partition
> scheme.
>
> Assumption - Average user would install 2-3 DEs and have 2-3 games.

I think that’s probably not a good assumption - a new user who’s getting
into Linux is going to probably continue to use the system, and shouldn’t
bump up against problems when installing more stuff.

These are also users who are more likely to benefit from snapshot
availability in order to fix something they break inadvertently because
of their inexperience.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

I’ve also been cut with this issue. One time system would’t boot anymore. I didn’t first have a clue that it was because of disk space was full because of snapper snapshots. Error messages were not about disk space, systemd services just failed to start. I was using 25gb partition for / with separated /home which was fine for ext4.

IMO Yast should give user warning at installation when root partition isn’t big (like <40gb) and btrfs and snapper are going to get enabled. It should inform the user about the extra space those are going to need, and could give an option to turn snapper off or use another fs.

Regular user may just google for convenient partition size and probably finds many suggestions that 10-20gb is perfectly fine or use same about he or she is used to with other fs. Little does he or she know that this doesn’t apply for brtfs + snapper.

There is an option to to turn off snapshots in the installer, and there is an option to use another fs.
If snapshots are enabled, the installer proposes a bigger root partition size too (40GB).

And they are turned off automatically if the partition is below a certain size.
I think the size limit is 10GB (i.e. snapshots are turned off if you install to a root partition with less than 10 GB).

Although, things could definitely be improved, I suppose.

Personally,
Have been studying BTRFS “Full” issues,

First, if installed in a partition that small, openSUSE should already be proposing a partitioning layout <without> a separate /home partition to be most efficient utilizing limited total disk space.

I’ve been contemplating the following…

  • A separate partition for BTRFS snapshots. It has always been a recognized valid reason for creating a partition to separate anything that might grow excessively from the OS, applications and at times data to ensure normal operations isn’t affected.
  • I have been reviewing the current 10/10/1 algorithm for saving snapshots, I have another idea in mind that isn’t based on saving specific snapshots but is directly based partly on the overall size of the proposed dedicated partition and current free space before applying any other rules about preserving snapshots.

From what I’ve read, it shouldn’t be difficult to insert a custom algorithm.
For now, I’m at the conceptualizing stage and haven’t created any code.
If anyone else is interested in working on this, I can set up and am willing to manage a github project to build something worthwhile.

TSU

On 2015-07-30 17:26, tsu2 wrote:
> - A separate partition for BTRFS snapshots. It has always been a
> recognized valid reason for creating a partition to separate anything
> that might grow excessively from the OS, applications and at times data
> to ensure normal operations isn’t affected.

I don’t think this is possible. Snapshots have to be in the same
partition, it works that way. Think of them as a kind of hard links.
Hard links can’t work across devices.

The definition is not exact, but it hints at the reason :slight_smile:


Cheers / Saludos,

Carlos E. R.
(from 13.1 x86_64 “Bottle” at Telcontar)

After some more reading, I haven’t found anything yet to suggest that snapshots can’t be stored in a separate partition but there is less of reason to do so. Snapshots are a special type of subvolume and when a snapshot is chosen to be utilized, it’s mounted. So, in theory (at least in my mind) since a mount can point almost anywhere (given expected permissions and a valid path) there isn’t likely an issue.

But, since the snapshot is mounted directly and not used to update a base file system, this invalidates my primary reason for considering a separate partition… essential running processes are not separated from environmental variables which might cause problems.

You might still be right, I just haven’t run into a technical description yet that says that snapshots have to be stored in the same partition as the file system it’s snapshotting.

Bottom line, if there is strong enough motivation to explore answers can be quickly found by simply deploying a test. But for now, the original motivation isn’t valid so for now isn’t a path worth exploring.

TSU

On 2015-07-31 18:56, tsu2 wrote:

> You might still be right, I just haven’t run into a technical
> description yet that says that snapshots have to be stored in the same
> partition as the file system it’s snapshotting.

With btrfs, yes, they must. I’ll try to explain.

When you do a snapshot what is done is tell the filesystem to mark the
filesystem status, but no file is written to the snapshot directory.
There is no movement or copying of data.

What happens is that the instant something is written, those new writes
go to new disk space, and the old chunks are simply addresses from… I
don’t know how to call that, structures from the snapshots that point to
them.

That is, in the same partition there are lists of pointers to the chunks
that constitute the new file, and another lists of pointers to the old
chunks of the previous version of the same files.

This can only work if all is in the same btrfs structure.

It can be several partitions, even several devices, but using something
akin to LVM, where you don’t know what is going where. In any case, once
a file is written, those chunks stay for life where they were initially
written to. Another version of the file may go elsewhere. Then there is
balancing.

That’s why you can not place the btrfs snapshots into a different
partition. This has been asked before, by the way; I don’t have a link
to point you to, but I have read it previously.


Cheers / Saludos,

Carlos E. R.
(from 13.1 x86_64 “Bottle” at Telcontar)

Personally, BTRFS is not ready for prime time as it can’t handle a 1tb hdd on an Acer Aspire E1-572 but change to ext4 and all is well.

On Wed, 14 Oct 2015 22:26:02 +0000, techwiz03 wrote:

> Personally, BTRFS is not ready for prime time as it can’t handle a 1tb
> hdd on an Acer Aspire E1-572 but change to ext4 and all is well.

What kind of load are you putting on it, and how is it configured?

I use it here just fine on my root partition - but I wouldn’t use it (no
matter how mature) on a volume that holds, for example, virtual disk
images. It’s not designed for use in that kind of setup.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

i’m now on btrfs + openSUSE leap 42.1 beta 1
root is 15GB big, and changed the snapper config to limit=5 (instead of default 10)
everyday do zypper dup with all my repo’s.
ruuning just fine and the left space is on ~6GB

I can believe it, and no Timeline snapshots (by default). I tend to have root partitions at 20GB these days, including Leap Beta1.

For a full openSUSE version upgrade (with zypper dup), say around 2000 package changes, your configuration with 15GB could easily run out of space. For Tumbleweed it wouldn’t be enough for its frequent updates using zypper dup.

My worst case so far was upgrading old Tumbleweed to re-base on 13.2, then onto new factory-based Tumbleweed, without space issues but Btrfs brought the last unused 5GB of the 35GB partition into its pool.

It came up as the default option of live install from a kde-live DVD . The plan was to install linux and
within the os run vm’s but I could not get install to finish without errors until I switched all partitions
being done to ext4.

On Thu, 15 Oct 2015 15:56:01 +0000, techwiz03 wrote:

> It came up as the default option of live install from a kde-live DVD .
> The plan was to install linux and within the os run vm’s but I could not
> get install to finish without errors until I switched all partitions
> being done to ext4.

Sure, using it as a host for virtual machines or other high-intensity
file I/O isn’t what btrfs is designed for.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C