btrfs is thrashing a volume wildly

opensuse v12.3
v3.7.10-1.16-desktop x86_64
amd athlon ii x4
8 GB RAM

$ btrfs filesystem show
failed to read /dev/sr0
Label: none  uuid: 098fea71-c70f-4fa1-854c-71d1fee8a999
        Total devices 1 FS bytes used 1.81GB
        devid    1 size 191.58GB used 12.04GB path /dev/sdb8

Label: none  uuid: 0617578a-edca-4e3b-9100-729ae05c9057
        Total devices 1 FS bytes used 22.26GB
        devid    1 size 40.00GB used 24.54GB path /dev/sdb7
Btrfs v0.19+

$ btrfs filesystem df /
Data: total=21.01GB, used=20.60GB
System, DUP: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, DUP: total=1.75GB, used=1.66GB
Metadata: total=8.00MB, used=0.00

I installed opensuse using btrfs. (I was unaware that it is considered beta.) It has worked fine for months. Now it has taken to thrashing wildly to do anything whatsoever and takes several seconds to minutes to accomplish actions as simple as changing window focus. A disk activity monitor shows an average of 3MB/sec transfer during to whole (in-)activity period.

There are 3 partitions on the drive: /, /home, /boot. Only the root (/) partition (/dev/sdb7) suffers from this malady; it is 41 GB and 60% used.
Smartctl has found no problem with the disk.
The swap disk is unused.
Rebooting takes up to a half hour and makes no difference.

Is there a (relatively) simple fix for this?

If not, where is a How-To or a FAQ that describes in extreme detail how to copy a system disk to another?

On the new drive, I assume I would have to create the partitions, format them (not with btrfs!), mount them, copy the volume contents, change fstab and grub, reboot. Unless there are specific tools that handle a lot the detail?

On 2013-07-24 02:06, jimoe666 wrote:
> opensuse v12.3

That’s not the beta version.


Cheers / Saludos,

Carlos E. R.
(from 11.4, with Evergreen, x86_64 “Celadon” (Minas Tirith))

Will you please p**s off? You responses here and in the “Install/…” forum have been completely devoid of information or assistance.

On Wed, 24 Jul 2013 02:50:28 GMT
“Carlos E. R.” <robin_listas@no-mx.forums.opensuse.org> wrote:

> On 2013-07-24 02:06, jimoe666 wrote:
> > opensuse v12.3
>
> That’s not the beta version.
>

But he’s saying btrfs is beta. So, where does his post belong if not
here? If you have a good reason for believing it belongs elsewhere,
perhaps you could be a little more helpful and suggest where?

I’m rather interested in your response as I have a problem with KDE 4.11
and, until your response here, I would have raised the problem in this
newsgroup.


Graham P Davis, Bracknell, Berks.
openSUSE 12.3 (64-bit); KDE 4.10.5; AMD Phenom II X2 550 Processor;
Kernel: 3.10.0; Video: nVidia GeForce 210 (using nouveau driver);
Sound: ATI SBx00 Azalia (Intel HDA); Wireless: BCM4306

He is saying that “opensuse 12.3” is not a beta version. So what? I am asking about btrfs; the btrfs version is indicated in the listings provided.

He apparently did not read any further than the first line. His other responses in https://forums.opensuse.org/english/get-technical-help-here/install-boot-login/488932-btrfs-doin-me-wrong-thrashing-wildly.html were equally unhelpful. In what way does his statement further my quest for a resolution to this problem?

Since btrfs is a beta product, I would say that this would be the place to pose your question.

On Wed, 24 Jul 2013 05:46:01 +0000, jimoe666 wrote:

> robin_listas;2574020 Wrote:
>> On 2013-07-24 02:06, jimoe666 wrote:
>> > opensuse v12.3
>>
>> That’s not the beta version.
> Will you please p**s off? You responses here and in the “Install/…”
> forum have been completely devoid of information or assistance.

Guys, again, let’s keep things civil.

Second warning. Again, I understand the frustration, but engaging in
personal attacks violates our terms and conditions.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

On Wed, 24 Jul 2013 08:16:02 +0000, jimoe666 wrote:

> Cloddy;2574041 Wrote:
>> >
>> > > opensuse v12.3
>> >
>> > That’s not the beta version.
>> >
>> But he’s saying btrfs is beta. So, where does his post belong if not
>> here? If you have a good reason for believing it belongs elsewhere,
>> perhaps you could be a little more helpful and suggest where?
> He is saying that “opensuse 12.3” is not a beta version. So what? I am
> asking about btrfs; the btrfs version is indicated in the listings
> provided.
>
> He apparently did not read any further than the first line. His other
> responses in http://tinyurl.com/l84y4km were equally unhelpful. In what
> way does his statement further my quest for a resolution to this
> problem?
>
> Since btrfs is a beta product, I would say that this would be the place
> to pose your question.

The typical “rule of thumb” here is that this forum is for beta releases,
not beta features. That said, I don’t see a problem with the post being
here, however you might get more useful answers in the installation
forum. BTRFS itself is still essentially a technology preview, but it’s
included in a release that’s non-beta.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

On 2013-07-24 15:59, Jim Henderson wrote:

> The typical “rule of thumb” here is that this forum is for beta releases,
> not beta features. That said, I don’t see a problem with the post being
> here, however you might get more useful answers in the installation
> forum. BTRFS itself is still essentially a technology preview, but it’s
> included in a release that’s non-beta.

Unfortunately, btrfs is undergoing very heavy development, and it has
features and behaviour in factory that 12.3 doesn’t have. Thus asking
about btrfs in the beta forum when you are using 12.3, is IMHO wrong.


Cheers / Saludos,

Carlos E. R.
(from 11.4, with Evergreen, x86_64 “Celadon” (Minas Tirith))

On Wed, 24 Jul 2013 14:37:23 +0000, Carlos E. R. wrote:

> On 2013-07-24 15:59, Jim Henderson wrote:
>
>> The typical “rule of thumb” here is that this forum is for beta
>> releases,
>> not beta features. That said, I don’t see a problem with the post
>> being here, however you might get more useful answers in the
>> installation forum. BTRFS itself is still essentially a technology
>> preview, but it’s included in a release that’s non-beta.
>
> Unfortunately, btrfs is undergoing very heavy development, and it has
> features and behaviour in factory that 12.3 doesn’t have. Thus asking
> about btrfs in the beta forum when you are using 12.3, is IMHO wrong.

Possibly, though in my mind it’s kinda sixes - it depends on whether one
sees this forum as “the place to ask about beta releases” or “the place
to ask about beta features and releases”. The latter certainly is a
valid interpretation, but not a common one.

Given that btrfs is under heavy development still, I can understand why
one might consider this the appropriate place to ask.

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

I ran “btrfs filesystem balance start /” a few times. That has made the system somewhat usable but it is still sluggish due to excessive disk activity.

Any suggestions for restoring btrfs to its usual spritely performance?

On 7/24/13 4:06 PM, jimoe666 wrote:
>
> I ran “btrfs filesystem balance start /” a few times. That has made the
> system somewhat usable but it is still sluggish due to excessive disk
> activity.
>
> Any suggestions for restoring btrfs to its usual spritely performance?

The next openSUSE 12.3 update comes with a massive btrfs update that
includes fixes and updates through the 3.10 kernel as well as a fix for
an ENOSPC that blocks users from removing snapshots.

By btrfs standards, the 3.7 implementation is “quite old.”

If you’re interested in confirming whether it will fix your issue, you
can grab the kernel update early via KOTD:
http://download.opensuse.org/repositories/Kernel:/openSUSE-12.3/standard/

That repository contains the latest kernels as we commit them (with a
minor delay for building).

-Jeff


Jeff Mahoney
SUSE Labs

I deleted all of the snapshots. That changed (after about 20 minutes)
Data: total=21.00GB, used=20.1GB
to
Data: total=21.00GB, used=10.58GB

The snapshots used 1/2 the available space; there were 138. Seems a bit excessive.

Which of these should I believe? “btrfs filesystem df” gives wildly different space usage than “df”. In particular the difference between btrfs and df for “/home” is dramatic. I am aware that the docs mention that df is unreliable for btrfs, but a 188GB error?

$ df -h / /home /boot
 Filesystem      Size  Used Avail Use% Mounted on 
/dev/sdb7        41G   12G   25G  33% / 
/dev/sdb8       192G  1.9G  182G   2% /home 
/dev/sdb5       151M  108M   35M  76% /boot
$ btrfs filesystem df / 
Data: total=21.00GB, used=10.58GB 
System, DUP: total=32.00MB, used=4.00KB 
System: total=4.00MB, used=0.00 
Metadata, DUP: total=2.25GB, used=1.35GB 

$ btrfs filesystem df /home 
Data: total=4.01GB, used=1.76GB 
System, DUP: total=8.00MB, used=4.00KB 
System: total=4.00MB, used=0.00 
Metadata, DUP: total=4.00GB, used=47.96MB 
Metadata: total=8.00MB, used=0.00 

$ btrfs filesystem df /boot 
ERROR: couldn't get space info on '/boot' - Inappropriate ioctl for device 

On 7/25/13 1:26 AM, jimoe666 wrote:
>
> I deleted all of the snapshots. That changed (after about 20 minutes)
> Data: total=21.00GB, used=20.1GB
> to
> Data: total=21.00GB, used=10.58GB
>
> The snapshots used 1/2 the available space; there were 138. Seems a bit
> excessive.
>
> Which of these should I believe? “btrfs filesystem df” gives wildly
> different space usage than “df”. In particular the difference between
> btrfs and df for “/home” is dramatic. I am aware that the docs mention
> that df is unreliable for btrfs, but a 188GB error?

There’s a difference between the outputs.

Btrfs divides its space up into ‘chunks’ that are used for metadata,
data, or system space (where system contains things like tree roots and
is usually quite small.) Individual block allocations come out of these
chunks. It works well to keep metadata and data in separate areas of the
disk since, due to the CoW nature of btrfs, the metadata areas will
fragment at a much faster rate than data. By avoiding interleaving the
allocations, we get less fragmentation for the data chunks.

btrfs df is showing the chunks that are allocated. The ‘missing’ space
is entirely unallocated since the file system isn’t full enough to have
needed more space yet.

In your Data: example, above, it didn’t free 50% of available space, it
freed 50% of allocated data chunks. There was still another 20GB of
completely unallocated space.

It’d probably be useful to make this distinction more explicit with an
“Unallocated” line in ‘btrfs fi df’ output.

-Jeff

>
> Code:
> --------------------
> $ df -h / /home /boot
> Filesystem Size Used Avail Use% Mounted on
> /dev/sdb7 41G 12G 25G 33% /
> /dev/sdb8 192G 1.9G 182G 2% /home
> /dev/sdb5 151M 108M 35M 76% /boot
> --------------------
>
>
> Code:
> --------------------
> $ btrfs filesystem df /
> Data: total=21.00GB, used=10.58GB
> System, DUP: total=32.00MB, used=4.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=2.25GB, used=1.35GB
>
> $ btrfs filesystem df /home
> Data: total=4.01GB, used=1.76GB
> System, DUP: total=8.00MB, used=4.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=4.00GB, used=47.96MB
> Metadata: total=8.00MB, used=0.00
>
> $ btrfs filesystem df /boot
> ERROR: couldn’t get space info on ‘/boot’ - Inappropriate ioctl for device
>
> --------------------
>
>

$ df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb7        41G  6.8G   30G  19% /

btrfs fi df /
  Data: total=21.00GB, used=5.94GB
  System, DUP: total=32.00MB, used=4.00KB
  System: total=4.00MB, used=0.00
  Metadata, DUP: total=2.25GB, used=395.02MB

Okay. So the 41GB partition is the real size, just that btrfs has not yet allocated the remaining 20GB?

If so, then why did btrfs not allocate more chunks when it filled the allocated space with snapshots? This was the source of all the thrashing I reported. After deleting the snapshots, the system’s performance returned to normal.

On 7/25/13 1:56 PM, jimoe666 wrote:
>
> jeff_mahoney;2574411 Wrote:
>> btrfs df is showing the chunks that are allocated. The ‘missing’ space
>> is entirely unallocated since the file system isn’t full enough to have
>> needed more space yet.
>
> Code:
> --------------------
> $ df -h /
> Filesystem Size Used Avail Use% Mounted on
> /dev/sdb7 41G 6.8G 30G 19% /
>
> btrfs fi df /
> Data: total=21.00GB, used=5.94GB
> System, DUP: total=32.00MB, used=4.00KB
> System: total=4.00MB, used=0.00
> Metadata, DUP: total=2.25GB, used=395.02MB
> --------------------
> Okay. So the 41GB partition is the real size, just that btrfs has not
> yet allocated the remaining 20GB?
>
> If so, then why did btrfs not allocate more chunks when it filled the
> allocated space with snapshots? This was the source of all the thrashing
> I reported. After deleting the snapshots, the system’s performance
> returned to normal.

Because correlation doesn’t equal causation. You mentioned the file
system was thrashing. Not that it was thrashing on allocation or in a
low-space situation. I wouldn’t go jumping to that conclusion just yet.

The only data points you provided were that df reported your 41 GB file
system 60% full and that there was ~ 3MB/s activity on the disk. There’s
not enough detail in that report to draw any conclusions about causes.

-Jeff

From my original posting:

$ btrfs filesystem df / 
Data: total=21.01GB, used=20.60GB 
System, DUP: total=8.00MB, used=4.00KB 
System: total=4.00MB, used=0.00 
Metadata, DUP: total=1.75GB, used=1.66GB 
Metadata: total=8.00MB, used=0.00

Total allocation was 21.0 GB, used was 20.6 GB. IMO low-space was the problem.

All I knew at the time of the posting was that the drive was thrashing; I did not know why. After deleting all those snapshots, there was a lot more unused space and the system became responsive again. I am inclined to point to low-space as the problem in this instance.

Which still leaves the question of why more space was not allocated. There is 20 GB waiting for allocation.