Page 1 of 2 12 LastLast
Results 1 to 10 of 15

Thread: btrfs frequently running out of space with lots unallocated

  1. #1
    Join Date
    Jun 2015
    Posts
    14

    Default btrfs frequently running out of space with lots unallocated

    Every time I do a system update, I run out of space on my root partition.

    btrfs is hellbent on using only 74GB out of 323GB total available on the partition so I have to then start deleting snapshots to try and complete the update.
    There must be something I'm missing so I'm hoping someone has any suggestions.

    Here's some output which might help diagnose this:

    Code:
    # btrfs device usage /
    /dev/mapper/cr_root, ID: 1
       Device size:           323.00GiB
       Device slack:              0.00B
       Data,single:            74.00GiB
       Metadata,single:         1.00GiB
       System,single:          32.00MiB
       Unallocated:           247.97GiB
    Code:
    # btrfs fi show /
    Label: none  uuid: fe9cc6a2-a504-4af7-8728-785068ad6dda
            Total devices 1 FS bytes used 73.44GiB
            devid    1 size 323.00GiB used 75.03GiB path /dev/mapper/cr_root
    Code:
    # blockdev --report
    RO    RA   SSZ   BSZ   StartSec            Size   Device
    rw  1024   512  4096          0    512110190592   /dev/nvme0n1
    rw  1024   512   512       2048       272629760   /dev/nvme0n1p1
    rw  1024   512  4096     534528        16777216   /dev/nvme0n1p2
    rw  1024   512  4096     567296    161061273600   /dev/nvme0n1p3
    rw  1024   512  4096  998166528      1048576000   /dev/nvme0n1p4
    rw  1024   512  4096  315140096    346818609152   /dev/nvme0n1p5
    rw  1024   512  4096  993972224      2147483648   /dev/nvme0n1p6
    rw  1024   512   512  992520192       743440384   /dev/nvme0n1p7
    rw  1024   512  4096          0    346816512000   /dev/dm-0
    I tried rebalancing to see what happens but the process ran out of space

    Code:
    # btrfs filesystem balance /              
    ERROR: error during balancing '/': No space left on device                                                                    
    There may be more info in syslog - try dmesg | tail

  2. #2
    Join Date
    Aug 2010
    Location
    Chicago suburbs
    Posts
    13,172
    Blog Entries
    3

    Default Re: btrfs frequently running out of space with lots unallocated

    I have seen some reports that there is a kernel bug (for "btrfs") involved in this.
    openSUSE Leap 15.1; KDE Plasma 5;
    testing Leap 15.2Alpha

  3. #3
    Join Date
    Jun 2015
    Posts
    14

    Default Re: btrfs frequently running out of space with lots unallocated

    Quote Originally Posted by nrickert View Post
    I have seen some reports that there is a kernel bug (for "btrfs") involved in this.
    It sounds similar to this: https://bugzilla.kernel.org/show_bug.cgi?id=74101
    But there's no help there and it appears to span multiple kernel versions (from 3 to, in my case, 5). Might be totally different issues with the same symptom.

  4. #4

    Default Re: btrfs frequently running out of space with lots unallocated


  5. #5
    Join Date
    Jan 2014
    Location
    Erlangen
    Posts
    1,186

    Default Re: btrfs frequently running out of space with lots unallocated

    Quote Originally Posted by laen View Post
    Every time I do a system update, I run out of space on my root partition.

    btrfs is hellbent on using only 74GB out of 323GB total available on the partition so I have to then start deleting snapshots to try and complete the update.
    There must be something I'm missing so I'm hoping someone has any suggestions.
    You may provide additional information:

    Code:
    erlangen:~ # btrfs filesystem usage -T /
    Overall:
        Device size:                  59.45GiB
        Device allocated:             15.03GiB
        Device unallocated:           44.42GiB
        Device missing:                  0.00B
        Used:                         13.08GiB
        Free (estimated):             44.83GiB      (min: 44.83GiB)
        Data ratio:                       1.00
        Metadata ratio:                   1.00
        Global reserve:               37.11MiB      (used: 0.00B)
    
                 Data     Metadata  System              
    Id Path      single   single    single   Unallocated
    -- --------- -------- --------- -------- -----------
     1 /dev/sdb5 13.00GiB   2.00GiB 32.00MiB    44.42GiB
    -- --------- -------- --------- -------- -----------
       Total     13.00GiB   2.00GiB 32.00MiB    44.42GiB
       Used      12.59GiB 502.31MiB 16.00KiB            
    erlangen:~ #
    Running "btrfs filesystem balance /" never worked on my machine. I always use "btrfs balance start -musage=50 /" and "btrfs balance start -musage=50 /". See also: https://ohthehugemanatee.org/blog/20...ency-response/
    AMD Athlon 4850e (2009), openSUSE 13.1, KDE 4, Intel i3-4130 (2014), i7-6700K (2016), i5-8250U (2018), openSUSE Tumbleweed, KDE Plasma 5

  6. #6
    Join Date
    Sep 2012
    Posts
    5,318

    Default Re: btrfs frequently running out of space with lots unallocated

    this is fixed since upstream kernel 5.5.2 and Tumbleweed kernel 5.5.1.

  7. #7
    Join Date
    Jun 2015
    Posts
    14

    Default Re: btrfs frequently running out of space with lots unallocated

    Quote Originally Posted by karlmistelberger View Post
    You may provide additional information:

    Code:
    erlangen:~ # btrfs filesystem usage -T /
    After running balance with the suggested musage (and I presume dusage):

    Code:
    # btrfs filesystem usage -T /
    Overall:
        Device size:                 323.00GiB
        Device allocated:             75.03GiB
        Device unallocated:          247.97GiB
        Device missing:                  0.00B
        Used:                         73.94GiB
        Free (estimated):            248.94GiB      (min: 248.94GiB)
        Data ratio:                       1.00
        Metadata ratio:                   1.00
        Global reserve:              126.08MiB      (used: 0.00B)
    
                           Data     Metadata  System              
    Id Path                single   single    single   Unallocated
    -- ------------------- -------- --------- -------- -----------
     1 /dev/mapper/cr_root 74.00GiB   1.00GiB 32.00MiB   247.97GiB
    -- ------------------- -------- --------- -------- -----------
       Total               74.00GiB   1.00GiB 32.00MiB   247.97GiB
       Used                73.03GiB 929.42MiB 16.00KiB
    In fact, after running balance, I have 0 available space once again.

    @arvidjaar, you have a link to the issue? Looks like the latest snapshot installs kernel 5.5.2 so I'll give that a try. Once I somehow clear enough space that is.
    Otherwise it's a wipe and move to ext4 but I'm not looking forward to the time spent doing that.

  8. #8
    Join Date
    Jan 2014
    Location
    Erlangen
    Posts
    1,186

    Default Re: btrfs frequently running out of space with lots unallocated

    Quote Originally Posted by laen View Post
    After running balance with the suggested musage (and I presume dusage):

    Code:
    # btrfs filesystem usage -T /
    Overall:
        Device size:                 323.00GiB
        Device allocated:             75.03GiB
        Device unallocated:          247.97GiB
        Device missing:                  0.00B
        Used:                         73.94GiB
        Free (estimated):            248.94GiB      (min: 248.94GiB)
        Data ratio:                       1.00
        Metadata ratio:                   1.00
        Global reserve:              126.08MiB      (used: 0.00B)
    
                           Data     Metadata  System              
    Id Path                single   single    single   Unallocated
    -- ------------------- -------- --------- -------- -----------
     1 /dev/mapper/cr_root 74.00GiB   1.00GiB 32.00MiB   247.97GiB
    -- ------------------- -------- --------- -------- -----------
       Total               74.00GiB   1.00GiB 32.00MiB   247.97GiB
       Used                73.03GiB 929.42MiB 16.00KiB
    In fact, after running balance, I have 0 available space once again.
    You may try the following command:

    Code:
    systemctl start btrfs-balance.service
    After completion display:
    Code:
    erlangen:~ # journalctl -u btrfs-balance.service 
    -- Logs begin at Fri 2020-02-14 07:59:41 CET, end at Sun 2020-02-16 17:42:02 CET. --
    Feb 15 09:34:45 erlangen systemd[1]: Started Balance block groups on a btrfs filesystem.
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Before balance of /
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Data, single: total=13.00GiB, used=12.45GiB
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: System, single: total=32.00MiB, used=16.00KiB
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Metadata, single: total=1.00GiB, used=501.11MiB
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: GlobalReserve, single: total=35.42MiB, used=0.00B
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Dateisystem    Größe Benutzt Verf. Verw% Eingehängt auf
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: /dev/sdb5        64G     14G   50G   23% /
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Done, had to relocate 0 out of 15 chunks
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Dumping filters: flags 0x1, state 0x0, force is off
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]:   DATA (flags 0x2): balancing, usage=5
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Done, had to relocate 0 out of 15 chunks
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Dumping filters: flags 0x1, state 0x0, force is off
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]:   DATA (flags 0x2): balancing, usage=10
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Done, had to relocate 0 out of 15 chunks
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Done, had to relocate 0 out of 15 chunks
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Dumping filters: flags 0x6, state 0x0, force is off
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]:   METADATA (flags 0x2): balancing, usage=3
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]:   SYSTEM (flags 0x2): balancing, usage=3
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Done, had to relocate 1 out of 15 chunks
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: After balance of /
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Data, single: total=13.00GiB, used=12.45GiB
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: System, single: total=32.00MiB, used=16.00KiB
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Metadata, single: total=1.00GiB, used=501.14MiB
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: GlobalReserve, single: total=35.44MiB, used=0.00B
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: Dateisystem    Größe Benutzt Verf. Verw% Eingehängt auf
    Feb 15 09:34:45 erlangen btrfs-balance.sh[20518]: /dev/sdb5        64G     14G   50G   23% /
    Feb 15 09:34:45 erlangen systemd[1]: btrfs-balance.service: Succeeded.
    erlangen:~ #
    AMD Athlon 4850e (2009), openSUSE 13.1, KDE 4, Intel i3-4130 (2014), i7-6700K (2016), i5-8250U (2018), openSUSE Tumbleweed, KDE Plasma 5

  9. #9
    Join Date
    Jun 2015
    Posts
    14

    Default Re: btrfs frequently running out of space with lots unallocated

    Quote Originally Posted by laen View Post
    @arvidjaar, you have a link to the issue? Looks like the latest snapshot installs kernel 5.5.2 so I'll give that a try. Once I somehow clear enough space that is.
    Nevermind. I see the links are in the reddit thread mentioned earlier.

    Thanks guys. Fingers crossed that's the issue I'm experiencing as well.

  10. #10
    Join Date
    Jun 2015
    Posts
    14

    Default Re: btrfs frequently running out of space with lots unallocated

    Quote Originally Posted by karlmistelberger View Post
    You may try the following command:

    Code:
    systemctl start btrfs-balance.service
    In case you're curious, it's still a mess:

    Code:
    Feb 16 12:04:30 thinkpad systemd[1]: Started Balance block groups on a btrfs filesystem.
    Feb 16 12:04:30 thinkpad btrfs-balance.sh[25953]: Before balance of /
    Feb 16 12:04:30 thinkpad btrfs-balance.sh[25953]: Data, single: total=73.00GiB, used=58.86GiB
    Feb 16 12:04:30 thinkpad btrfs-balance.sh[25953]: System, single: total=32.00MiB, used=16.00KiB
    Feb 16 12:04:30 thinkpad btrfs-balance.sh[25953]: Metadata, single: total=1.00GiB, used=910.11MiB
    Feb 16 12:04:30 thinkpad btrfs-balance.sh[25953]: GlobalReserve, single: total=110.73MiB, used=0.00B
    Feb 16 12:04:30 thinkpad btrfs-balance.sh[25953]: Filesystem           Size  Used Avail Use% Mounted on
    Feb 16 12:04:30 thinkpad btrfs-balance.sh[25953]: /dev/mapper/cr_root  347G   65G     0 100% /
    Feb 16 12:04:30 thinkpad btrfs-balance.sh[25953]: Done, had to relocate 0 out of 75 chunks
    Feb 16 12:04:33 thinkpad btrfs-balance.sh[25953]: Dumping filters: flags 0x1, state 0x0, force is off
    Feb 16 12:04:33 thinkpad btrfs-balance.sh[25953]:   DATA (flags 0x2): balancing, usage=5
    Feb 16 12:04:33 thinkpad btrfs-balance.sh[25953]: Done, had to relocate 9 out of 75 chunks
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]: Dumping filters: flags 0x1, state 0x0, force is off
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]:   DATA (flags 0x2): balancing, usage=10
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]: Done, had to relocate 1 out of 66 chunks
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]: Done, had to relocate 0 out of 65 chunks
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]: Dumping filters: flags 0x6, state 0x0, force is off
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]:   METADATA (flags 0x2): balancing, usage=3
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]:   SYSTEM (flags 0x2): balancing, usage=3
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]: Done, had to relocate 1 out of 65 chunks
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]: After balance of /
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]: Data, single: total=63.00GiB, used=58.86GiB
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]: System, single: total=32.00MiB, used=16.00KiB
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]: Metadata, single: total=1.00GiB, used=910.09MiB
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]: GlobalReserve, single: total=110.70MiB, used=0.00B
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]: Filesystem           Size  Used Avail Use% Mounted on
    Feb 16 12:04:34 thinkpad btrfs-balance.sh[25953]: /dev/mapper/cr_root  347G   65G     0 100% /
    Feb 16 12:04:34 thinkpad systemd[1]: btrfs-balance.service: Succeeded.
    I deleted some personal files just to make up enough space for the update. The result was it just runs out of space sooner (at 60 gigs instead of 74).

    After some more snapshot deletions and collecting a few megabytes along the way, the update finally managed to finish. Now running 5.5.2 and hoping this goes away.

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •