No disk space to boot, despite 81% disk use

English Install/Boot/Login
1

Hi all,

I have installed Tumbleweed two months ago and I have been struggling with the free disk space for some weeks.

Problem: unable to boot anymore. Latest boot log: https://pastebin.com/RppEv5YH, I had also submit a bugreport which led to the same conclusion: https://bugzilla.novell.com/show_bug.cgi?id=1096831

What I do not understand, is that df still shows me sufficient disk space:


Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        3.9G     0  3.9G   0% /dev
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           3.9G   18M  3.9G   1% /run
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/sdb7        10G  7.9G  1.9G  81% /
/dev/sdb7        10G  7.9G  1.9G  81% /root
/dev/sdb7        10G  7.9G  1.9G  81% /boot/grub2/i386-pc
/dev/sdb7        10G  7.9G  1.9G  81% /var
/dev/sdb7        10G  7.9G  1.9G  81% /tmp
/dev/sdb7        10G  7.9G  1.9G  81% /usr/local
/dev/sdb7        10G  7.9G  1.9G  81% /boot/grub2/x86_64-efi
/dev/sdb7        10G  7.9G  1.9G  81% /opt
/dev/sdb7        10G  7.9G  1.9G  81% /srv
/dev/sdb6       500M  5.0M  495M   1% /boot/efi
/dev/sdb9        49G   14G   36G  28% /home

My root filesystem is in BTRFS and 10GB big. I have been using (open)SUSE for many years and never fell short on disk space. How can it be that the system claims the disk is full and yet I do not see what I filling up the 1.9GB that are still available according to df?

Cheers,
Quentin

2

Hi
Having 10GB for / on a Tumbleweed system due to the way it upgrades (not update!) to a new release is the likely issue and the way btrfs will allocate disk space.

Can you boot from a Tumbleweed live system and look at running some btrfs maintenance routines?

Else can you boot the system in rescue mode at grub, select advanced and recovery mode?

3

“For many years” you likely would have had / using EXT3 or ReiserFS at first, depending on when first was, later EXT4, depending on how you upgraded and whether you formatted / at upgrade time. Now with BTRFS the default installation space requirement has skyrocketed, making 10GB for / crowded if not unusable, unless snapshotting is curtailed. If you want to keep 10GB usable you’ll need to make adjustments to the BTRFS maintenance/snapshot system, or reinstall fresh using EXT4. Mere updates via zypper dup need not be a problem, because atomic updates are not mandatory, controlled by commit.downloadMode = in /etc/zypp/zypp.conf or cmdline option.

4

Even with ext4 for / here, I have a 24G partition, 9.5GB free, and the recent huge gcc8 upgrade temporarily used 3.5GB of that. I did not expect to use 75% of a 24GB root partition. Next time I will allocate 31.9GB for a bit more breathing room, things only seem to get bigger …

5

Quoting this log:

Jun 26 23:21:24 linux-q7ny systemd[1]: Startup finished in 16.893s (firmware) + 3.817s (loader) + 6.939s (kernel) + 2.690s (initrd) + 1.982s (userspace) = 32.324s.
-- Subject: System start-up is now complete
6

With btrfs, df and du no longer work so well, you will only see the full picture by using:

  btrfs filesystem df
  btrfs filesystem du

Maybe df and du should be made to error for btrfs, or to report something different, but I guess that would break a lot of scripts.

Btrfs is not like the old file systems, and even more so when used for the root file system for rollbacks. Here are the notes I made when seeking find out more about btrfs:

https://forums.opensuse.org/showthread.php/521277-LEAP-42-2-btrfs-root-filesystem-subvolume-structure

And specifically to your situation, here is a link to some SUSE SLES documentation on Btrfs no space left on device:
https://www.suse.com/documentation/sles11/stor_admin/data/trbl_btrfs_volfull.html

I think it would also apply to tumbleweed.