OpenSuse42.3: After repairing a btrfs drive and rebooting, no space left on device is detected , even if df -hT shows that only 27G used out of 41G: “Size 41G Used 27G Avail 0 Use 100%”
https://imagizer.imageshack.com/img923/6969/4oAn9T.jpg
The result of the btrfsck /dev/sda6 --repair is shown in this picture
https://imagizer.imageshack.com/img923/534/bXlP16.jpg
Please help !
Update the kernel. This bug was fixed in upstream 5.4.18 and 5.5.2, and didn’t exist in 5.3 and older.
You should not use 42.3 anymore, it’s way past it’s EOL. And you should not use df on btrfs
For now:
run
sudo snapper list
It will show a list of snapshots, each with an index no. To remove snapshot ## to ###, do
sudo snapper delete ##-###
and reboot. It may take some minutes to actually free the allocated space.
Please, if you read the post, you’ll see that his / filesystem is full. So updating to a newer kernel is not an option. Apart from that, the OP is on 42.3
I was planning to upgrade to latest Suse version. But it was running so well so I’ve always postponed it by the principle “Never change a running system” …
Thanks for the tip with df on btrfs. Should I try
btrfs filesystem df /
?
I will try it later in the evening to clean up some snapshots.
One problem I have although… after getting that “No space left on device” I’ve cleaned up my /tmp folder. I thought that about 3-4 GB of space were freed.
With all this, after reboot I had the same “No space left on device”. Can you explain this somehow?
I cannot currently access the laptop having this problem so I will have to wait until evening
Thanks !
Running snapper and removing old snapshots should help. If that fixes the case you’d need to review the policy on keeping old snapshots.
Apart from snapshots, how did you cleaned up /tmp? Are you sure the files are really removed from disk?
The root partition is quite small. I found this: https://github.com/kdave/btrfs-progs/blob/d39ab9fafeb425d28fcaac645f7d663555fa654d/Documentation/mkfs.btrfs.asciidoc#known-issues, see if that does apply for you.
It’s not clear what’s emitting “no space left on device”. The specifics are really useful towards resolution.
# inspect the output to understand what's taking up space
btrfs filesystem usage /
# free up empty blocks, this will not fail
btrfs balance start -dusage=0 -musage=0 /
# if the above command deallocate empty blocks, you might try increasing the usage factor (5 -> 10 -> 20 -> etc...)
btrfs balance start -dusage=5 -musage=5 /
After you get this issue fixed, I’d advise to upgrade the distro and merge /home with the / partition. On current tumbleweed /home is a subvolume on the btrfs partition. You wouldn’t miss xfs. This way you get a bigger partition with enough room for snapshots.
Good luck!
It looks very strange, I’have no idea how to go on . It says that 13GB are free. I do not understand …
after deleting 2 snapshots and booting again, nothing changed in the output of the command
btrfs filesystem usage /
I think it says “no space left on device” because there’s about 1MiB unused on the partition. The 13.77GiB free is distributed inside allocated chunks. Maybe to mount the fs it needs some unallocated space on device?
Free more space inside allocated blocks with the snapper command, then run balance as I referred to before, so completely empty blocks will be unallocated, and with 5, 10, 20, etc., usage factors data will be moved around in order to increase density.
Please show these two things:
sudo snapper list
and
sudo ls -l /.snapshots
You can paste them both on paste.opensuse.org
Thank you so much, the rebalancing after deleting the snapshots worked. What is your advice for upgrading to the newest suse ? Online upgrade procedure or upgrade using DVD image ?
I’ve never used Leap but I know it’s advised to do all upgrades to the next version, one at a time. I’d backup any important data, burn the Tumbleweed DVD image to a thumb drive, boot from the image, remove all partitions and re-create a single btrfs partition (besides ESP and maybe swap). The installer should create default subvolumes including home. Then restore /home.
Personally I keep two partitions reserved for the OS. Then always do a fresh install rotating the the partitions each version. I keep a short list of the programs I need and simply install them as part of the process. This way I can test the newer OS before committing the mounting of my home on the new OS and drop back to the previous if need be. Note I install the new OS without mounting home just use the default home which is created with the OS on root. This can be removed latter if space is tight on root
Using the DVD image will have you download many packages you never install. Thus I always use https://download.opensuse.org/tumbleweed/iso/openSUSE-Tumbleweed-NET-x86_64-Current.iso. Don’t run out of space:
Thank you ! I will still have a question … In case that something goes wrong during the installation, it should still be possible to switch back to a previous snapshot ?
Assuming zypper dup upgrade path yes and assuming you have sufficient disk space too. Not sure how snapper works from a DVD/USB upgrade. I guess you could manually take a snapshot before DVD/USB upgrade???
Assuming a brand new installation, well… no, it wouldn’t. But it is very unlikely that something would prevent you from having a working installation. Backup first it’s a good measure though.