Root Partition Full and out of ideas

Hi all. System is Leap 15.1 with updates. The small ssd that houses OpenSuse is full and I am struggling to create space. I have googled for solutions on this and my attempts have failed. I have looked at removing logs, kernels and updating /tmp. I am a fan of syncthing, but synchronizing external folders requires disk space it seems. Snapper is also a big user. I have tried to reduce its footprint but without success. Do you have any feedback, ideas that may help?

df -h

Filesystem Size Used Avail Use% Mounted on
devtmpfs 5.9G 0 5.9G 0% /dev
tmpfs 5.9G 47M 5.9G 1% /dev/shm
tmpfs 5.9G 10M 5.9G 1% /run
tmpfs 5.9G 0 5.9G 0% /sys/fs/cgroup
/dev/sdd1 32G 32G 129M 100% /
/dev/sdd1 32G 32G 129M 100% /boot/grub2/x86_64-efi
/dev/sdd2 80G 78G 2.0G 98% /home
/dev/sdd1 32G 32G 129M 100% /root
/dev/sdd1 32G 32G 129M 100% /opt
/dev/sdd1 32G 32G 129M 100% /usr/local
/dev/sdd1 32G 32G 129M 100% /.snapshots
/dev/sdd1 32G 32G 129M 100% /srv
/dev/sdd1 32G 32G 129M 100% /var
/dev/sdd1 32G 32G 129M 100% /boot/grub2/i386-pc
/dev/sdd1 32G 32G 129M 100% /tmp
/dev/sdc4 259G 30G 230G 12% /mnt/sdc4
/dev/loop3 56M 56M 0 100% /snap/core18/1932
/dev/loop2 7.9M 7.9M 0 100% /snap/syncthing-arubislander/7
/dev/loop1 32M 32M 0 100% /snap/snapd/10492
/dev/loop4 32M 32M 0 100% /snap/snapd/10707
/dev/loop0 56M 56M 0 100% /snap/core18/1944
192.xxx.xxx.xxx:/volume1/Archive 1.8T 1.2T 610G 67% /mnt/Archive
tmpfs 1.2G 32K 1.2G 1% /run/user/1000

Leap 15.1 is near end of life.

I suggest you install Leap 15.2. You can use the same partitions, so you will keep “/home”. I notice, however, that “/home” is almost full, so you might want to do some cleaning up there.

My advice: do not use “btrfs” for your root file system. A 32G partition is just too small for using “btrfs” effectively (in my opinion). So use “ext4” or “xfs” or something else suitable. I’ve been using “ext4” and it has been working fine for me.

+1
I see no use for BTRFS on a 128GB disk. Doing well with a 27GB / root on EXT4 here.

In fact, if the disk is so small, I would recommend a single ext4 partition – for everything …

  • This approach ensures that, /var/ and /tmp/ always have enough free space – I know, it’s brutal but, effective …

I am confused by the listing of Folders. The root partition is listed many times with its respective folder. Is there another way to show data and free space?

A “btrfs” partition is divided into subvolumes. And you are seeing a line for each subvolume.

Ok, thanks for pointing that out. Should one dive into these and look for old obsolete files? To some extent I have already done this by rationalizing /temp and trying to reduce snapshots (unsuccessfully).

NB I am a long - long user of OpenSuse having started at Release 6.3 I believe, but have run foul of changes introduced last year requiring me to register again.

One place to look for old obsolete files, is “/var/log”. Any file whose name ends in “.xz” is an archive of older logs. Removing those files won’t hurt anything.

Basically sit back and let the maintenance scripts do their job. Currently I am using them daily:

**erlangen:~ #** grep PERIOD /etc/sysconfig/btrfsmaintenance             
BTRFS_DEFRAG_**PERIOD**="daily" 
BTRFS_BALANCE_**PERIOD**="daily" 
BTRFS_SCRUB_**PERIOD**="daily" 
BTRFS_TRIM_**PERIOD**="daily" 
**erlangen:~ #**

Limit the number of snapshots:

**erlangen:~ #** grep NUMBER /etc/snapper/configs/root  
**NUMBER**_CLEANUP="yes" 
**NUMBER**_MIN_AGE="1800" 
**NUMBER**_LIMIT="20" 
**NUMBER**_LIMIT_IMPORTANT="10" 
**erlangen:~ #**

Snapshot #1 below is old and thus very big. You may want to rollback and delete it:

**erlangen:~ #** snapper list 
   # | Type   | Pre # | Date                     | User | Cleanup | Description           | Userdata      
-----+--------+-------+--------------------------+------+---------+-----------------------+-------------- 
  0  | single |       |                          | root |         | current               |               
  1* | single |       | Sun May 17 12:35:27 2020 | root |         | first root filesystem |               
496  | pre    |       | Tue Jan  5 15:02:33 2021 | root | number  | zypp(zypper)          | important=yes 
497  | post   |   496 | Tue Jan  5 15:03:03 2021 | root | number  |                       | important=yes 
504  | pre    |       | Sat Jan  9 18:09:27 2021 | root | number  | zypp(zypper)          | important=no  
505  | post   |   504 | Sat Jan  9 18:10:51 2021 | root | number  |                       | important=no  
506  | pre    |       | Mon Jan 11 13:31:04 2021 | root | number  | zypp(zypper)          | important=no  
507  | post   |   506 | Mon Jan 11 13:37:16 2021 | root | number  |                       | important=no  
508  | pre    |       | Tue Jan 12 12:28:40 2021 | root | number  | zypp(zypper)          | important=yes 
509  | post   |   508 | Tue Jan 12 12:35:01 2021 | root | number  |                       | important=yes 
510  | pre    |       | Tue Jan 12 12:37:47 2021 | root | number  | zypp(zypper)          | important=yes 
511  | post   |   510 | Tue Jan 12 12:38:16 2021 | root | number  |                       | important=yes 
512  | pre    |       | Wed Jan 13 17:51:25 2021 | root | number  | zypp(zypper)          | important=no  
513  | post   |   512 | Wed Jan 13 17:52:36 2021 | root | number  |                       | important=no  
514  | pre    |       | Fri Jan 15 05:18:12 2021 | root | number  | zypp(zypper)          | important=no  
515  | post   |   514 | Fri Jan 15 05:20:22 2021 | root | number  |                       | important=no  
516  | pre    |       | Sat Jan 16 15:37:16 2021 | root | number  | zypp(zypper)          | important=yes 
517  | post   |   516 | Sat Jan 16 15:40:42 2021 | root | number  |                       | important=yes 
518  | pre    |       | Sat Jan 16 16:29:18 2021 | root | number  | zypp(zypper)          | important=yes 
519  | post   |   518 | Sat Jan 16 16:29:52 2021 | root | number  |                       | important=yes 
520  | pre    |       | Tue Jan 19 05:59:18 2021 | root | number  | zypp(zypper)          | important=no  
521  | post   |   520 | Tue Jan 19 06:05:11 2021 | root | number  |                       | important=no  
522  | pre    |       | Wed Jan 20 06:39:33 2021 | root | number  | zypp(zypper)          | important=no  
523  | post   |   522 | Wed Jan 20 06:39:36 2021 | root | number  |                       | important=no  
**erlangen:~ #**

Check for the big chunks of data and shorten if appropriate:

**erlangen:~ #** du -xhd1 -t1M / /var 
20M     /etc 
139M    /boot 
11G     /usr 
359M    /lib 
7.3M    /lib64 
9.3M    /sbin 
11G     / 
625M    /var/log 
142M    /var/lib 
718M    /var/cache 
2.3M    /var/tmp 
330M    /var/adm 
1.8G    /var 
**erlangen:~ #**

That’s all.

Note: You need unallocated space notwithstanding contradictory claims. On rare occasions btrfs will claim one or more new chunks of 1.0 GiB size.

**erlangen:~ #** btrfs filesystem usage -T /            
Overall: 
    Device size:                  51.69GiB 
    Device allocated:             31.04GiB 
**    Device unallocated:           20.65GiB **
    Device missing:                  0.00B 
    Used:                         21.55GiB 
    Free (estimated):             28.54GiB      (min: 28.54GiB) 
    Data ratio:                       1.00 
    Metadata ratio:                   1.00 
    Global reserve:               70.58MiB      (used: 0.00B) 
    Multiple profiles:                  no 

                  Data     Metadata System               
Id Path           single   single   single   Unallocated 
-- -------------- -------- -------- -------- ----------- 
 1 /dev/nvme0n1p3 28.01GiB  3.00GiB 32.00MiB    20.65GiB 
-- -------------- -------- -------- -------- ----------- 
   Total          28.01GiB  3.00GiB 32.00MiB    20.65GiB 
   Used           20.11GiB  1.44GiB 16.00KiB             
**erlangen:~ #**

I was getting warnings about limited free disk space from (df) disk free (df said 93g used out of 100g) in root (/) but (du) disk usage said only 20% disk usage. This was after I had just increased root (/) partition from 40G to 100g based upon previous df warnings.

Further investigation showed large image files that I had deleted were still on the disk in /var/.Trash-0. There were two 35g image files I had been experimenting with on virtual machines and later deleted. Their filenames included word (deleted) but they were still occupying disk space per df command. Furthermore I found more deleted files still on disk in /opt/.Trash-0.

I manually deleted all these files as root and my df and du lined up at about only 20% disk usage.

I think many people have enlarged root partitions and maybe even changed out disk drives based upon erroneous data from df warnings not knowing that supposedly deleted files were still on their disks.
Check for deleted files still occupying disk space in .Trash-0 in root directories such as /var and /opt before you go and resize root.

btrfs file system was used for root at that time. I have since gone back to ext4.

Snapshot #1 below is old and thus very big. You may want to rollback and delete it:

**erlangen:~ #** snapper list 
   # | Type   | Pre # | Date                     | User | Cleanup | Description           | Userdata      
-----+--------+-------+--------------------------+------+---------+-----------------------+-------------- 
  0  | single |       |                          | root |         | current               |               
  1* | single |       | Sun May 17 12:35:27 2020 | root |         | first root filesystem |               
496  | pre    |       | Tue Jan  5 15:02:33 2021 | root | number  | zypp(zypper)          | important=yes 
497  | post   |   496 | Tue Jan  5 15:03:03 2021 | root | number  |                       | important=yes 
504  | pre    |       | Sat Jan  9 18:09:27 2021 | root | number  | zypp(zypper)          | important=no  
505  | post   |   504 | Sat Jan  9 18:10:51 2021 | root | number  |                       | important=no  
506  | pre    |       | Mon Jan 11 13:31:04 2021 | root | number  | zypp(zypper)          | important=no  
507  | post   |   506 | Mon Jan 11 13:37:16 2021 | root | number  |                       | important=no  
508  | pre    |       | Tue Jan 12 12:28:40 2021 | root | number  | zypp(zypper)          | important=yes 
509  | post   |   508 | Tue Jan 12 12:35:01 2021 | root | number  |                       | important=yes 
510  | pre    |       | Tue Jan 12 12:37:47 2021 | root | number  | zypp(zypper)          | important=yes 
511  | post   |   510 | Tue Jan 12 12:38:16 2021 | root | number  |                       | important=yes 
512  | pre    |       | Wed Jan 13 17:51:25 2021 | root | number  | zypp(zypper)          | important=no  
513  | post   |   512 | Wed Jan 13 17:52:36 2021 | root | number  |                       | important=no  
514  | pre    |       | Fri Jan 15 05:18:12 2021 | root | number  | zypp(zypper)          | important=no  
515  | post   |   514 | Fri Jan 15 05:20:22 2021 | root | number  |                       | important=no  
516  | pre    |       | Sat Jan 16 15:37:16 2021 | root | number  | zypp(zypper)          | important=yes 
517  | post   |   516 | Sat Jan 16 15:40:42 2021 | root | number  |                       | important=yes 
518  | pre    |       | Sat Jan 16 16:29:18 2021 | root | number  | zypp(zypper)          | important=yes 
519  | post   |   518 | Sat Jan 16 16:29:52 2021 | root | number  |                       | important=yes 
520  | pre    |       | Tue Jan 19 05:59:18 2021 | root | number  | zypp(zypper)          | important=no  
521  | post   |   520 | Tue Jan 19 06:05:11 2021 | root | number  |                       | important=no  
522  | pre    |       | Wed Jan 20 06:39:33 2021 | root | number  | zypp(zypper)          | important=no  
523  | post   |   522 | Wed Jan 20 06:39:36 2021 | root | number  |                       | important=no  
**erlangen:~ #**

How to go about this?

More info

# btrfs filesystem usage -T /
Overall:
    Device size:                  32.00GiB
    Device allocated:             32.00GiB
    Device unallocated:            1.00MiB
    Device missing:                  0.00B
    Used:                         31.76GiB
    Free (estimated):            124.66MiB      (min: 124.66MiB)
    Data ratio:                       1.00
    Metadata ratio:                   1.00
    Global reserve:               80.95MiB      (used: 0.00B)

             Data     Metadata System              
Id Path      single   single   single   Unallocated
-- --------- -------- -------- -------- -----------
 1 /dev/sdd1 30.72GiB  1.25GiB 32.00MiB     1.00MiB
-- --------- -------- -------- -------- -----------
   Total     30.72GiB  1.25GiB 32.00MiB     1.00MiB
   Used      30.60GiB  1.17GiB 16.00KiB

#grep NUMBER /etc/snapper/configs/root
NUMBER_CLEANUP="yes"
NUMBER_MIN_AGE="1800"
NUMBER_LIMIT="2-6"
NUMBER_LIMIT_IMPORTANT="4"

Hope this may be useful

# du -xhd1 -t1M / /var
27M     /etc
84M     /boot
12G     /usr
1.1G    /lib
5.2M    /bin
11M     /lib64
13M     /sbin
14G     /
623M    /var/log
419M    /var/lib
580M    /var/cache
440M    /var/adm
125M    /var/spool
2.2G    /var

You perfectly ran out of unallocated space. Before considering rollback you may show space allocated to current snapshot:

**erlangen:~ #** du -xhd1 -t1M / /var             
20M     /etc 
139M    /boot 
11G     /usr 
359M    /lib 
7.3M    /lib64 
9.3M    /sbin 
11G     / 
647M    /var/log 
93M     /var/lib 
718M    /var/cache 
2.3M    /var/tmp 
330M    /var/adm 
1.8G    /var 
**erlangen:~ #**

Depending on the results you may consider deleting unneeded files first.

Interrupted my typing and thus missed your comment. There isn’t much to delete. Go for the snapshots first. Show the output:

**erlangen:~ #** snapper list 
   # | Type   | Pre # | Date                     | User | Cleanup | Description           | Userdata      
-----+--------+-------+--------------------------+------+---------+-----------------------+-------------- 
  0  | single |       |                          | root |         | current               |               
  1* | single |       | Sun May 17 12:35:27 2020 | root |         | first root filesystem |               
496  | pre    |       | Tue Jan  5 15:02:33 2021 | root | number  | zypp(zypper)          | important=yes 
497  | post   |   496 | Tue Jan  5 15:03:03 2021 | root | number  |                       | important=yes 
506  | pre    |       | Mon Jan 11 13:31:04 2021 | root | number  | zypp(zypper)          | important=no  
507  | post   |   506 | Mon Jan 11 13:37:16 2021 | root | number  |                       | important=no  
508  | pre    |       | Tue Jan 12 12:28:40 2021 | root | number  | zypp(zypper)          | important=yes 
509  | post   |   508 | Tue Jan 12 12:35:01 2021 | root | number  |                       | important=yes 
510  | pre    |       | Tue Jan 12 12:37:47 2021 | root | number  | zypp(zypper)          | important=yes 
511  | post   |   510 | Tue Jan 12 12:38:16 2021 | root | number  |                       | important=yes 
512  | pre    |       | Wed Jan 13 17:51:25 2021 | root | number  | zypp(zypper)          | important=no  
513  | post   |   512 | Wed Jan 13 17:52:36 2021 | root | number  |                       | important=no  
514  | pre    |       | Fri Jan 15 05:18:12 2021 | root | number  | zypp(zypper)          | important=no  
515  | post   |   514 | Fri Jan 15 05:20:22 2021 | root | number  |                       | important=no  
516  | pre    |       | Sat Jan 16 15:37:16 2021 | root | number  | zypp(zypper)          | important=yes 
517  | post   |   516 | Sat Jan 16 15:40:42 2021 | root | number  |                       | important=yes 
518  | pre    |       | Sat Jan 16 16:29:18 2021 | root | number  | zypp(zypper)          | important=yes 
519  | post   |   518 | Sat Jan 16 16:29:52 2021 | root | number  |                       | important=yes 
520  | pre    |       | Tue Jan 19 05:59:18 2021 | root | number  | zypp(zypper)          | important=no  
521  | post   |   520 | Tue Jan 19 06:05:11 2021 | root | number  |                       | important=no  
522  | pre    |       | Wed Jan 20 06:39:33 2021 | root | number  | zypp(zypper)          | important=no  
523  | post   |   522 | Wed Jan 20 06:39:36 2021 | root | number  |                       | important=no  
524  | pre    |       | Thu Jan 21 12:27:08 2021 | root | number  | zypp(zypper)          | important=no  
525  | post   |   524 | Thu Jan 21 12:33:22 2021 | root | number  |                       | important=no  
526  | pre    |       | Thu Jan 21 15:14:46 2021 | root | number  | zypp(zypper)          | important=no  
527  | post   |   526 | Thu Jan 21 15:15:41 2021 | root | number  |                       | important=no  
528  | pre    |       | Fri Jan 22 06:21:27 2021 | root | number  | zypp(zypper)          | important=no  
529  | post   |   528 | Fri Jan 22 06:21:32 2021 | root | number  |                       | important=no  
**erlangen:~ #**

Ok, here is the output.

# snapper list
    # | Type   | Pre # | Date                          | User | Cleanup | Description           | Userdata     
------+--------+-------+-------------------------------+------+---------+-----------------------+--------------
   0  | single |       |                               | root |         | current               |              
   1* | single |       | Sun 29 Jul 2018 08:00:23 ACST | root |         | first root filesystem |              
1148  | pre    |       | Wed 30 Dec 2020 08:16:28 ACDT | root | number  | zypp(packagekitd)     | important=yes
1149  | post   |  1148 | Wed 30 Dec 2020 08:28:55 ACDT | root | number  |                       | important=yes
1159  | pre    |       | Sun 17 Jan 2021 21:00:21 ACDT | root | number  | zypp(packagekitd)     | important=yes
1160  | pre    |       | Mon 18 Jan 2021 20:05:47 ACDT | root | number  | zypp(packagekitd)     | important=yes
1161  | post   |  1160 | Mon 18 Jan 2021 20:15:19 ACDT | root | number  |                       | important=yes
1164  | pre    |       | Wed 20 Jan 2021 12:35:12 ACDT | root | number  | yast repositories     |              
1165  | post   |  1164 | Wed 20 Jan 2021 12:37:09 ACDT | root | number  |                       | 

Re cleaning up files and directories. I found old files in /var/log/messages. But it turns out to be a read only filesystem. Any ideas on how to address this? It could lead to recovering a fair amount of disk space.

Try to boot into default and run the command ‘snapper rollback’.

It ain’t “read-only” – the system as such can happily write to it …

  • You seem to be unaware that, UNIX® and Linux and macOS are all multi-user operating systems …
  • And, the system as such, runs under the supervision of “System Users” – such as “root” …

 > ls -l --directory /var
drwxr-xr-x 13 root root 4096 10. Nov 15:59 /var
 > 
 > ls -l --directory /var/log
drwxr-xr-x 23 root root 4096 22. Jan 09:16 /var/log
 > 
 > ls -l --human-readable /var/log/messages*
-rw-r----- 1 root root  74K 22. Jan 11:26 /var/log/messages
-rw-r----- 1 root root 287K  5. Jan 08:24 /var/log/messages-20210105.xz
-rw-r----- 1 root root 310K 14. Jan 09:08 /var/log/messages-20210114.xz
-rw-r----- 1 root root 291K 22. Jan 09:16 /var/log/messages-20210122.xz
 > 

Therefore, if you login as the user “root” you can remove the older files – assuming that, “logrotate” hasn’t already done the job for you – take a look at the configuration (man page) …

  • BTW, the “logrotate
    ” package should have been installed on your system –

 > rpm --query --whatrecommends logrotate
snapper-0.8.9-lp152.2.2.x86_64
rsync-3.1.3-lp152.5.7.x86_64
patterns-base-enhanced_base-20171206-lp152.34.2.x86_64
quagga-1.1.1-lp152.4.6.x86_64
samba-4.11.14+git.202.344b137b75d-lp152.3.16.1.x86_64
samba-winbind-4.11.14+git.202.344b137b75d-lp152.3.16.1.x86_64
dracut-049.1+suse.171.g65b2addf-lp152.2.12.1.x86_64
aaa_base-84.87+git20180409.04c9dae-lp152.14.7.1.x86_64
mariadb-10.4.17-lp152.2.8.1.x86_64
libzypp-17.25.5-lp152.2.16.1.x86_64
zypper-1.14.41-lp152.2.12.1.x86_64
 > 

I am root when I try to remove files in /var/log/messages. I am faced with a read-only file system.

# rpm --query --whatrecommends logrotate
snapper-0.8.3-lp151.1.1.x86_64
patterns-base-enhanced_base-20171206-lp151.29.1.x86_64
dnf-conf-4.2.6-lp151.1.1.noarch
rsync-3.1.3-lp151.4.3.1.x86_64
dracut-044.2-lp151.2.27.1.x86_64
libzypp-17.25.1-lp151.2.25.1.x86_64
zypper-1.14.40-lp151.2.20.1.x86_64
mariadb-10.2.36-lp151.2.18.1.x86_64
samba-winbind-4.9.5+git.383.7b7f8f14df8-lp151.2.36.1.x86_64
samba-4.9.5+git.383.7b7f8f14df8-lp151.2.36.1.x86_64
aaa_base-84.87+git20180409.04c9dae-lp151.5.33.1.x86_64