System down!

English Install/Boot/Login
1

openSUSE 15.3
KDE desktop
Intel 8 core processor
64 G ram
3 drives 1T each

Yesterday, I went to my computer and the fans were running fast. The desktop widget showed all 8 cores were maxed out for a while but had just settled down. Konsole would not start, nor would Libreoffice.

I tried to reboot but could not log in to KDE, so, at the log in screen I pressed Ctrl-Alt-2 and could log in to the CLI. Using df I found that the root partition was full. The system log also showed a very large block of garbage where udev had been having trouble mounting one of the drives. So I have two problems: a full disk on root and a bad drive on the system.

The root partition is 100G But I have 3 versions of the kernel and I’m using a nVidia card so there are 3 copies of kernel source on that drive along with a large amount of other stuff. 100 G is NOT enough!

My questions are:
Can I delete the contents of /var/log to give me some space to log in using KDE?

If I can get in, can I use gparted to resize the partitions, taking space from /home and move it to /?

Did the full disk cause the processor to max out? Or is there yet another problem to look for?

Bart

2

I suggest you delete only the files with filename ending in “.xz”. Those are compressed logfiles kept in case you ever need them. Best to keep the current log files.

As to what happened – I don’t know. Probably a program ran wild. Maybe it was javascript in an advertisement pulled in by your browser. But, of course, I am only guessing.

3

I think you need to investigate just what is occupying that 100G - I have a Leap 15.3 system, also with the proprietary nVidia drivers and 3 versions of the kernel, and that’s no where near filling the 32G partition I allocated it (74% free space).

Can I delete the contents of /var/log to give me some space to log in using KDE?

If you print quite a lot you could probably also gain some free space by deleting the files in “/var/spool/cups/”

4

Well, I gained about 700k

I looked for one huge file in root and couldn’t find anything that stood out.

There’s got to be more fluff that can go.

I’m still a little concerned about resizing the partitions. Will gparted do it without problems? I really don’t want to do a fresh install.

Bart

5

In my temp directory, there are about 15M of files named tmpaddon-<some number> and preview<some number>.pdf

Can these files go?

Bart

6

[QUOTE=montana_suse_user;3082586]Can these files go?/QUOTE]

You would know that better than anyone else. I would guess that they can go.

7

[quote="“nrickert,post:6,topic:148511”]

:shame: I have absolutely no idea what they are! Years ago, when I ran Windows, everything in mp could go without problems. My understanding is that is not the case with linux.

8

Some reading for you…
https://en.opensuse.org/SDB:Cleanup_system
https://forums.opensuse.org/showthread.php/535050-tmp-amp-var-tmp-safe-to-remove

9

Hi
What filesystem in use on /?

Big journal?


journalctl --disk-usage

Might have to cycle through some directories with for example;


du -sh /opt
du -sh /var
etc....
10

I use ext4 on all disks except the efi

Big journal?

journalctl --disk-usage

Archived and active journals take up 16.0M in the file system

Might have to cycle through some directories with for example;

du -sh /opt
du -sh /var
etc…

du -sh /
du: cannot access '/proc/7078/task/7078/fd/3' : No such file or directory
du: cannot access '/proc/7078/task/7078/fdinfo/3' : No such file or directory
du: cannot access '/proc/7078/fd/4' : No such file or directory
du: cannot access '/proc/7078/fdinfo/4' : No such file or directory
338G   /

du -sh /home
178G  /home

Sure wish I could copy and paste but have to use two machines! :frowning:

11

Hi
What about sub directories and usage?

12

I have never backed up /tmp on 1000’s of SLES linux boxes and OpenSUSE 15.x machines and restored them with no problems.

I have done a rm -rf /tmp/* before a reboot with no issues.

If you have a bootable Linux USB or CD - you can safely shrink your home partition and add it to the root partition with gparted.

You cannot resize any mounted partition so umount the partitions or tell gparted to unmount them before resizing.

13

Hi
A root file system @338GB? Must be some cruft somewhere I have all mine split out on to xfs partitions (specifically for docker and libvirt).

14

Not just cruft, but some runaway process(es) created trash files. Use ncdu if installed to navigate by directory size to locate and inspect the problem source.

Anything in /tmp/ that isn’t current boot timestamped can be deleted. Nothing in /tmp/ should be surviving reboot if /tmp is on tmpfs.

What’s in /var/log/journal/*? If it’s an old installation upgraded to 15.3 there could be quite a bit there, or it might not even exist.

Is there anything in /var/lib/systemd/coredump/? It’s all expendable.

Does /lib/modules/ have only the installed 3 versions, or some older cruft too?

15

Like usual, it was a stupid mistake. I am in the process of setting up an automated backup system for all the computers on my network, using cron. I had the backup program set up on this machine BUT I had not finished the process far enough to have it map the NAS drive. Therefore, 85.3 G went into /mnt/NAS-Backup.

My thanks to mrmazda for the suggestion of a most useful tool ncdu!

I booted a live copy of Kali and used gparted to resize the partitions on the root drive and the system came right up. My next step is to clean out /mnt/NAS-Backup. That’ll be easy!

Again, thanks to all who were right there with help! Don’t know what this old-timer would do without you.

Bart