Console login only AGAIN and rootfs full

sebyan · August 30, 2012, 6:04pm

Hi All,

My 12.1 machine has been running smoothly for one month since the last problem (http://forums.opensuse.org/forums/english/get-technical-help-here/install-boot-login/477083-console-login-only-startx-failed.html).

I have updated the Latex package TexLive this morning. Something started going wrong after that including: 1) YaST failed to start; 2) brower Opera failed for some websites. To kill these bugs I restarted the machine and couldn’t go back to KDE or even IceWM.

export WINDOWMANAGER=icewm;startx

returns error msg:

*The XKEYBOARD key map compiler (xkbcomp) reports:
> Error: Cannot close “/var/lib/xkb/compiled/server-0.xkm” properly (not enough space?)
> Output file “/var/lib/xkb/compiled/server-0.xkm” removed

Keyboard initialization failed. This could be a missing or incorrect setup of xkeyboard-config.

Fatal server error:
Failed to activate core devices.
*
I have done a space checking which shows me

*Filesystem Size Used Avail Use% Mounted on
rootfs 20G 20G 0 100% /
/dev/sdb2 20G 20G 0 100% /
*
Also, I have failed to shutdown the machine as it halts at:

*Stopping LSB configure the remote-fs depending network interfaces …
*
Anyone can tell me what’s the problem?

Many thanks!
Seb

ab · August 30, 2012, 6:16pm

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

If you are out of space on your drive then things will not work. Clear
out space, perhaps starting in /tmp or deleting log files from /var/log

Good luck.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQIcBAEBAgAGBQJQP5HzAAoJEF+XTK08PnB5ai0QAJsilzVMPWHQbahSGqId33+7
KPtG6XUhAyxLuGxz4NJY87nrZMO8awPoJDnRupezILZoqqsGutdOzR5F4lBwBwhs
zcULMOIWSQ1oMZB7Ap4w825BU0T3Qyw9L2oXnclShGRLzMIP3uyZj7zulG8ElzyD
iSG7dUptEciAVonbZkyYK5Kbur3ttveJMNu5Y9LJhm3kKZ9upu1FzHSSEEBPyZzP
OHfl4KfFdYTkLbeF8cb4Zi1w3bfxRb6PETzr4ug2BWJPdvStgRZWjHNA9JU892wQ
BEw9h80i9OboOYheBJoxxKQmuJDDGY+/XgexAWbs89dxwA2somq5+2kCxQrEixro
LL0tvXi4avICrIPT+xAZSLq3aPBQQporDK7UViqavDYzOoTULV8qKB5DxpDUfmEB
Z+PtFXDS0UtPvaOapBVVYTY/9uQvrNVOPZd2kwplHiQtLlYSRTfgU9vttv5OQEMp
5kE1wblNT9qyOBRyGhFi2PGEsItNlM1ZmiDIU1AO9Kvxw/RP5rdm+3REjIKt9cC+
3eLsmhooCsE1lc96x4SXo6I4+di8cU++FXv+KJHSuGidpnklEv35LULdi8CWCMx5
3Ua97mpQBxlEe+Minj/p3OokpNVXwO2GHTzujWOqUB+yQo+ib+q+WRbFFOMEWBDG
MGdFobPJiZGKSKKeay9t
=RjQs
-----END PGP SIGNATURE-----

sebyan · August 30, 2012, 7:29pm

Thanks ab, it’s sounds a good idea but really I don’t know where to start. I’ve checked the two directories but there’s no huge log file in /var/log and there’r lots of sub-directories which I’m not sure if I should alter them.

hcvv · August 30, 2012, 7:41pm

It is allways difficult to tell what filled up everything. In any case, removing from* /tmp to get space in / will not help in 12.1, because /tmp *is a (temporary) file system of it’s own.

Do you have a separate /home? When not, I would try to check there.

The normal way to find this is using (as* root)*

cd /
du -sh *

starting from the root directory and then see which of the directories there is far to big. Then cd to it and repeat the process.

ab · August 30, 2012, 10:13pm

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> It is allways difficult to tell what filled up everything. In any
> case, removing from- /tmp -to get space in- / -will not help in 12.1,
> because- /tmp -is a (temporary) file system of it’s own.

Maybe I setup things wrong with 12.1, and now again with 12.2, but /tmp
still shows as taking space from /; it does not show up in the ‘mount’
command output, but df does show it as a mountpoint. All very odd.
Anyway, it’s acting decidedly permanent with files (on my 12.1 laptop)
from February (I"ve rebooted since then). Perhaps temporary means it
keeps itself clean somehow, but I’m not seeing a lot of that yet.

Good luck.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQIcBAEBAgAGBQJQP8l3AAoJEF+XTK08PnB5CeEQALXWOKT1Gva1rxGRJZ/NmxQq
3qejgH8slyNlXcWJdEcRyAfyczFCi4no1wvMAjbD1BQzTd3bQFQGablJ+vppQrpU
PVhnEiDklyMtv5lIxbA81DEwrKR764Oc0bo1KGG9P5qofLX9NpREQ/wG5VfuA/eI
uYpf/IBCyno3ZDMAUEozmOxUegK+DowXrkkJL1RHqWnF39OqoZkH6zjBSCZhLutE
QKZPEW/LCpdsk6l2OdtIZV0f6lEzFtLaiAFci5tE1khpN0fsMj1S9GrksPBXgjS0
045hoxEpuDNNmjMnM2MJV8ahw95tTMTVbrH8zWfoCLYAkGka1xkSRBC3YK3TMG+B
0hyrdBQuZRP6h7E+1ZUuHqBoBI1xHE42NPfwI6zJSlehfxDfzs9qrXssoeg8nksq
A0mNtKvdcNy0r6uSOCWm3iXIA25Rldhq9P5O2S3wBMBDezG0lO7AkAHuxl7CZxFi
WFNgLBVJWRXZK4LIG+ufVmcYbTvCUeLVeuxQhivFceWaXY2BCxZspSpj6ORJY/Bf
7vBo7NOklvUm6qDenp332MkhUAMYF6tR3uI0PWngTnIrsg5+jNrZN1zzNz3B4Ksb
E1W23gTffbJHuBcwpm6RtKeoxPT2ux/muYxZlbIlPkgC6gyUFh3O/KTXA416Zfij
VJWGnuXTQxits/Xj8rMC
=YlQA
-----END PGP SIGNATURE-----

hcvv · August 30, 2012, 10:35pm

Hm, you could be correct. And I aws maybe to quick and did not check.

In any case it is a good idea for te OP to check there. (and setting it to clean at boot in YaST > System > /etc/sysconfig editor and then - System - Cron - CLEAR_TMP_DIRS_AT_BOOTUP to yes as soon as his system is running fully again).

user · August 31, 2012, 5:16am

On 08/30/2012 06:06 PM, sebyan wrote:
>
> I have done a space checking which shows me
>
> -Filesystem Size Used Avail Use% Mounted on
> rootfs 20G 20G 0 100% /
> /dev/sdb2 20G 20G 0 100% /
> -
> Anyone can tell me what’s the problem?

your disk is full…

are you running BTRFS ??
if yes, i guess you have lots of snapshots…
if no, i guess you have a runway process filling /var/log/messages
or: the drive is just filled up with movies, music, photos, etc…
or: junk in /tmp and/or /var/tmp

follow Henks tip to learn where the hogs live…

–
dd

sebyan · August 31, 2012, 12:19pm

Hi guys, much appreciated for your kindly replies. A space checking shows

*/dev/sdb3 437G 230G 207G 53% /home

*Following Henks tips, #du -sh * returns two large items

*230G home
19G usr

*I’m starting with /home first.

sebyan · August 31, 2012, 12:25pm

I’m not sure whether I was running BTRFS. Apart from some data, I don’t store movie/music stuff on this machine. Btw two Matlabs have been left running/idling for days before the restart, might be eating up the space somewhere?

hcvv · August 31, 2012, 12:43pm

That is useless! /home is on a separate partition and as you show, it is only 53% full.

You must concentrate on /.
My /usr is only about 4 GB. What did you put there? Did you install masses and masses of software?

sebyan · August 31, 2012, 1:16pm

Thanks Henk, I don’t think I’ve installed huge amount of softwares. But /usr #du -sh * shows some large ones

2.1G lib64
12G local
3.5G share

and /usr/local

4.2G MATLAB
7.0G texlive

and /usr/local/texlive

3.1G 2011
3.9G 2012

which means the problem is from the update of texlive yesterday. By update, I thought the old version was removed inthe mean time, however the ‘installation from internet’ just brought another copy on the machine. With such little usable disk space, is there anyway to uninstall the 2011?

Usually I just installed/update softwares as default but it looks like I shouldn’t have put bulky things like MATLAB and texlive in /usr/local. Is there any way to alter the installation to somewhere else?

please_try_again · August 31, 2012, 1:34pm

No, unless you explicitely mount it in RAM, which is not the default under 12.1. But it might change (or not) in future releases: AJ’s openSUSE ramblings: /tmp as tmpfs for openSUSE?

hcvv · August 31, 2012, 1:51pm

i allready admitted that I was in error here. Applogize again.

please_try_again · August 31, 2012, 1:52pm

@sebyan

Please just post the ouptut of this command inside code tags ([noparse]

...

[/noparse]) and do not try to interpret it:

# df -hl -x tmpfs -x devtmpfs -x rootfs

hcvv · August 31, 2012, 2:05pm

Well, not standard openSUSE ways of installing should be watched with suspicion.

I do know nothing about uninstalling that non-standard installation. You should consult the documentation that came with it. Or unravel the way you installed it and do that backward (I hate products that have only install instructions/scripts and not the same for uninstall).

I have only 860K in /usr/local. When you decide to use your system for products that want to find themselves back inside /usr/local, you should take decissions in the design stage (designing the sizes of your partitions). I know one can not allways see the future ;). But one can in any case watch the used GB especialy after one installs a bunch of softeware beyond what is normal (and by “normal” I mean that a 20GB /, with /home excluded, is adequate enough for s a standad openSUSE install and a lot of extra packages, I have 14G for /). But that is the way it is. Changing later is allways more problematic.

sebyan · August 31, 2012, 2:17pm

Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb2        20G   19G  574M  98% /
/dev/sda2       369G  309G   60G  84% /windows/D
/dev/sda1        98G   89G  9.4G  91% /windows/C
/dev/sdb3       437G  229G  208G  53% /home

This is produced when I finally come back to KDE:)
Before I could login to KDE, the same command produced similar output with one more item in IceWM

devtmpfs 3.9G 40K 3.9G 1% /dev

sebyan · August 31, 2012, 2:30pm

hcvv:

Well, not standard openSUSE ways of installing should be watched with suspicion.

I do know nothing about uninstalling that non-standard installation. You should consult the documentation that came with it. Or unravel the way you installed it and do that backward (I hate products that have only install instructions/scripts and not the same for uninstall).

I have only 860K in /usr/local. When you decide to use your system for products that want to find themselves back inside /usr/local, you should take decissions in the design stage (designing the sizes of your partitions). I know one can not allways see the future ;). But one can in any case watch the used GB especialy after one installs a bunch of softeware beyond what is normal (and by “normal” I mean that a 20GB /, with /home excluded, is adequate enough for s a standad openSUSE install and a lot of extra packages, I have 14G for /). But that is the way it is. Changing later is allways more problematic.

Back to KDE!
Still need to figure out how to get rid of the duplicate texlive, but it should be ok. I haven’t really thought too much before hand, simply followed the suggested partitions. When I decided to install some software, I didn’t pay much attention to it either. Now it’s time to keep an eye on my system from time to time. On the good side, I think the system has got everything I need, hopefully no many coming softwares.
Well it is a pain to have a few dark days once a month, but it is a gain to learn something once a month ;).

Thanks again to everyone!

please_try_again · August 31, 2012, 2:38pm

sebyan:

Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb2       ** 20G   19G  574M  98%** /
/dev/sda2       369G  309G   60G  84% /windows/D
/dev/sda1        98G   89G  9.4G  91% /windows/C
/dev/sdb3       437G  229G  208G  53% /home

Thank you. It is more readable. Now it becomes obvious that you should free some space in your root partition. 20 GiB is not very much btw… considering all the free space left in /home (about 208 GiB). You have 3 options:

You can clear what needs to be cleared, such as /tmp and different cache directorie, and you can uninstall some programs.
You can try to shrink /home, then resize / from a live system … but be warned that resizing partitions and file systems is not 100% safe.
You can move stuff (maybe MatLab) from / to /home and create symlinks.

please_try_again · August 31, 2012, 2:43pm

The suggestion was not that clever. You should have taken 40 GiB instead of 20 for / - and/or put /usr/local on a separate partition (but not /usr - for other reasons).

sebyan · August 31, 2012, 3:03pm

Yes I think I’d take the third option to move something first. Moving /usr/local away sounds a good idea. Will change the partition at next update, maybe 12.2?