Why am I running out of hard disk space?

We’ve set up a XEN virtual server with 12G hard disk space. The XEN hosts all our network services, configuration and applications (eg. samba, apache, openldap, …) for our office (about 5 persons). All files and data are not stored on this virtual server, but on another server.

This worked perfectly, until today. Nothing is working anymore because 99% of our hard disk space is already used.

How is it possible that so much hard disk space is being used? The XEN virtual server is command line, so a lot of applications (e.g. OpenOffice, …) are not running on this virtual server.

If I know why I have run out of hard disk space, I also hope to find a solution to avoid this problem in the future.

Ivan

eulaersivan wrote:
> This worked perfectly, until today. Nothing is working anymore because
> 99% of our hard disk space is already used.
>
> How is it possible that so much hard disk space is being used?

Well what have you done to try and find out?

What does du tell you?

On 2011-10-04 14:46, eulaersivan wrote:
>
> We’ve set up a XEN virtual server with 12G hard disk space.

That’s not a lot.

> This worked perfectly, until today. Nothing is working anymore because
> 99% of our hard disk space is already used.

Well, find out where the space is being used, no?
We can’t do that for you :slight_smile:

You have tools like “du” that will tell you. Or baobab in gnome or kdirstat
in kde. Also “mc” in console.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

The only thing I could think of, is a flooded /tmp. You can set it to be cleaned up at boot in Yast - System - sysconfigeditor, search for “TMP”.

The largest directory is /usr.

I’ve also looked in /tmp, but there is not so much to be cleaned up.

Are you saying that 12GB is not enough hard disk space for a server installation of OpenSuse 11.4 (so no graphical front end). Users do not have direct access to this virtual server, so it is not possible that the hard disk is filled up with user data (technically it is always possible, but it is very unlikely). All data and records are saved in another server instance and this XEN server mounts (through nfs) the other file server.

So the hard disk should be filled up via regular Yast-updates, or not?

Ivan

12 GB for a headless server install should be enough, IMHO. But I cannot see what’s installed on this server. My install is 12GB but that’s including a full blown KDE, GNOME3, LXDE, XFCE, vdrift and a lot more.

Updates shouldn’t be the issue, since what’s in /usr won’t grow that much. If they’re kept on the system for whatever reason, they should be in /var. You might want to use “find” to look for extraordinary large files or use “du -h --max-depth=1” for large folders.

On 2011-10-04 15:16, eulaersivan wrote:
>
> The largest directory is /usr.

Well, find out what is inside /usr, don’t stop there.

That directory holds programs and libraries, and can be huge.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

Knurpht wrote:
> use “du -h --max-depth=1” for large folders.

Or even

du -sh *

Tx for your answers.

We’ve restored a backup image of 4 months ago (because the server doesn’t store any data, the setup hasn’t changed). Based on this old image, we still have 8.7GB of the 12GB free disk. I have absolutely no idea why the rest of this disk has been used since 4 months.

I’ll write a short script to monitor the disk usage in the future.

Ivan

On 10/04/2011 02:46 PM, eulaersivan wrote:
>
> How is it possible that so much hard disk space is being used?

you might have a runaway process that is sending repetitive error
messages to /var/log/messages (or other log files in the directory)…are
any there over (say) 5 MB look through them and you might find
thousands of repeating messages…if so, return the error to this
thread)…if any are over a gig you probably need to work on your the
means to auto compact, rotate and discard logs…

or, you might have loads of junk in your /tmp (do you?) do NOT manually
try to clean it out while the system is running, that could kill your
system…and, linux uses files in /tmp, and will clean them out
normally…but, if won’t be able to clean out if it is frozen because
the disk is full! so if /tmp is full set it to self clean on next (and
every boot) by following this: http://tinyurl.com/yzmzp5b

or, maybe you have something stuffing its own goodies into /var/tmp and
then not cleaning up…


DD
Caveat-Hardware-Software-
openSUSE®, the “German Automobiles” of operating systems

On 10/04/2011 04:36 PM, rainbowsally wrote:

>> djh-novell;2390247 Wrote:
>> What does du tell you?
>
>
> you can at least launch konqueror (from the commandline if not as a button).

konqueror is a tool not possible to launch!

the OP had specified this is a proper server (no X, and with no X there
is no kdm, no KDE, and no Konqueror)…

Dave had previously given the correct answer in the form of a question:
What does du tell you? and Knurpht expanded the correct answer with “You
might want to use “find” to look for extraordinary large files or use
“du -h --max-depth=1” for large folders.”


DD
Caveat
openSUSE®, the “German Automobiles” of operating systems

On 2011-10-04 16:36, rainbowsally wrote:
> Doing that could be a bit tricky though, but let’s say you can at least
> launch konqueror (from the commandline if not as a button).

How so, if it is a server with no X?


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

On 2011-10-04 17:16, eulaersivan wrote:
>
> Tx for your answers.
>
> We’ve restored a backup image of 4 months ago (because the server
> doesn’t store any data, the setup hasn’t changed). Based on this old
> image, we still have 8.7GB of the 12GB free disk. I have absolutely no
> idea why the rest of this disk has been used since 4 months.

You should have studied the issue first.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

Again, sorry to say so: plain wrong. This is about a headless server, so no gnome, no kde. If you want to help, first listen to those asking for help.

Tx for all the answers.

I’m afraid that I can no longer retrace the problem and post the source of that problem for your reference. We’ve reinstalled a backup of our XEN server and now only 19% is being used.

We are currently writing a script to monitor when that usage expands so we can take action when it is still possible.

I’ll keep your suggestions at hand, but I apologize that I cannot help to identify the problem.

Ivan

On 2011-10-05 09:36, eulaersivan wrote:

> I’ll keep your suggestions at hand, but I apologize that I cannot help
> to identify the problem.

One possible culprit is kernel sources + keeping several version of
kernels. They are large. Otherwise, you probably installed something large.

No matter.

If it happens again, please investigate, track the directories inside for size.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

I’ve followed the directions here to clear /tmp on boot. Thought I did this before, but … guess it needs to be redone. I saw a blog post that says /var can get clogged up and cause problems. Above, it says playing with /var can be dangerous. Is there a safe way to “clean it out?” Right now, /var is only 21.5 MB.

Running 11.4 and KDE 4.7.2

On 2011-10-08 18:46, Prexy wrote:
>
> I’ve followed the directions here to clear /tmp on boot. Thought I did
> this before, but … guess it needs to be redone. I saw a blog post
> that says /var can get clogged up and cause problems. Above, it says
> playing with /var can be dangerous. Is there a safe way to “clean it
> out?” Right now, /var is only 21.5 MB.

/etc/sysconfig/cron
CLEAR_TMP_DIRS_AT_BOOTUP=“yes”

Clear /var? Forget it.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

Well, that’s easy enough. :slight_smile: