Memory usage problem with 11.4 on VmWare

ubuuntu · October 5, 2011, 6:56pm

Hi,

A spot of bother here, perhaps somebody can help.

The setup: vanilla openSuse 11.4, minimum/headless install. VmWare ESX server. The vmware-checkvm tool reports “VM’s hw version is 4” and “VMware software version 6 (good)”. Setup is 2 CPU, 8Gb ram etc.

Quickly I’ve noticed a curious problem. The memory usage is consistently reported as:

indcldaesvs1002:/home/aess # free
             total       used       free     shared    buffers     cached
Mem:       8180204    5766872    2413332          0      85640     321804
-/+ buffers/cache:    5359428    2820776
Swap:      4192252          0    4192252

Something seems to be using ~5Gb of RAM. Strangely, no user-side process seems to be anywhere near this figure. I’ve run top and sorted by memory usage yielding:


top - 22:19:00 up 1 day,  2:53,  1 user,  load average: 0.03, 0.04, 0.08
Tasks:  81 total,   1 running,  80 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.2%us,  0.2%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   8180204k total,  5767004k used,  2413200k free,    85720k buffers
Swap:  4192252k total,        0k used,  4192252k free,   321804k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2364 mysql     20   0  111m  20m 5648 S    0  0.3   1:05.15 mysqld
 2600 root      20   0 24552 3380 2936 S    0  0.0   0:11.45 polkitd
 1652 root      20   0 24092 3332 2764 S    0  0.0   1:53.09 vmtoolsd
 2532 root      20   0 27624 3168 2560 S    0  0.0   0:00.42 console-kit-dae

etc

Since I’ve never come across anything like this, I’ve tried a few things - stopping firewall and a few other services and checking reported ram. vmstat -m didn’t yield anything obvious.

Any help would be appreciated.

Many thanks,
Ub

DenverD · October 5, 2011, 7:22pm

On 10/05/2011 07:06 PM, ubuuntu wrote:
>
> Quickly I’ve noticed a curious problem. The memory usage is
> consistently reported as:
>
> indcldaesvs1002:/home/aess # free
> total used free shared buffers cached
> Mem: 8180204 5766872 2413332 0 85640 321804
> -/+ buffers/cache: 5359428 2820776
> Swap: 4192252 0 4192252
>
> Any help would be appreciated.

looks normal to me…i guess you maybe don’t yet have a good
understanding of the differences between the way Linux and consumer
systems uses memory…review those differences here and then ask more if
you need: http://www.linuxatemyram.com/

–
DD
Caveat
openSUSE®, the “German Automobiles” of operating systems

ubuuntu · October 5, 2011, 8:19pm

looks normal to me…i guess you maybe don’t yet have a good
understanding of the differences between the way Linux and consumer
systems uses memory…review those differences here and then ask more if
you need: Help! Linux ate my RAM!

Thanks for your reply. Perhaps you get too many silly queries so your defaul answer is a bit condescending. Never mind.

The numbers show buffers+cache using only ~400Mb. Total available memory, i.e. free + buffers + cache, is 2.8Gb out of 8Gb.

To summarise:

None of the user side processes are using much RAM (vanilla installation - on my other boxes this is ~300Mb or so).
Buffers and cache total is ~400Mb

… yet the system reports RAM usage of 5.3Gb

In any case my suspicion is with VmWare / drivers.

ubuuntu · October 5, 2011, 8:45pm

Ok, I have clue. Stopping vmtoolsd releases the memory. Clearly a VmWare problem.

$ /etc/init.d/vmtoolsd stop
$ free
             total       used       free     shared    buffers     cached
Mem:       8180204    1131776    7048428          0      92736     342600
-/+ buffers/cache:     696440    7483764
Swap:      4192252          0    4192252

DenverD · October 5, 2011, 9:14pm

On 10/05/2011 08:26 PM, ubuuntu wrote:

> Thanks for your reply. Perhaps you get too many silly queries so your
> defaul answer is a bit condescending. Never mind.

i didn’t think you question was silly…uninformed, yes…silly no! i
asked just about the very same question myself, a long time ago…i
understand it is somewhat difficult to fully understand how Linux uses
memory (in a much better way, that some other systems)

it was not a “default answer” and it wasn’t intended to be condescending…

it was on the other hand intended to be both factual and helpful…

but, i guess not helpful enough because…well, hmmmmm…evidently you
didn’t fully grasp the meaning of this (in the article i pointed you to):

Why does top and free say all my ram is used if it isn’t?

This is just a misunderstanding of terms. Both you and Linux agrees that
memory taken by applications is “used”, while memory that isn’t used for
anything is “free”.

But what do you call memory that is both used for something and
available for applications?

You would call that “free”, but Linux calls it “used”.

> The numbers show buffers+cache using only ~400Mb. Total available
> memory, i.e. free + buffers + cache, is 2.8Gb out of 8Gb.

to me it looks like

total used free
8,180,204 = 5,766,872 + 2,413,332 (nothing missing)

and

buffers cache
5359428 + 2820776 = 8180204 (and, nothing missing)

> To summarise:
> - None of the user side processes are using much RAM (vanilla
> installation - on my other boxes this is ~300Mb or so).
> - Buffers and cache total is ~400Mb
>
> … yet the system reports RAM usage of 5.3Gb

yes, the RAM usage is 5.3 Gb…but as that article i pointed you to
explains that Linux uses RAM which is still available for use when
needed…

let me say it this way: other operating systems empty RAM as fast as
possible…

Linux on the other hand has the philosophy that unused RAM is wasted RAM!!

where other systems dump everything out of RAM when you (say) close IE
then everything associated with it is dumped instantly and all that
RAM is freed up, ready to use again…

…no so in Linux…like: you just closed IE to free up some ram but what
if the photo you just saved to your desktop (from an IE download) you
now wanna open in Photoshop? the system has to go find it on the disk,
and read it back into memory…but i Linux BLAM it flashes it to your
screen almost instantly because it is already in memory…

so, Linux tries to keep RAM in use…as new stuff comes along needing
memory the old stuff is dumped, automatically (and at the speed of
light, almost)…so, the RAM get pretty full pretty quickly and stays
that way, with almost no relationship to what top says is going on in
user space…

see now? if any of that sounds condescending it is not intended…

> In any case my suspicion is with VmWare / drivers.

nope, no problem…your memory is fine…

–
DD
openSUSE®, the “German Automobiles” of operating systems

ubuuntu · October 6, 2011, 1:22am

In fact, the memory was alocated by a leaky driver - on the kernel side. This is why it was not visible in any user-side processes. Shutting down vmtoolsd unloads the driver which also freed up memory.

I don’t think you really understood my question from the outset. Also, your assumption that you are dealing with a rookie is quite wrong. But thanks for chatting.

hendersj · October 6, 2011, 1:58am

On Wed, 05 Oct 2011 23:26:03 +0000, ubuuntu wrote:

> I don’t think you really understood my question from the outset. Also,
> your assumption that you are dealing with a rookie is quite wrong. But
> thanks for chatting.

Just a point here - we don’t know anything about you other than what you
tell us. You hadn’t indicated your experience level, and in my own
experience (20+ years) of working in online forums, assuming expertise
that isn’t there is far more dangerous than assuming inexperience.

DD’s assumption, given the information provided, was entirely appropriate.

Rather than make your own assumptions about answers being condescending
and that we’ll assume an appropriate level of expertise for a user who is
relatively new to the forums, we ask that you focus on what your question
is rather than engaging in behaviour that may be construed as a personal
attack.

In the openSUSE forums, we have users of all experience levels and
backgrounds (and many of them are not from an IT background). None of
those trying to help out has a crystal ball that will give us information
that isn’t provided by those asking the questions.

Jim

–
Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

robin_listas · October 6, 2011, 3:08am

On 2011-10-06 01:26, ubuuntu wrote:
> Also,
> your assumption that you are dealing with a rookie is quite wrong.

We have no way to know that for certain, unless you tell your experience
level on the first post, so that we can give a reply tailored to your level.

Lacking that info, we assume you are a novice.

–
Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

ubuuntu · October 6, 2011, 2:10pm

All right guys, I thought it pretty obvious from my first post that the discrepancy has nothing to do with buffers/cache or my misunderstanding. DD first tried to convince me that it is perfectly normal that clean minimal install of openSUSE 11.4 uses 6Gb of ram (lol). And that my question is “uninformed”!

I’ll admit to having a short fuse for this kind of stuff. Apologies if I offended anybody.

robin_listas · October 7, 2011, 12:48am

On 2011-10-06 14:16, ubuuntu wrote:
>
> All right guys, I thought it pretty obvious from my first post that the
> discrepancy has nothing to do with buffers/cache or my misunderstanding.
> DD first tried to convince me that it is perfectly normal that clean
> minimal install of openSUSE 11.4 uses 6Gb of ram (lol). And that my
> question is “uninformed”!

It is a typical question here, we have seen it so many times
At least, that’s what we all thought at first look.

Then I had a second look, and thought that there was something strange,
there was memory missing from top’s map. But I could not see a reason. Then
I saw your other post blaming the driver. That made sense, although I do
not see how you found out. I’m curious.

> I’ll admit to having a short fuse for this kind of stuff. Apologies if
> I offended anybody.

No problem. It happens to everybody now and then.

–
Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

DenverD · October 7, 2011, 10:36am

On 10/07/2011 12:48 AM, Carlos E. R. wrote:

> Then I had a second look, and thought that there was something strange,
> there was memory missing from top’s map. But I could not see a reason. Then
> I saw your other post blaming the driver. That made sense, although I do
> not see how you found out. I’m curious.

as far as i can see the info offered in the OP’s initial post still
looks absolutely normal to me…

so, lets open the school house and you (or ubuuntu, or anyone else) can
teach me what looks “strange”…or which part of top’s output is missing
“~5Gb of RAM”…

thanks in advance…i really need this additional education in order
to find how the OP’s “Something seems to be using ~5Gb of RAM” can be
seen in the post #1 output…

i just do not see it…but, i admit to having been blind to the obvious
before!! help please.

–
DD
openSUSE®, the “German Automobiles” of operating systems

robin_listas · October 7, 2011, 1:18pm

On 2011-10-07 10:36, DenverD wrote:
> On 10/07/2011 12:48 AM, Carlos E. R. wrote:
>
>> Then I had a second look, and thought that there was something strange,
>> there was memory missing from top’s map. But I could not see a reason. Then
>> I saw your other post blaming the driver. That made sense, although I do
>> not see how you found out. I’m curious.
>
> as far as i can see the info offered in the OP’s initial post still looks
> absolutely normal to me…
>
> so, lets open the school house and you (or ubuuntu, or anyone else) can
> teach me what looks “strange”…or which part of top’s output is missing
> “~5Gb of RAM”…
>
> thanks in advance…i really need this additional education in order to
> find how the OP’s “Something seems to be using ~5Gb of RAM” can be seen in
> the post #1 output…
>

The display by “free” I’ve never got around to understand. So I use top
top’s part instead:



top - 22:19:00 up 1 day,  2:53,  1 user,  load average: 0.03, 0.04, 0.08
Tasks:  81 total,   1 running,  80 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.2%us,  0.2%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,
0.0%st
Mem:   8180204k total,  5767004k used,  2413200k free,    85720k buffers
Swap:  4192252k total,        0k used,  4192252k free,   321804k cached

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
2364 mysql     20   0  111m  20m 5648 S    0  0.3   1:05.15 mysqld
2600 root      20   0 24552 3380 2936 S    0  0.0   0:11.45 polkitd
1652 root      20   0 24092 3332 2764 S    0  0.0   1:53.09 vmtoolsd
2532 root      20   0 27624 3168 2560 S    0  0.0   0:00.42 console-kit-dae

etc

Ok, I see 8 GiB of ram, of which 5.7 are in use, 2.4 free. There are just
321 MiB in use in cache, and 85 MiB for buffers - that’s almost nothing.
Those 5 gigabytes used have to be inside programs - but the system is just
started and the sort by memory shows that the biggest user is mysql with
just 111M virtual. There is something in hiding

True, sort by memory sorts by the RES column, not virt, I think. It might
show if we could sort by virtual memory instead.

Another curious thing is that Ub must have huge amounts of ram to be able
to give 8 GiB to a VM!

–
Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)