numactl error on Nehalem

English Hardware
#1

Hi all,

well, I’m trying opensuse11.1 on a Nehalem machine (two sockest mounting Intel Xeon X5570 2.93GHz), and I’ve noted that running the command “numactl --hardware” I don’t get the inventory of available nodes and associated memory for each!

I get the following output:

available: 3 nodes (0-2)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
node 0 size: 6135 MB
node 0 free: 5398 MB
libnuma: Warning: /sys not mounted or invalid. Assuming one node: No such file or directory
node 1 cpus:
node 1 size: <not available>
node 1 free: <not available>
node 2 cpus:
node 2 size: 6144 MB
node 2 free: 6023 MB
No distance information available.

Seems that this goes beyond a simple visualization error, since I’ve run also the STREAM memory benchmark and I’ve got very poor preformance while running the same benchmark on opensuse10.3 installed on the same machine I get triple of the performance registered on opensuse11.1!!

I would appreciate a lot if any one have suggestions

#2

Hello

i have a quad / 4 node system

to let it work proper i deleted some modules from lib/modules

in /lib/modules/2.6.29-6-default/kernel/arch/x86/perfmon/
delete all modules

in /lib/modules/2.6.29-6-default/kernel/drivers/mtd/maps
delete ck804xrom.ko

This will help

#3

Hello,

well, I’ve tried to delete those modules and didn’t see any performance improvements, moreover, the command “numactl --hardware” still unable to detect the numa architecutre on that machine.

I’ve read in the following link
http://download.opensuse.org/factory/repo/oss/ChangeLog
that there is an error (that should be already fixed!!) in the numactl command in which it doesn’t print all available CPUs!! but again I think it goes beyond a simple visualization error since I get very poor memory perofrmance results (using the STREAM benchmark) with rispect to openSuse10.3 installed on the same machine with the same HW configurations!!

does anyone have an’idea of what’s happening here with opensuse 11.1 ??

Best Regards

#4

wissame4,

sorry it didnt work out four you.

But to see if all the cpu’s are working:

install GKRELLM a gnome app wich let you see all the cpu’s
the load on the cpu’s
how proc is behaving and a lot more

Succes

#5

It looks as a bug in the kernel. I found the same problem on a server w/dual Nehalem Xeon E5520 w/Supermicro X8DT3 mobo. I used default 2.6.27.7-9 kernel.
It gives erroneous /sys/devices/system/node content (subdirs node0 and node2 instead of node0 and node1).
Theoreticllly it may be also BIOS bug.

  1. I didn’t find in bugzilla any corresponding bug messages

  2. Do I understand you correctly that numactl works OK ON Nehalem w/10.3 kernel ? (I know that for Opteron Barcelona it’s OK).

  3. Do somebody know about numactl+Nehalem behaviour w/updated 11.1 kernels, in particular 2.6.27.25-0.1.1 ?
    or may be in 11.2 ?

  4. If nobody knows about this situation - is there other way to communicate w/OpenSUSE developers than write formal bug message ?