CPU affinity

English Hardware
#1

hello,

I’m running the following command to benchmark my system:

dd if=/dev/zero bs=1M count=10000 |openssl enc -k qqqq -aes-128-cbc |pv > /dev/null

this command starts 3 processes: dd, openssl and pv. when I start top I see that all of them run on the same core and are competing for it. to workaround this problem I use taskset to place processes on different cores:

taskset -c 0 dd if=/dev/zero bs=1M count=10000 |taskset -c 1 ./openssl enc -k qqqq -aes-128-cbc |taskset -c 2 pv > /dev/null

now the benchmark run on 3 cores and gives waay better results.

question: how to change default (wrong?) scheduling policy?

thanks in advance!

ps. my system is openSUSE 11.1 on Intel Core i7 CPU 940, HT enabled.

#2

You may want to do some googling with regards to CFS tuning, it has vexed many people in the past. There are some people that have managed to tune it for better performance, but may involve a custom-config/recompile of the kernel. Make sure you reference CFS for 2.6.27, since it evolves with each release since 2.6.23.

One other thing to try is to make sure that your cpufreq governer is set to performance. On dynamic or powersave modes, and where CPUs support different power states per core, the balancer could start scheduling threads and processes to a single core to reduce power draw on the remaining ones. Not sure if that would be the case, here, though.

Hope this helps…

Cheers,
KV

#3

OK, started digging about CFS and I’m lost. It’s hardly possible to find any reasonable docs for a non-kernel developer. I have not even found how to check what scheduler I am using right now…

according to Ingo’s email:

There is only one central tunable:
/proc/sys/kernel/sched_granularity_ns

but this file does not exist on my system (default oS 11.1 kernel). :o

any hints?

#4

Hi
There is an error in the documentation it’s actually
sched_min_granularity_ns. There is a document in the kernel source;


/usr/src/linux-2.6.27.7-9/Documentation/scheduler/sched-design-CFS.txt

If you don’t have the source, I’ve put a copy on nopaste.com;
Link to
sched-design-CFS.txt

–
Cheers Malcolm °¿° (Linux Counter #276890)
openSUSE 11.1 x86 Kernel 2.6.27.7-9-default
up 1 day 4:12, 2 users, load average: 0.37, 0.21, 0.19
GPU GeForce 6600 TE/6200 TE - Driver Version: 180.22

#5

yes, I have installed kernel docs. the mentioned bug about proc entry is also there :slight_smile:

now, whatever I put in /proc/sys/kernel/sched_min_granularity_ns - big number, small number - always all the processes stick to one core…

#6

Just had a look at the kernel config;

CONFIG_USER_SCHED is not set
CONFIG_FAIR_CGROUP_SCHED=y

So, I would imagine according to that document you would need to follow
the example for creating cpu.shares?

–
Cheers Malcolm °¿° (Linux Counter #276890)
openSUSE 11.1 x86 Kernel 2.6.27.7-9-default
up 1 day 5:51, 4 users, load average: 0.04, 0.13, 0.16
GPU GeForce 6600 TE/6200 TE - Driver Version: 180.22

#7

If I understand correctly, you want me to setup cpu.shares for each process, right? I believe that this is a tool to achieve rather a kind of cpu time priorities, but let’s go:


root@MiP /usr/src/linux/Documentation/scheduler mkdir /dev/cpuctl
root@MiP /usr/src/linux/Documentation/scheduler mount -t cgroup -ocpu none /dev/cpuctl
root@MiP /usr/src/linux/Documentation/scheduler cd /dev/cpuctl
root@MiP /dev/cpuctl mkdir dd openssl pv
root@MiP /dev/cpuctl echo 1024 > dd/cpu.shares
root@MiP /dev/cpuctl echo 1024 > openssl/cpu.shares
root@MiP /dev/cpuctl echo 1024 > pv/cpu.shares
root@MiP /dev/cpuctl # start dd|openssl|pv on another console
root@MiP /dev/cpuctl echo `pidof dd` > dd/tasks
root@MiP /dev/cpuctl echo `pidof openssl` > openssl/tasks
root@MiP /dev/cpuctl echo `pidof pv` > pv/tasks
root@MiP /dev/cpuctl pidstat 1 1
Linux 2.6.27.7-9-default (MiP)  01/20/09        _x86_64_

18:15:49          PID    %usr %system  %guest    %CPU   CPU  Command
18:15:50        32477    0.00    3.96    0.00    3.96     5  dd
18:15:50        32478   86.14    5.94    0.00   92.08     5  openssl
18:15:50        32479    0.00    5.94    0.00    5.94     5  pv

as we can see, dd, openssl and pv processes are still on the same core.

#8

Hi
:frowning: Maybe it’s worthwhile posting on the kernel mailing list, I’m sure
it’s something simple…

–
Cheers Malcolm °¿° (Linux Counter #276890)
openSUSE 11.1 x86 Kernel 2.6.27.7-9-default
up 2 days 13:07, 1 user, load average: 1.55, 0.88, 0.37
GPU GeForce 6600 TE/6200 TE - Driver Version: 180.22