hyper threading and pthread_setaffinity_np

amin63 · November 16, 2011, 1:11pm

hi, i install opensuse 11.4 on intel core i7 with 4 processor.
i see 8 core because hyperthread is enable .
i want write a multithread program and use pthread_setaffinity_np for assign threads to cores,
which core numbers are real and wich cores are virtual.

martin_helm · November 16, 2011, 10:17pm

Am 16.11.2011 13:16, schrieb amin63:
>
> hi, i install opensuse 11.4 on intel core i7 with 4 processor.
> i see 8 core because hyperthread is enable .
> i want write a multithread program and use pthread_setaffinity_np for
> assign threads to cores,
> which core numbers are real and wich cores are virtual.
>
>
Just treat all 8 as real this is the way the operating system sees them,
since you are on top of the operating system with your programming you
have no way to see them in a different way.

staffantj · November 16, 2011, 11:55pm

If you look at /sys/devices/system/nodeX/cpuY/cache/indexZ/shared_cpu_list you’ll get a list of which cores are hyper-threaded together.

To see which cpu numbers correspond to which actual cores (it’s probably not the obvious scenario), you can look at the Physical Id and Core Id in /proc/cpuinfo.

martin_helm · November 17, 2011, 12:29am

Am 16.11.2011 23:56, schrieb staffantj:
>
> If you look at
> /sys/devices/system/nodeX/cpuY/cache/indexZ/shared_cpu_list you’ll get a
> list of which cores are hyper-threaded together.
>
> To see which cpu numbers correspond to which actual cores (it’s
> probably not the obvious scenario), you can look at the Physical Id and
> Core Id in /proc/cpuinfo.
>

Maybe I misunderstood the original question. But again what the OP has
are 8 virtual cores, there are no first class or second class cores here
(cores which are more real and others which are less real), that’s the
point. Of course if the intention of the original question was somehow
to ensure that no hyperthreading (or less hyperthreading) takes place
and only one virtual core per physical core shall mainly be used one can
use this information (though I fail to see the benefit).
But one can not use it to decide between the “quality” of the 8 cores
all of them are “virtual”.

staffantj · November 17, 2011, 1:16am

The point is that each hyperthread on an i7shares a level 3 cache with only one other hyper thread (it’s partner).

It’s also useful to know exactly which pair of hyperthreads form a real thread, for purposes of sharing level-2 cache lines, as well as balancing their memory requests (so that one doesn’t starve the other).

When dealing with HPC type problems, all of these sorts of things come in to play (unfortunately).

martin_helm · November 17, 2011, 1:52am

Am 17.11.2011 01:26, schrieb staffantj:
>
> The point is that each hyperthread on an i7shares a level 3 cache with
> only one other hyper thread (it’s partner).
>
> It’s also useful to know exactly which pair of hyperthreads form a real
> thread, for purposes of sharing level-2 cache lines, as well as
> balancing their memory requests (so that one doesn’t starve the other).
>
> When dealing with HPC type problems, all of these sorts of things come
> in to play (unfortunately).
>

Thanks for clarifying. I did not even think about the caching here.
Guess I need to play with that with some appropriate model problem to
bring me up to date what performance one can gain with it and for which
kind of problems one can find real benefits.
A study I have seen some years ago (don’t remember which, but I can find
it again) did not show that much performance boost coming from
hyperthreading tuned software (at least the figures did not impress me
it “boosted” the performance only by a few percent at a high cost of
additional coding effort so I never looked back at that, but of course
all that can be outdated meanwhile).

Do you have a good pointer to something more actual worth reading? I
started google right now after your post and obviously I can spend the
rest of my live looking at all the articles (which is not my intention).

Hope this question from my side here does not go too much off topic in
this thread now. If it does, ignore it.

staffantj · November 17, 2011, 2:18am

Hyperthreading is the ultimate classic “Your Mileage Will Vary” scenario.

If you have threads that are memory intensive, it can be a real boon.

If you have process intensive threads, it’s a nightmare.

Balancing it all out for any given hardware is a black art. I’ve not unfortunately come across anything particularly definitive on the subject, although I seem to recall some good work on the Intel Software Development wiki of what tests to try out.

In short, to see what the effect is on any given system / software, you have to experiment. What we’ve found is that no matter what your ideas about what will be an optimal setup going into the tests, they will almost invariably be wrong by the time the tests are completed.

One thing to mention (although for our home systems it’s probably not an issue) is that as of the i3/5/7 series CPUs (and their Xeon counterparts), the memory architecture on Intel CPUs is now NUMA if you have multiple sockets. So some memory can take 4-10 times as long to get to as other addresses …