Machines with large number of CPUs and a lot of RAM

(1) What are the limitations of the Linux kernel to handle large number of SMP CPUs?
(2) What are the problems with huge RAM in SMP environment with large number of CPUs?
(3) What about the availability of such machines (to run Linux)?

I am looking for Linux machines similar to Sun Enterprise servers that run Solaris and can scale to 100+ processors with terabytes of RAM.

I know that some of the vendors like HP etc. make Intel/AMD based systems but they don’t have machines with a lot of CPUs.

syampillai wrote:
> (1) What are the limitations of the Linux kernel to handle large number
> of SMP CPUs?
> (2) What are the problems with huge RAM in SMP environment with large
> number of CPUs?

Supercomputer operating systems, today are most often variants of
Linux… “http://en.wikipedia.org/wiki/Supercomputer

“Linux now has become so technically powerful that it lays claim to a
prestigious title–it runs more of the world’s top supercomputers than
any other operating system.”
http://www.forbes.com/2005/03/15/cz_dl_0315linux.html

Linux is an industrial strength OS, next question:

> (3) What about the availability of such machines (to run Linux)?
>
> I am looking for Linux machines similar to Sun Enterprise servers that
> run Solaris and can scale to 100+ processors with terabytes of RAM.
>
> I know that some of the vendors like HP etc. make Intel/AMD based
> systems but they don’t have machines with a lot of CPUs.

contact Cray, IBM, and SGI if HPs are not strong enough for you they
can provide what you need up to the top supercomputers in the world…
http://www.top500.org/


deConficter

All of those Supercomputers are not SMP machines. No SMP offering from any of those manufacturers (including Cray, IBM, SGI etc.) with Linux OS.

I want SMP machines because the current limitation of database implementation is that it requires huge amount of shared memory. In a distributed computing environment, efficient shared memory implementation is not very efficient. So, no database vendor is in a position to offer a good solution that can run on distributed computing environment. Commercial offering from database vendors are there but solutions are very limited now.

> All of those Supercomputers are not SMP machines. No SMP offering from
> any of those manufacturers (including Cray, IBM, SGI etc.) with Linux
> OS.

you seem to know more about their offerings than i do, so i’ll fall
silent and listen.

but, before i go: how can you define “SMP machine” so that none of the
top 500 supercomputers (most of which run some flavor of Linux) are
not one?


deConficter

How many CPUs are we talking about?

I’ve only gone up to 32 on my openSUSE boxes so I can vouch for excellent performance up to that point.

Novell: SLE 11 Kernel Limits

IBM: Linux and symmetric multiprocessing

> (http://www.novell.com/linux/techspecs.html?tab=1)
> (http://tinyurl.com/cdzfuh)

ah! you spoiled my fun…i was waiting for him to explain SMP to me
and why no one is selling a linux machine strong enough for his needs…


deConficter

If you look back ten years I think you’ll find one of the reasons Linux managed to gain traction in businesses initially is that it scaled well compared to NT. Do a Google search and you’ll find an enormous amount of Microsoft FUD on this issue–they were getting hurt.

On Wed, 2009-04-08 at 04:06 +0000, syampillai wrote:
> All of those Supercomputers are not SMP machines. No SMP offering from
> any of those manufacturers (including Cray, IBM, SGI etc.) with Linux
> OS.
>
> I want SMP machines because the current limitation of database
> implementation is that it requires huge amount of shared memory. In a
> distributed computing environment, efficient shared memory
> implementation is not very efficient. So, no database vendor is in a
> position to offer a good solution that can run on distributed computing
> environment. Commercial offering from database vendors are there but
> solutions are very limited now.

Sigh… not sure what happened to my post yesterday.

HP makes an ia64 box that’s 128way with 2TB of memory where
SLES is fully supported.

HP Superdome models support Linux, but in a limited way. For example, they don’t support dynamic de-allocation of processors when running Linux. Another issue with HP is that I am not sure about the future of this model since it uses Itanium processors. (When they dumped other processors, they announced End-Of-Life of other servers in the past).

64+ CPUs. I know that Linux, especially 2.6 kernels, can scale on SMP machines. Which were the machines you used with 32 CPUs?