Concurrency, was Re: Doh! Stupid Programming Mistakes <humor>

Levi Pearson levi at
Fri Oct 27 13:43:23 MDT 2006

On Oct 27, 2006, at 1:04 PM, Bryan Sant wrote:
>> This is especially true the less tighly connected your compute nodes
>> get.  A multi-processor computer with a Hypertransport bus can
>> probably get away with abstracting away local vs. remote memory
> I believe LinuxNetworx does this, but like you said, they have
> intimate control over the nodes involved and their interconnect.  It's
> more similar to an integrated circuit than a loose network cluster.

Linux Networx systems are typically multi-processor nodes (Intel or  
AMD), where the system board takes care of routing memory load  
instructions properly,  combined with high-speed interconnects such  
as Infiniband and Myrinet.  Infiniband was conceived as a system- 
level bus, but it hasn't really caught on there as far as I can  
tell.  We use it at the same level we use Ethernet; as a network  
rather than a system bus.  Most customers request some sort of MPI  
layer to support their distributed applications, which are typically  
written in C, C++, or Fortran; and each node runs its own Linux image  
upon which processes can run and communicate with one another via the  
MPI library.

So there are really two levels of concurrency going on; there's node- 
local concurrency based on multi-core processors and multiple CPUs  
that have shared memory, and then there's the layer that connects  
many of those systems together with a message-passing model over a  
high-speed network.  There are two advantages to this: It takes  
advantage of the price/performance sweet spot of commodity hardware,  
and it logically separates near-calls from far-calls.  I'm not  
familiar enough with shared-memory supercomputers to know how they  
handled the latter issue, but they handled the first by being really,  
really expensive. :)


More information about the PLUG mailing list