Cluster computing with Linux & Beowulf

Lloyd Brown lloyd_brown at byu.edu
Mon Feb 24 09:01:04 MST 2014


As others have said, Beowulf is a now relatively disused name for the
concept of using commodity hardware to perform large computational tasks
that historically have required large, "bigiron" servers.

As a concept, it's definitely alive today, and very active in the HPC
industry, where I work.  We use it quite extensively in our facility,
since it's a very cost-effective way to get a large, general-purpose HPC
computing facility.

Having said that, it does sound like you're talking about something
that's more of a network-load situation, not a computational-load
situation.  More of a network-balanced cluster, than a computational
cluster.  To be honest, I'm not certain how you'll handle that large of
a number of simultaneous connections.  You'll definitely need to
distribute the connections among a pool of servers in some fashion.  The
trouble is, especially if you're using a hardware network load balancer
like LVS, it's going to have a very hard time keeping up with that
number of connections.  Possibly some hardware solutions exist that can
utilize special-purpose ASICs to balance that many connections, but
you'll have to dig into that.  I honestly don't know.  Possibly one of
Brocade's ServerIron devices or similar?

Either way, it's not the type of load that one generally associates with
HPC-type clusters that the "Beowulf" term usually implies.



Lloyd Brown
Systems Administrator
Fulton Supercomputing Lab
Brigham Young University
http://marylou.byu.edu

On 02/22/2014 02:42 AM, Dan Egli wrote:
> Program X is intended to handle quite literally up to hundreds of millions
> of simultaneous TCP connections across a number of network interfaces (the
> exact number is unimportant).


More information about the PLUG mailing list