Distributed filesystems

Shane Hathaway shane at hathawaymix.org
Fri Dec 16 20:25:16 MST 2005

Dan Hanks wrote:
> Lately I've been looking into distributed/cluster filesystems and wanted 
> to see if other people have had experience and recommendations in this 
> area.
> In a utopian world, I'd love to see a storage cluster, where to add to 
> the storage capacity, all I'd have to do is add another commodity 
> machine with whatever storage it happened to have and tell the system to 
> add its storage
> to the cluster.

Well, I've just put together this kind of system.  I'm working with my 
employer to release it as open source software.  My system aims for high 
reliability and simplicity.

> I've been looking into Lustre (http://www.lustre.org/), and it looks to 
> be a close contender, but still has a bit to go in terms of my utopian 
> world above.

IMHO and AFAIK, Lustre aims for high speed rather than reliability and 
ease of administration.  It expects each node to be very reliable.

> Mogilefs (http://www.danga.com/mogilefs/) seems to be another possibility.

IMHO Mogilefs is currently the best of the open source systems because:

   - It's simple and short

   - You can balance reliability vs. storage cost per file by adjusting 
the number of replicas

   - It's running now


   - MogileFS is not a POSIX filesystem

   - MogileFS is currently missing features like scanning all files

> I'd love to be able to replace a NetApp with such a cluster. Lots of 
> potential cost-savings, etc. Essentially, I'm looking for something like 
> GoogleFS, but able to handle any size file.

I think the existence of GoogleFS is inspiring a lot of people to build 
their own storage systems rather than buy from the big, proprietary 
companies.  Within a few years, building your own thousand-drive cluster 
is going to be easy.

About the hardware: if you're building a few terabytes, the best 
hardware deal I've found is at coraid.com.  My employer won't consider 
buying from them because they're not a huge company, but if everyone on 
the list buys from them, maybe coraid will grow and we'll be able to buy 
from them after all! ;-)


More information about the PLUG mailing list