Distributed filesystems

Dan Hanks hanksdc at plug.org
Fri Dec 16 10:47:23 MST 2005


Hi folks,

Lately I've been looking into distributed/cluster filesystems and wanted 
to see if other people have had experience and recommendations in this 
area.

In a utopian world, I'd love to see a storage cluster, where to add to the 
storage capacity, all I'd have to do is add another commodity machine with 
whatever storage it happened to have and tell the system to add its storage
to the cluster.

In that same utopian world, if I brought a box in the storage cluster 
down, or if it died or something like that, it wouldn't be a real issue 
since the data would be stored across N other of the servers. When I 
brought the machine back up it would intelligently sync back up with the 
cluster.

Ideally I'd want a shared namespace across the cluster (think NFS), and 
synchronous updates (I.e, if one client machine makes an update, then that 
update is 'immediately' visible to the rest of the client machines).

I've been looking into Lustre (http://www.lustre.org/), and it looks to be 
a close contender, but still has a bit to go in terms of my utopian world 
above.

Mogilefs (http://www.danga.com/mogilefs/) seems to be another possibility.

I'd love to be able to replace a NetApp with such a cluster. Lots of 
potential cost-savings, etc. Essentially, I'm looking for something like 
GoogleFS, but able to handle any size file.

Thanks for any pointers,

-- Dan



More information about the PLUG mailing list