Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> 2 General questions - NameNode SPOF, Byzantine fault tolerance


Copy link to this message
-
Re: 2 General questions - NameNode SPOF, Byzantine fault tolerance
Hey Maxim,

Check out https://issues.apache.org/jira/browse/HDFS-1064 for the umbrella
JIRA for NN HA. For specific solutions, you may be interested in Facebook's
AvatarNode (https://issues.apache.org/jira/browse/HDFS-976), or China
Mobile's approach (http://github.com/gnawux/hadoop-cmri).

As you can see, multiple solution paths are being explored right now, and
your input would be valuable.

There are no BFT algorithms in the Hadoop codebase.

Thanks,
Jeff

On Tue, Aug 3, 2010 at 1:34 AM, Maxim Veksler <[EMAIL PROTECTED]> wrote:

> Hi,
>
> New to Hadoop. Learning the architecture, code, ecosystem and everything
> else... love what I've seen so far. Some parts are brilliant.
>
> I've been wondering. If the NameNode is a single point of failure why not
> employ a highly available solution to replace the current
> implementation? For ex: Is there any reason why not to mimic the fsImage
> file structure in Cassandra? or in ZooKeeper?
>
> Another thing, I've only recently learned about BFT. Does hadoop uses
> this algorithm somewhere in it's logic? I would love to see the
> implementation if it does. Are there any plans for this? Seems like a good
> fit for a large scale cluster.
>
>
> Thanks for the continuous commits flow guys! :)
> Great work.
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB