Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> current direction in namenode HA


+
Sujee Maniyam 2012-09-04, 22:37
+
Uma Maheswara Rao G 2012-09-04, 23:09
+
Uma Maheswara Rao G 2012-09-04, 23:17
+
Ted Yu 2012-09-04, 23:13
Copy link to this message
-
Re: current direction in namenode HA
Hi Sujee,

On Tue, Sep 4, 2012 at 3:37 PM, Sujee Maniyam <[EMAIL PROTECTED]> wrote:

> I am trying to understand the current state / direction of  namenode
> HA implementation.
>

Thanks for the interest.
>
> For using shared directory, I see the following options
> (from
> http://www.cloudera.com/blog/2012/03/high-availability-for-the-hadoop-distributed-file-system-hdfs/
>   and  https://issues.apache.org/jira/browse/HDFS-3278)
>
> 1) rely on external HA filer
>

This is currently committed / released.
>  2) multiple edit directories
>

This was an option that was considered (and mentioned in the blog post you
referenced) but I don't know of anyone actively working on this.
> 3) book keeper
>

This is being actively developed. There's not an umbrella JIRA or label
that I'm aware of which tracks this work, but if you search the HDFS JIRA
for "bookkeeper" or "bkjm" (Bookkeeper journal manager) you should find
more info.
>  4) keep edits in HDFS / quorum based
>

This is also being actively developed. See this JIRA for more info:
https://issues.apache.org/jira/browse/HDFS-3077
>
> is there going to be an 'official / supported' method, or it is going
> to be a configurable choice when setting up a cluster?
>

At least for the time being, only option 1 is released. Options 3 and 4
will likely be completed / released soon, and once that happens all of
these will be mutually-exclusive options in HDFS. Each of these approaches
will coexist as a different option that can be configured when setting up
an HA HDFS cluster.

I hope that clears things up.

--
Aaron T. Myers
Software Engineer, Cloudera
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB