Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # dev - current direction in namenode HA


+
Sujee Maniyam 2012-09-04, 22:37
+
Uma Maheswara Rao G 2012-09-04, 23:09
+
Uma Maheswara Rao G 2012-09-04, 23:17
+
Ted Yu 2012-09-04, 23:13
Copy link to this message
-
Re: current direction in namenode HA
Aaron T. Myers 2012-09-04, 22:55
Hi Sujee,

On Tue, Sep 4, 2012 at 3:37 PM, Sujee Maniyam <[EMAIL PROTECTED]> wrote:

> I am trying to understand the current state / direction of  namenode
> HA implementation.
>

Thanks for the interest.
>
> For using shared directory, I see the following options
> (from
> http://www.cloudera.com/blog/2012/03/high-availability-for-the-hadoop-distributed-file-system-hdfs/
>   and  https://issues.apache.org/jira/browse/HDFS-3278)
>
> 1) rely on external HA filer
>

This is currently committed / released.
>  2) multiple edit directories
>

This was an option that was considered (and mentioned in the blog post you
referenced) but I don't know of anyone actively working on this.
> 3) book keeper
>

This is being actively developed. There's not an umbrella JIRA or label
that I'm aware of which tracks this work, but if you search the HDFS JIRA
for "bookkeeper" or "bkjm" (Bookkeeper journal manager) you should find
more info.
>  4) keep edits in HDFS / quorum based
>

This is also being actively developed. See this JIRA for more info:
https://issues.apache.org/jira/browse/HDFS-3077
>
> is there going to be an 'official / supported' method, or it is going
> to be a configurable choice when setting up a cluster?
>

At least for the time being, only option 1 is released. Options 3 and 4
will likely be completed / released soon, and once that happens all of
these will be mutually-exclusive options in HDFS. Each of these approaches
will coexist as a different option that can be configured when setting up
an HA HDFS cluster.

I hope that clears things up.

--
Aaron T. Myers
Software Engineer, Cloudera