Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> [Updated] RE: current direction in namenode HA


Copy link to this message
-
[Updated] RE: current direction in namenode HA
My previous reply messed up the options numbering, It may confuse some people. I am resending it again by correcting that. Appologies for confusions.

>>>1) rely on external HA filer
    If you can invest on NFS filers, it is one option in HA.  If you want to try this, you can use released Hadoop-2 version and try. but below #3 and #4 will avoid this external hardware dependency.
  
>>>2) multiple edit directories
    HDFS already supports, multiple edit directory configuration. I am not exactly sure what option you are referring here.

>>>3) book keeper
   Please take a look at https://issues.apache.org/jira/browse/HDFS-3399
  We are testing with BookeKeeper from last 2/3 months and going well. BK is progressing on autorecovery and security  parts. Almost auto recoverry done (BOOKKEEPER-237) and will be released in BK 4.2 version very soon. BK already started work on security part as well. Also this integration part will come out with next hadoop-2 release as well. Also attached tested scenarios in HDFS-3399 for your reference if you want to take a look. Also there is one subTask in that umbrella  JIRA for user manual  information. BookKeeper Wiki https://cwiki.apache.org/confluence/display/BOOKKEEPER/BookKeeper
    

>>>4) keep edits in HDFS / quorum based
    you can take a look at HDFS-3077. In this umbrella JIRA, work is going actively.
    https://issues.apache.org/jira/browse/HDFS-3077 you can get more design details in that JIRA.

Regards,
Uma

________________________________________
From: Sujee Maniyam [[EMAIL PROTECTED]]
Sent: Wednesday, September 05, 2012 4:07 AM
To: [EMAIL PROTECTED]
Subject: current direction in namenode HA

Hello devs,

I am trying to understand the current state / direction of  namenode
HA implementation.

For using shared directory, I see the following options
(from http://www.cloudera.com/blog/2012/03/high-availability-for-the-hadoop-distributed-file-system-hdfs/
  and  https://issues.apache.org/jira/browse/HDFS-3278)

1) rely on external HA filer
2) multiple edit directories
3) book keeper
4) keep edits in HDFS / quorum based

is there going to be an 'official / supported' method, or it is going
to be a configurable choice when setting up a cluster?

thanks
Sujee
http://sujee.net
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB