Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> question about ZKFC daemon


+
ESGLinux 2012-12-27, 12:03
+
rahul p 2012-12-28, 07:00
+
Harsh J 2012-12-27, 16:34
+
ESGLinux 2012-12-28, 09:25
Copy link to this message
-
Re: question about ZKFC daemon
You need the following:

- active namenode + zkfc
- standby namenode + zkfc
- pool of journal nodes (odd number, 3 or more)
- pool of zookeeper nodes (odd number, 3 or more)

As the journal nodes hold the namesystem transactions they should not be
co-located with the namenodes in case of failure.  I distribute the journal
and zookeeper nodes across the hosts running datanodes or as Harsh says you
could co-locate them on dedicated hosts.

ZKFC does not monitor the JobTracker.

Regards,
Craig
On Dec 28, 2012 9:25 AM, "ESGLinux" <[EMAIL PROTECTED]> wrote:

> Hi,
>
> well, If I have understand you I can configure my NN HA cluster this way:
>
> - Active NameNode + 1 ZKFC daemon + Journal Node
> - Standby NameNode + 1 ZKFC daemon + Journal Node
> - JobTracker node + 1 ZKFC daemon + Journal Node,
>
> Is this right?
>
> Thanks in advance,
>
> ESGLinux,
>
> 2012/12/27 Harsh J <[EMAIL PROTECTED]>
>
>> Hi,
>>
>> There are two different things here: Automatic Failover and Quorum
>> Journal Manager. The former, used via a ZooKeeper Failover Controller,
>> is to manage failovers automatically (based on health checks of NNs).
>> The latter, used via a set of Journal Nodes, is a medium of shared
>> storage for namesystem transactions that helps enable HA.
>>
>> In a typical deployment, you want 3 or more (odd) JournalNodes for
>> reliable HA, preferably on nodes of their own if possible (like you
>> would for typical ZooKeepers, and you may co-locate with those as
>> well) and one ZKFC for each NameNode (connected to the same ZK
>> quorum).
>>
>> On Thu, Dec 27, 2012 at 5:33 PM, ESGLinux <[EMAIL PROTECTED]> wrote:
>> > Hi all,
>> >
>> > I have a doubt about how to deploy the Zookeeper in a NN HA  cluster,
>> >
>> > As far as I know, I need at least three nodes to run three ZooKeeper
>> > FailOver Controller (ZKFC). I plan to put these 3 daemons this way:
>> >
>> > - Active NameNode + 1 ZKFC daemon
>> > - Standby NameNode + 1 ZKFC daemon
>> > - JobTracker node + 1 ZKFC daemon, (is this right?)
>> >
>> > so the quorum is formed with these three nodes. The nodes that runs a
>> > namenode are right because the ZKFC monitors it, but what does the third
>> > daemon?
>> >
>> > as I read from this url:
>> >
>> https://ccp.cloudera.com/display/CDH4DOC/Software+Configuration+for+Quorum-based+Storage#SoftwareConfigurationforQuorum-basedStorage-AutomaticFailoverConfiguration
>> >
>> > this daemons are only related with NameNodes, (Health monitoring - the
>> ZKFC
>> > pings its local NameNode on a periodic basis with a health-check
>> command.)
>> > so what does the third ZKFC? I used the jobtracker node but I could use
>> > another node without any daemon on it...
>> >
>> > Thanks in advance,
>> >
>> > ESGLInux,
>> >
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
>
+
ESGLinux 2012-12-28, 10:37
+
Craig Munro 2012-12-28, 10:51
+
ESGLinux 2012-12-28, 11:02
+
Colin McCabe 2013-01-14, 19:49
+
Colin McCabe 2013-01-14, 20:34
+
ESGLinux 2013-01-15, 09:53
+
Harsh J 2013-01-15, 09:55
+
ESGLinux 2013-01-15, 10:08
+
Harsh J 2013-01-15, 10:11
+
ESGLinux 2013-01-15, 10:17
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB