Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> question about ZKFC daemon


+
ESGLinux 2012-12-27, 12:03
+
rahul p 2012-12-28, 07:00
+
Harsh J 2012-12-27, 16:34
+
ESGLinux 2012-12-28, 09:25
Copy link to this message
-
Re: question about ZKFC daemon
You need the following:

- active namenode + zkfc
- standby namenode + zkfc
- pool of journal nodes (odd number, 3 or more)
- pool of zookeeper nodes (odd number, 3 or more)

As the journal nodes hold the namesystem transactions they should not be
co-located with the namenodes in case of failure.  I distribute the journal
and zookeeper nodes across the hosts running datanodes or as Harsh says you
could co-locate them on dedicated hosts.

ZKFC does not monitor the JobTracker.

Regards,
Craig
On Dec 28, 2012 9:25 AM, "ESGLinux" <[EMAIL PROTECTED]> wrote:

> Hi,
>
> well, If I have understand you I can configure my NN HA cluster this way:
>
> - Active NameNode + 1 ZKFC daemon + Journal Node
> - Standby NameNode + 1 ZKFC daemon + Journal Node
> - JobTracker node + 1 ZKFC daemon + Journal Node,
>
> Is this right?
>
> Thanks in advance,
>
> ESGLinux,
>
> 2012/12/27 Harsh J <[EMAIL PROTECTED]>
>
>> Hi,
>>
>> There are two different things here: Automatic Failover and Quorum
>> Journal Manager. The former, used via a ZooKeeper Failover Controller,
>> is to manage failovers automatically (based on health checks of NNs).
>> The latter, used via a set of Journal Nodes, is a medium of shared
>> storage for namesystem transactions that helps enable HA.
>>
>> In a typical deployment, you want 3 or more (odd) JournalNodes for
>> reliable HA, preferably on nodes of their own if possible (like you
>> would for typical ZooKeepers, and you may co-locate with those as
>> well) and one ZKFC for each NameNode (connected to the same ZK
>> quorum).
>>
>> On Thu, Dec 27, 2012 at 5:33 PM, ESGLinux <[EMAIL PROTECTED]> wrote:
>> > Hi all,
>> >
>> > I have a doubt about how to deploy the Zookeeper in a NN HA  cluster,
>> >
>> > As far as I know, I need at least three nodes to run three ZooKeeper
>> > FailOver Controller (ZKFC). I plan to put these 3 daemons this way:
>> >
>> > - Active NameNode + 1 ZKFC daemon
>> > - Standby NameNode + 1 ZKFC daemon
>> > - JobTracker node + 1 ZKFC daemon, (is this right?)
>> >
>> > so the quorum is formed with these three nodes. The nodes that runs a
>> > namenode are right because the ZKFC monitors it, but what does the third
>> > daemon?
>> >
>> > as I read from this url:
>> >
>> https://ccp.cloudera.com/display/CDH4DOC/Software+Configuration+for+Quorum-based+Storage#SoftwareConfigurationforQuorum-basedStorage-AutomaticFailoverConfiguration
>> >
>> > this daemons are only related with NameNodes, (Health monitoring - the
>> ZKFC
>> > pings its local NameNode on a periodic basis with a health-check
>> command.)
>> > so what does the third ZKFC? I used the jobtracker node but I could use
>> > another node without any daemon on it...
>> >
>> > Thanks in advance,
>> >
>> > ESGLInux,
>> >
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
>
+
ESGLinux 2012-12-28, 10:37
+
Craig Munro 2012-12-28, 10:51
+
ESGLinux 2012-12-28, 11:02
+
Colin McCabe 2013-01-14, 19:49
+
Colin McCabe 2013-01-14, 20:34
+
ESGLinux 2013-01-15, 09:53
+
Harsh J 2013-01-15, 09:55
+
ESGLinux 2013-01-15, 10:08
+
Harsh J 2013-01-15, 10:11
+
ESGLinux 2013-01-15, 10:17