Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Question about namenode HA


Copy link to this message
-
Re: Question about namenode HA
Steve Loughran 2012-10-15, 10:16
On 15 October 2012 04:01, 谢良 <[EMAIL PROTECTED]> wrote:

> Hi Todd and other HA experts,
>
> I've two question:
>
> 1) why the zkfc is a seperate process, i mean, what's the primary design
> consideration that we didn't integrate zkfc features into namenode self ?
>

I don't know the answer to that, except I do know that you are usually best
a monitoring the state of a process externally, as liveness is defined as
"doing useful work to external callers".

>
> 2) If i deploy CDH4.1(included QJM feature),  since QJM can do fencing
> writer,  so can i just config like this safely ?
>         <name>dfs.ha.fencing.methods</name>
>         <value>shell(/bin/true)</value>
>
>
I would never encourage anyone to skip fencing, as its the only way to
reliably ensure the far end that appears to be offline really is offline.
Fencing PSUs aren't that expensive; talking to an ILO management port on
the server via a separate 100 MBit/s management network a less satisfactory
option.

The risk here is that both NN's can talk to the DNs -if they emit
conflicting operations, you are in trouble. That situation could
theoretically arise if both NNs were visible to the DNs, but not to each
other. ZK quorum logic should recognise and handle this, and I have faith
in the mathematicians behind the proof of ZK's operations -but I'd
encourage you to spend the extra amount for proper fencing -the extra
hardware is minimal compared to the value of data in a cluster.

-steve