Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - A question of QJM with HDFS federation


Copy link to this message
-
Re: A question of QJM with HDFS federation
Azuryy Yu 2013-04-08, 06:57
Thank you very much, Harsh.

not yet question now.

--Send from my Sony mobile.
On Apr 8, 2013 2:51 PM, "Harsh J" <[EMAIL PROTECTED]> wrote:

> Hi Azurry,
>
> QJM:
>
> Yes, multiple nameservices can share a single QJM set. The QJM
> configuration allows for a journal ID prefix path which you should
> configure to be the nameservice ID. You do not need to change disk
> paths/etc. at all.
>
> For example,
>
> NS1 NNs can have the dfs.namenode.shared.edits.dir configured as:
> qjournal://node1.example.com:8485;node2.example.com:8485;
> node3.example.com:8485/NS1
>
> NS2 NNs can have the dfs.namenode.shared.edits.dir configured as:
> qjournal://node1.example.com:8485;node2.example.com:8485;
> node3.example.com:8485/NS2
>
> Which will separate both of them logically and still make them use the
> same QJM set of nodes.
>
> ZKFC:
>
> Each NN needs its own HDFS ZKFC daemon, but all ZKFCs across multiple
> NSes can share a single ZK cluster. All ZKFC's core-site.xml can have
> the same ha.zookeeper.quorum value since the ZKFC automatically reuses
> the nameservice ID as its parent znode name on the ZK instance, and
> won't collide with another NSs' ZKFCs.
>
> Do post back if there are still some more doubts.
>
> On Mon, Apr 8, 2013 at 10:53 AM, Azuryy Yu <[EMAIL PROTECTED]> wrote:
> > Hi dears,
> > I deployed Hadoopv2 with HA enabled using QJM, so my question is:
> >
> > 1) if we also configured HDFS federation,  such as:
> >
> >     NN1 is active, NN2 is standby
> >     NN3 is active, NN4 is standby
> >
> > they are configured as HDFS federation, then,
> > Can these four NNs using the same Journal nodes and ZKs?
> >
> > if your answer is yes, does that enough just using different
> > dfs.journalnode.edits.dir?
> >
> > such as:
> > NN1 and NN2 configured dfs.journalnode.edits.dir as /data1,
> > NN3 and NN4 configured dfs.journalnode.edits.dir as /data2
> >
> >
> > Thanks.
>
>
>
> --
> Harsh J
>