Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> federated cluster

Visioner Sadak 2012-11-05, 12:20
Copy link to this message
Re: federated cluster
Hi Visioner,

If you aren't using HA, you will need two SNNs - one that checkpoints
IP1 and the other for IP2. If you lack more hardware, you may probably
reside them on each other (i.e. SNN of IP2 on IP1, etc.) although this
wouldn't work so well after a while (when metadata at each NN has
grown much), but for starters this is good enough I guess.

The ResourceManager/NodeManager are YARN daemons useful for processing
data. If your need is just storage, you are correct in not needing
them deployed.

You are also correct on the federated NNs part. If a file resides in
the NS of IP1, you need to access IP1 to fetch its details.

On Mon, Nov 5, 2012 at 5:50 PM, Visioner Sadak <[EMAIL PROTECTED]> wrote:
> Hello friends i have configured a 3 node federated hadoop cluster with 2
> namenodes and 3 datanodes
> 2 namenodes -- ip1,ip2
> 3 datanodes   --ip1,ip2,ip3
> wht nodes shud i make as my secondary namenode,is it mandatory to have
> resource manager and node manager,i m using hadoop just for storing purpose
> so i dunt need them
> some data goes into namenode 1 which is ip1, some goes in to namenode 2
> which is ip2
> i hope i cant access the data of nn1 using nn2 ip or any nn ip in my cluster
> right,
> i can access data only using the nn1 ip for nn1 data ....correct me if i am
> wrong...

Harsh J
Visioner Sadak 2012-11-05, 16:10