-Re: Hadoop Cluster Configuration
Hemanth Yamijala 2010-07-28, 08:02
> While setting hadoop cluster, does configuration files (conf/core-site.xml,
> conf/mapred-site.xml,conf/hdfs-site.xml) in every node(name node and data
> nodes) needs to be configured in the same manner?
This is a complicated question to answer :-). There are certain
configuration variables that need to be defined to be the same between
the master and the slaves and some that don't need to be. Pre Hadoop
0.21, there is no easy way other than documentation for the variables
(hopefully) to determine if this is the case or not. I think in Hadoop
0.21 and since, we have tried to adopt a convention to include the
daemon name to specify which variables are used by which daemons. And
those that are cluster-wide, that need to be the same throughout all
the nodes will have something like 'cluster' in the name.
Your best bet in any case is possibly to sift through the
documentation of the variables you are interested in. Or else to post
a query here.
> How does configuration of name node differs from configuration of data
Not sure about this one.