Hi,

I am trying to setup 0.23 on a cluster and am stuck with errors while
starting the NodeManager. The slaves file is proper and I am able to do a
password-less ssh from the master to the slaves. The ResourceManager also
starts properly.

On running the below command from the master node.

>> bin/yarn-daemons.sh start nodemanager

I get the following error from the yarn script

>> praveen-laptop: No HADOOP_CONF_DIR set.
>> praveen-laptop: Please specify it either in yarn-env.sh or in the
environment.

The following is set in .bashrc for all the slaves

>> export HADOOP_DEV_HOME=/home/praveensripati/Installations/hadoop-0.23.0
>> export HADOOP_CONF_DIR=$HADOOP_DEV_HOME/conf

For some reason the HADOOP_CONF_DIR and other variables are not visible in
the yarn-daemon.sh (could verify if from the echo statements also) which
calls the yarn script.

I am able to run the below command successfully from the terminal, but
the yarn-daemons.sh
(plural) command exits with the above error.

>> ssh praveen-laptop cd
/home/praveensripati/Installations/hadoop-0.23.0/bin/.. ;
/home/praveensripati/Installations/hadoop-0.23.0/bin/yarn-daemon.sh
--config /home/praveensripati/Installations/hadoop-0.23.0/conf start
nodemanager

I looked at the code and could not figure it out. Did I miss anything or is
there a bug in the code?

Regards,
Praveen

Are you setting HADOOP_CONF_DIR in hadoop-env.sh? Do you have YARN_HOME set in hadoop-env.sh?  Is your yarn-env.sh just the standard one from ./hadoop-mapreduce-project/hadoop-yarn/conf/yarn-env.sh?

Tom
On 1/9/12 6:16 AM, "Praveen Sripati" <[EMAIL PROTECTED]> wrote:

Hi,

I am trying to setup 0.23 on a cluster and am stuck with errors while starting the NodeManager. The slaves file is proper and I am able to do a password-less ssh from the master to the slaves. The ResourceManager also starts properly.

On running the below command from the master node.

>> bin/yarn-daemons.sh start nodemanager

I get the following error from the yarn script

>> praveen-laptop: No HADOOP_CONF_DIR set.
>> praveen-laptop: Please specify it either in yarn-env.sh or in the environment.

The following is set in .bashrc for all the slaves

>> export HADOOP_DEV_HOME=/home/praveensripati/Installations/hadoop-0.23.0
>> export HADOOP_CONF_DIR=$HADOOP_DEV_HOME/conf

For some reason the HADOOP_CONF_DIR and other variables are not visible in the yarn-daemon.sh (could verify if from the echo statements also) which calls the yarn script.

I am able to run the below command successfully from the terminal, but the yarn-daemons.sh (plural) command exits with the above error.

>> ssh praveen-laptop cd /home/praveensripati/Installations/hadoop-0.23.0/bin/.. ; /home/praveensripati/Installations/hadoop-0.23.0/bin/yarn-daemon.sh --config /home/praveensripati/Installations/hadoop-0.23.0/conf start nodemanager

I looked at the code and could not figure it out. Did I miss anything or is there a bug in the code?

Regards,
Praveen

Not sure why the the properties in the .bashrc are not being picked by the
scripts. Why is it? I am using Ubuntu 11.10. I had to export some variables
in the `libexec/hadoop-config.sh` and `conf/yarn-env.sh` to get 0.23 work
in a cluster.

Also, I had to add `yarn.resourcemanager.resource-tracker.address`,
`yarn.resourcemanager.scheduler.address` in the yarn-site.xml for the NM to
know the location of the RM.

Regards,
Praveen

On Mon, Jan 9, 2012 at 8:15 PM, Thomas Graves <[EMAIL PROTECTED]> wrote:

>  Are you setting HADOOP_CONF_DIR in hadoop-env.sh? Do you have YARN_HOME
> set in hadoop-env.sh?  Is your yarn-env.sh just the standard one from
> ./hadoop-mapreduce-project/hadoop-yarn/conf/yarn-env.sh?
>
> Tom
>
>
>
> On 1/9/12 6:16 AM, "Praveen Sripati" <[EMAIL PROTECTED]> wrote:
>
> Hi,
>
> I am trying to setup 0.23 on a cluster and am stuck with errors while
> starting the NodeManager. The slaves file is proper and I am able to do a
> password-less ssh from the master to the slaves. The ResourceManager also
> starts properly.
>
> On running the below command from the master node.
>
> >> bin/yarn-daemons.sh start nodemanager
>
> I get the following error from the yarn script
>
> >> praveen-laptop: No HADOOP_CONF_DIR set.
> >> praveen-laptop: Please specify it either in yarn-env.sh or in the
> environment.
>
> The following is set in .bashrc for all the slaves
>
> >> export HADOOP_DEV_HOME=/home/praveensripati/Installations/hadoop-0.23.0
> >> export HADOOP_CONF_DIR=$HADOOP_DEV_HOME/conf
>
> For some reason the HADOOP_CONF_DIR and other variables are not visible in
> the yarn-daemon.sh (could verify if from the echo statements also) which
> calls the yarn script.
>
> I am able to run the below command successfully from the terminal, but the
> yarn-daemons.sh (plural) command exits with the above error.
>
> >> ssh praveen-laptop cd
> /home/praveensripati/Installations/hadoop-0.23.0/bin/.. ;
> /home/praveensripati/Installations/hadoop-0.23.0/bin/yarn-daemon.sh
> --config /home/praveensripati/Installations/hadoop-0.23.0/conf start
> nodemanager
>
> I looked at the code and could not figure it out. Did I miss anything or
> is there a bug in the code?
>
> Regards,
> Praveen
>
>