Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - HELP - Problem in setting up Hadoop - Multi-Node Cluster


Copy link to this message
-
Re: HELP - Problem in setting up Hadoop - Multi-Node Cluster
anil gupta 2012-02-11, 01:31
Hi,

Is your datanode initially able to connect to Namenode? Have you disabled
all the firewalls related services? Do you see any errors at the startup
log of Namenode or Datanode?

I have dealt with similar kind of this problem earlier.
So here is what you can try to do:
First, test that ssh is working fine to ensure network is working fine. Ssh
into slave from master and ssh into master from the same slave. Leave the
ssh session open for as long as u can.
In my case when I did the above experiment the ssh session was dropping so
I got to know that it's a network related problem. It has got nothing to do
with Hadoop.

This post might be helpful for u:
https://groups.google.com/a/cloudera.org/group/cdh-user/browse_thread/thread/4165f39d8b0bbc56

Best Regards,
Anil

On Fri, Feb 10, 2012 at 1:43 AM, Guruprasad B <[EMAIL PROTECTED]>wrote:

> Dear Robin,
>
> Yes, it is possible.
>
> Regards,
> Guru
>
> On Fri, Feb 10, 2012 at 1:23 PM, Robin Mueller-Bady <
> [EMAIL PROTECTED]> wrote:
>
>>  Dear Guruprasad,
>>
>> is it possible to ping both machines with their hostnames ? (ping master
>> / ping slave) ?
>>
>> Regards,
>>
>> Robin
>>
>>
>> On 10.02.2012 07:46, Guruprasad B wrote:
>>
>> Dear Robin,
>>
>> Thanks for your valuable time and response. please find the attached
>> namenode logs and configurations files.
>>
>> I am using 2 ubuntu boxes.One as master & slave and other as slave.
>> below given is the environment set-up in both the machines.
>>
>> :
>> Hadoop : hadoop_0.20.2
>> Linux: Ubuntu Linux 10.10(master) and Ubuntu Linux 11.04(Slave)
>> Java: java-7-oracle
>> JAVA_HOME and HADOOP_HOME configuration is done in .bashrc file.
>>
>> Both the machines are in LAN and able to ping each other. IP address's of both the machines are configured in /etc/hosts.
>>
>> I do have SSH access to both master and slave as well.
>>
>> please let me know if you need any other information.
>>
>> Thanks in advance.
>>
>> Regards,
>> Guruprasad
>>
>>
>>
>>
>>
>>
>>
>>  On Thu, Feb 9, 2012 at 1:06 AM, Robin Mueller-Bady <
>> [EMAIL PROTECTED]> wrote:
>>
>>>  Dear Guruprasad,
>>>
>>> it would be very helpful to provide details from your configuration
>>> files as well as more details on your setup.
>>> It seems to be that the connection from slave to master cannot be
>>> established ("Connection reset by peer").
>>> Do you use a virtual environment, physical master/slaves or all on one
>>> machine ?
>>> Please paste also the output of "kingul2" namenode logs.
>>>
>>> Regards,
>>>
>>> Robin
>>>
>>>
>>> On 02/08/12 13:06, Guruprasad B wrote:
>>>
>>> Hi,
>>>
>>> I am Guruprasad from Bangalore (India). I need help in setting up hadoop
>>> platform. I am very much new to Hadoop Platform.
>>>
>>> I am following the below given articles and I was able to set up
>>> "Single-Node Cluster"
>>> http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/#what-we-want-to-do
>>>
>>> Now I am trying to set up " <http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/#what-we-want-to-doNowIamtryingtosetup>Multi-Node Cluster" by following the below given
>>> article.http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/
>>>
>>> Below given is my setup:
>>> Hadoop : hadoop_0.20.2
>>> Linux: Ubuntu Linux 10.10
>>> Java: java-7-oracle
>>>
>>>
>>> I have successfully reached till the topic "Starting the multi-node
>>> cluster" in the above given article.
>>> When I start the HDFS/MapReduce daemons it is getting started and going
>>> down immediately both in master & slave as well,
>>> please have a look at the below logs,
>>>
>>> hduser@kinigul2:/usr/local/hadoop$ bin/start-dfs.sh
>>> starting namenode, logging to
>>> /usr/local/hadoop/bin/../logs/hadoop-hduser-namenode-kinigul2.out
>>> master: starting datanode, logging to
>>> /usr/local/hadoop/bin/../logs/hadoop-hduser-datanode-kinigul2.out
>>> slave: starting datanode, logging to
>>> /usr/local/hadoop/bin/../logs/hadoop-hduser-datanode-guruL.out
Thanks & Regards,
Anil Gupta