Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - TaskTracker and DataNodes cannot connect to master node (NoRouteToHostException)


Copy link to this message
-
Re: TaskTracker and DataNodes cannot connect to master node (NoRouteToHostException)
Hemanth Yamijala 2010-05-26, 02:05
Erik,

>
> I've been unable to resolve this problem on my own so I've decided to ask
> for help. I've pasted the logs I have for the DataNode on of the slave
> nodes. The logs for TaskTracker are essentially the same (i.e. same
> exception causing a shutdown).
>
> Any suggestions or hints as to what could be causing this problem?
>
> STARTUP_MSG: Starting DataNode
> STARTUP_MSG:   host = localhost.localdomain/127.0.0.1
> STARTUP_MSG:   args = []
> STARTUP_MSG:   version = 0.20.2
> STARTUP_MSG:   build > https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r
> 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
> ************************************************************/
> 2010-05-25 13:59:56,504 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 0 time(s).
> 2010-05-25 13:59:57,506 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 1 time(s).
> 2010-05-25 13:59:58,508 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 2 time(s).
> 2010-05-25 13:59:59,510 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 3 time(s).
> 2010-05-25 14:00:00,512 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 4 time(s).
> 2010-05-25 14:00:01,514 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 5 time(s).
> 2010-05-25 14:00:02,516 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 6 time(s).
> 2010-05-25 14:00:03,518 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 7 time(s).
> 2010-05-25 14:00:04,520 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 8 time(s).
> 2010-05-25 14:00:05,522 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 9 time(s).
> 2010-05-25 14:00:05,525 ERROR
> org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Call
> to master1/192.168.151.1:54310 failed on local exception:
> java.net.NoRouteToHostException: No route to host

Can you connect to 192.168.151.1 from the host running your datanode -
outside of Hadoop ? It seems like it is not reachable. Also, what
platform are you running this on ?
>        at org.apache.hadoop.ipc.Client.wrapException(Client.java:775)
>        at org.apache.hadoop.ipc.Client.call(Client.java:743)
>        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
>        at $Proxy4.getProtocolVersion(Unknown Source)
>        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
>        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:346)
>        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:383)
>        at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:314)
>        at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:291)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:269)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:216)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1283)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1238)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1246)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1368)
> Caused by: java.net.NoRouteToHostException: No route to host
>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>        at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>        at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)