Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> TaskTracker and DataNodes cannot connect to master node (NoRouteToHostException)


Copy link to this message
-
Re: TaskTracker and DataNodes cannot connect to master node (NoRouteToHostException)
Erik,

>
> I've been unable to resolve this problem on my own so I've decided to ask
> for help. I've pasted the logs I have for the DataNode on of the slave
> nodes. The logs for TaskTracker are essentially the same (i.e. same
> exception causing a shutdown).
>
> Any suggestions or hints as to what could be causing this problem?
>
> STARTUP_MSG: Starting DataNode
> STARTUP_MSG:   host = localhost.localdomain/127.0.0.1
> STARTUP_MSG:   args = []
> STARTUP_MSG:   version = 0.20.2
> STARTUP_MSG:   build > https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r
> 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
> ************************************************************/
> 2010-05-25 13:59:56,504 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 0 time(s).
> 2010-05-25 13:59:57,506 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 1 time(s).
> 2010-05-25 13:59:58,508 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 2 time(s).
> 2010-05-25 13:59:59,510 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 3 time(s).
> 2010-05-25 14:00:00,512 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 4 time(s).
> 2010-05-25 14:00:01,514 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 5 time(s).
> 2010-05-25 14:00:02,516 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 6 time(s).
> 2010-05-25 14:00:03,518 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 7 time(s).
> 2010-05-25 14:00:04,520 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 8 time(s).
> 2010-05-25 14:00:05,522 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: master1/192.168.151.1:54310. Already tried 9 time(s).
> 2010-05-25 14:00:05,525 ERROR
> org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Call
> to master1/192.168.151.1:54310 failed on local exception:
> java.net.NoRouteToHostException: No route to host

Can you connect to 192.168.151.1 from the host running your datanode -
outside of Hadoop ? It seems like it is not reachable. Also, what
platform are you running this on ?
>        at org.apache.hadoop.ipc.Client.wrapException(Client.java:775)
>        at org.apache.hadoop.ipc.Client.call(Client.java:743)
>        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
>        at $Proxy4.getProtocolVersion(Unknown Source)
>        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
>        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:346)
>        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:383)
>        at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:314)
>        at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:291)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:269)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:216)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1283)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1238)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1246)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1368)
> Caused by: java.net.NoRouteToHostException: No route to host
>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>        at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>        at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB