Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Problem running PI example in Hadoop 2.0.0


Copy link to this message
-
Re: Problem running PI example in Hadoop 2.0.0
Looks like misconfiguration

Make sure that your master and slaves connect to externally available IP

Thanks
Serge

From: Arjun Reddy <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Wed, 8 Aug 2012 15:39:30 -0600
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Problem running PI example in Hadoop 2.0.0
I am trying to setup a small cluster using hadoop 2.0.0 and using PI example to validate the setup.  When I have 1 master and 1 slave the example works fine.  I am getting exceptions with the PI example when additional slave nodes are added to the cluster.  The syslogs for failed  tasks are as follows.  Any ideas why this is happening.

2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2012-08-08 15:41:19,973 WARN [main] org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty
2012-08-08 15:41:20,142 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.namenode.name.dir;  Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.datanode.data.dir;  Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 0 time(s).
2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 1 time(s).
2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 2 time(s).
2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 3 time(s).
2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 4 time(s).
2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 5 time(s).
2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 6 time(s).
2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 7 time(s).
2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 8 time(s).
2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 9 time(s).
2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.net.ConnectException: Call From node2/127.0.1.1<http://127.0.1.1> to node2:45965 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
        at org.apache.hadoop.ipc.Client.call(Client.java:1165)
        at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
        at $Proxy6.getTask(Unknown Source)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
        at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
        at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
        at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
        at org.apache.hadoop.ipc.Client.call(Client.java:1141)
        ... 3 more

2012-08-08 15:41:30,493 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...
2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.
2012-08-
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB