|
Arjun Reddy
2012-08-08, 21:39
Harit Himanshu
2012-08-08, 21:47
Serge Blazhiyevskyy
2012-08-08, 21:49
Mohammad Tariq
2012-08-08, 21:50
Steve Loughran
2012-08-09, 16:47
Arjun Reddy
2012-08-09, 17:31
Steve Loughran
2012-08-09, 20:15
Arjun Reddy
2012-08-10, 17:54
Steve Loughran
2012-08-10, 18:23
anil gupta
2012-08-11, 04:50
|
-
Problem running PI example in Hadoop 2.0.0Arjun Reddy 2012-08-08, 21:39
I am trying to setup a small cluster using hadoop 2.0.0 and using PI
example to validate the setup. When I have 1 master and 1 slave the example works fine. I am getting exceptions with the PI example when additional slave nodes are added to the cluster. The syslogs for failed tasks are as follows. Any ideas why this is happening. 2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. 2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. 2012-08-08 15:41:19,973 WARN [main] org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty 2012-08-08 15:41:20,142 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started 2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.namenode.name.dir; Ignoring. 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.datanode.data.dir; Ignoring. 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. 2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now. 2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 0 time(s). 2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 1 time(s). 2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 2 time(s). 2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 3 time(s). 2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 4 time(s). 2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 5 time(s). 2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 6 time(s). 2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 7 time(s). 2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 8 time(s). 2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 9 time(s). 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.net.ConnectException: Call From node2/ 127.0.1.1 to node2:45965 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727) at org.apache.hadoop.ipc.Client.call(Client.java:1165) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224) at $Proxy6.getTask(Unknown Source) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123) Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489) at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566) at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271) at org.apache.hadoop.ipc.Client.call(Client.java:1141) ... 3 more 2012-08-08 15:41:30,493 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system... 2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped. 2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.
-
Re: Problem running PI example in Hadoop 2.0.0Harit Himanshu 2012-08-08, 21:47
- when adding slaves are you adding slaves in conf/slaves file? (I assume yes)
- are these slaves mapped in your /etc/hosts file? - in your slaves con/masters do you have entry for master host? - in your slaves /etc/hosts does master resolves to the correct IP? On Aug 8, 2012, at 2:39 PM, Arjun Reddy wrote: > I am trying to setup a small cluster using hadoop 2.0.0 and using PI example to validate the setup. When I have 1 master and 1 slave the example works fine. I am getting exceptions with the PI example when additional slave nodes are added to the cluster. The syslogs for failed tasks are as follows. Any ideas why this is happening. > > 2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. > 2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. > 2012-08-08 15:41:19,973 WARN [main] org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty > 2012-08-08 15:41:20,142 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties > 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). > 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started > 2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.namenode.name.dir; Ignoring. > 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. > 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.datanode.data.dir; Ignoring. > 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. > 2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now. > 2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 0 time(s). > 2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 1 time(s). > 2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 2 time(s). > 2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 3 time(s). > 2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 4 time(s). > 2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 5 time(s). > 2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 6 time(s). > 2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 7 time(s). > 2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 8 time(s). > 2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 9 time(s). > 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.net.ConnectException: Call From node2/127.0.1.1 to node2:45965 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
-
Re: Problem running PI example in Hadoop 2.0.0Serge Blazhiyevskyy 2012-08-08, 21:49
Looks like misconfiguration
Make sure that your master and slaves connect to externally available IP Thanks Serge From: Arjun Reddy <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> Date: Wed, 8 Aug 2012 15:39:30 -0600 To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> Subject: Problem running PI example in Hadoop 2.0.0 I am trying to setup a small cluster using hadoop 2.0.0 and using PI example to validate the setup. When I have 1 master and 1 slave the example works fine. I am getting exceptions with the PI example when additional slave nodes are added to the cluster. The syslogs for failed tasks are as follows. Any ideas why this is happening. 2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. 2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. 2012-08-08 15:41:19,973 WARN [main] org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty 2012-08-08 15:41:20,142 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started 2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.namenode.name.dir; Ignoring. 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.datanode.data.dir; Ignoring. 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. 2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now. 2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 0 time(s). 2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 1 time(s). 2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 2 time(s). 2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 3 time(s). 2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 4 time(s). 2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 5 time(s). 2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 6 time(s). 2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 7 time(s). 2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 8 time(s). 2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 9 time(s). 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.net.ConnectException: Call From node2/127.0.1.1<http://127.0.1.1> to node2:45965 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727) at org.apache.hadoop.ipc.Client.call(Client.java:1165) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224) at $Proxy6.getTask(Unknown Source) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123) Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489) at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566) at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271) at org.apache.hadoop.ipc.Client.call(Client.java:1141) ... 3 more 2012-08-08 15:41:30,493 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system... 2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped. 2012-08-
-
Re: Problem running PI example in Hadoop 2.0.0Mohammad Tariq 2012-08-08, 21:50
Hello Arjun,
Seems 'loopback' prob to me..Make a check on what Harit has pointed. Regards, Mohammad Tariq On Thu, Aug 9, 2012 at 3:17 AM, Harit Himanshu <[EMAIL PROTECTED]> wrote: > - when adding slaves are you adding slaves in conf/slaves file? (I assume > yes) > - are these slaves mapped in your /etc/hosts file? > - in your slaves con/masters do you have entry for master host? > - in your slaves /etc/hosts does master resolves to the correct IP? > > > On Aug 8, 2012, at 2:39 PM, Arjun Reddy wrote: > > I am trying to setup a small cluster using hadoop 2.0.0 and using PI example > to validate the setup. When I have 1 master and 1 slave the example works > fine. I am getting exceptions with the PI example when additional slave > nodes are added to the cluster. The syslogs for failed tasks are as > follows. Any ideas why this is happening. > > 2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration: > job.xml:an attempt to override final parameter: > mapreduce.job.end-notification.max.retry.interval; Ignoring. > 2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration: > job.xml:an attempt to override final parameter: > mapreduce.job.end-notification.max.attempts; Ignoring. > 2012-08-08 15:41:19,973 WARN [main] > org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5 > configuration not found, setting default realm to empty > 2012-08-08 15:41:20,142 INFO [main] > org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from > hadoop-metrics2.properties > 2012-08-08 15:41:20,221 INFO [main] > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period > at 10 second(s). > 2012-08-08 15:41:20,221 INFO [main] > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system > started > 2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration: > job.xml:an attempt to override final parameter: dfs.namenode.name.dir; > Ignoring. > 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: > job.xml:an attempt to override final parameter: > mapreduce.job.end-notification.max.retry.interval; Ignoring. > 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: > job.xml:an attempt to override final parameter: dfs.datanode.data.dir; > Ignoring. > 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: > job.xml:an attempt to override final parameter: > mapreduce.job.end-notification.max.attempts; Ignoring. > 2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild: > Sleeping for 0ms before retrying again. Got null now. > 2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying > connect to server: node2/127.0.1.1:45965. Already tried 0 time(s). > 2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying > connect to server: node2/127.0.1.1:45965. Already tried 1 time(s). > 2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying > connect to server: node2/127.0.1.1:45965. Already tried 2 time(s). > 2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying > connect to server: node2/127.0.1.1:45965. Already tried 3 time(s). > 2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying > connect to server: node2/127.0.1.1:45965. Already tried 4 time(s). > 2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying > connect to server: node2/127.0.1.1:45965. Already tried 5 time(s). > 2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying > connect to server: node2/127.0.1.1:45965. Already tried 6 time(s). > 2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying > connect to server: node2/127.0.1.1:45965. Already tried 7 time(s). > 2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying > connect to server: node2/127.0.1.1:45965. Already tried 8 time(s). > 2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying
-
Re: Problem running PI example in Hadoop 2.0.0Steve Loughran 2012-08-09, 16:47
On 8 August 2012 14:39, Arjun Reddy <[EMAIL PROTECTED]> wrote:
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI > example to validate the setup. When I have 1 master and 1 slave the > example works fine. I am getting exceptions with the PI example when > additional slave nodes are added to the cluster. The syslogs for failed > tasks are as follows. Any ideas why this is happening. > > > . > 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: > Exception running child : java.net.ConnectException: Call From node2/ > 127.0.1.1 to node2:45965 failed on connection exception: > java.net.ConnectException: Connection refused; For more details see: > http://wiki.apache.org/hadoop/ConnectionRefused > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727) > at org.apache.hadoop.ipc.Client.call(Client.java:1165) > at > org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224) > at $Proxy6.getTask(Unknown Source) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123) > did you follow the link in this message? If so, did you follow all the actions?
-
Re: Problem running PI example in Hadoop 2.0.0Arjun Reddy 2012-08-09, 17:31
I did go through the 6 steps listed at the link and telnet to port 45965 is
failing. Netstat -a shows nothing is listing on port 45965. We can run hadoop 1.0.3 version without any issues on this cluster and I am new to 2.0.0 version. I followed the document http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to setup the config files. Is there any other good document that describes how to set up version 2.0.0. Thanks Arjun On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <[EMAIL PROTECTED]>wrote: > > > On 8 August 2012 14:39, Arjun Reddy <[EMAIL PROTECTED]> wrote: > >> I am trying to setup a small cluster using hadoop 2.0.0 and using PI >> example to validate the setup. When I have 1 master and 1 slave the >> example works fine. I am getting exceptions with the PI example when >> additional slave nodes are added to the cluster. The syslogs for failed >> tasks are as follows. Any ideas why this is happening. >> >> >> . >> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: >> Exception running child : java.net.ConnectException: Call From node2/ >> 127.0.1.1 to node2:45965 failed on connection exception: >> java.net.ConnectException: Connection refused; For more details see: >> http://wiki.apache.org/hadoop/ConnectionRefused >> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727) >> at org.apache.hadoop.ipc.Client.call(Client.java:1165) >> at >> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224) >> at $Proxy6.getTask(Unknown Source) >> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123) >> >> did you follow the link in this message? > > If so, did you follow all the actions? > >
-
Re: Problem running PI example in Hadoop 2.0.0Steve Loughran 2012-08-09, 20:15
On 9 August 2012 10:31, Arjun Reddy <[EMAIL PROTECTED]> wrote:
> I did go through the 6 steps listed at the link and telnet to port > 45965 is failing. Netstat -a shows nothing is listing on port 45965. > That's a high number - I wonder if it's something transient that didn't start. I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts to 127.0.0.1 to see if that helps; it could just be some listening on a different port > > We can run hadoop 1.0.3 version without any issues on this cluster and I > am new to 2.0.0 version. I followed the document > http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to > setup the config files. Is there any other good document that describes > how to set up version 2.0.0. > > Thanks > Arjun > > On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <[EMAIL PROTECTED]>wrote: > >> >> >> On 8 August 2012 14:39, Arjun Reddy <[EMAIL PROTECTED]> wrote: >> >>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI >>> example to validate the setup. When I have 1 master and 1 slave the >>> example works fine. I am getting exceptions with the PI example when >>> additional slave nodes are added to the cluster. The syslogs for failed >>> tasks are as follows. Any ideas why this is happening. >>> >>> >>> . >>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: >>> Exception running child : java.net.ConnectException: Call From node2/ >>> 127.0.1.1 to node2:45965 failed on connection exception: >>> java.net.ConnectException: Connection refused; For more details see: >>> http://wiki.apache.org/hadoop/ConnectionRefused >>> at >>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727) >>> at org.apache.hadoop.ipc.Client.call(Client.java:1165) >>> at >>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224) >>> at $Proxy6.getTask(Unknown Source) >>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123) >>> >>> did you follow the link in this message? >> >> If so, did you follow all the actions? >> >> > -- Steve Loughran Hortonworks Inc [EMAIL PROTECTED] skype: steve_loughran tel: +1 408 400 3721 <http://hortonworks.com/download/>
-
Re: Problem running PI example in Hadoop 2.0.0Arjun Reddy 2012-08-10, 17:54
I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the
cluster is working fine now. Thank you very much for your help. On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <[EMAIL PROTECTED]>wrote: > > > On 9 August 2012 10:31, Arjun Reddy <[EMAIL PROTECTED]> wrote: > >> I did go through the 6 steps listed at the link and telnet to port >> 45965 is failing. Netstat -a shows nothing is listing on port 45965. >> > > > That's a high number - I wonder if it's something transient that didn't > start. > > I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts > to 127.0.0.1 to see if that helps; it could just be some listening on a > different port > > >> >> We can run hadoop 1.0.3 version without any issues on this cluster and I >> am new to 2.0.0 version. I followed the document >> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to >> setup the config files. Is there any other good document that describes >> how to set up version 2.0.0. >> >> Thanks >> Arjun >> >> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <[EMAIL PROTECTED]>wrote: >> >>> >>> >>> On 8 August 2012 14:39, Arjun Reddy <[EMAIL PROTECTED]> wrote: >>> >>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI >>>> example to validate the setup. When I have 1 master and 1 slave the >>>> example works fine. I am getting exceptions with the PI example when >>>> additional slave nodes are added to the cluster. The syslogs for failed >>>> tasks are as follows. Any ideas why this is happening. >>>> >>>> >>>> . >>>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: >>>> Exception running child : java.net.ConnectException: Call From node2/ >>>> 127.0.1.1 to node2:45965 failed on connection exception: >>>> java.net.ConnectException: Connection refused; For more details see: >>>> http://wiki.apache.org/hadoop/ConnectionRefused >>>> at >>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727) >>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165) >>>> at >>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224) >>>> at $Proxy6.getTask(Unknown Source) >>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123) >>>> >>>> did you follow the link in this message? >>> >>> If so, did you follow all the actions? >>> >>> >> > > > -- > Steve Loughran > Hortonworks Inc > [EMAIL PROTECTED] > skype: steve_loughran > tel: +1 408 400 3721 > > <http://hortonworks.com/download/> > > >
-
Re: Problem running PI example in Hadoop 2.0.0Steve Loughran 2012-08-10, 18:23
that's ubuntu for you. I'll update the wiki
On 10 August 2012 10:54, Arjun Reddy <[EMAIL PROTECTED]> wrote: > I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the > cluster is working fine now. Thank you very much for your help. > > > On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <[EMAIL PROTECTED]>wrote: > >> >> >> On 9 August 2012 10:31, Arjun Reddy <[EMAIL PROTECTED]> wrote: >> >>> I did go through the 6 steps listed at the link and telnet to port >>> 45965 is failing. Netstat -a shows nothing is listing on port 45965. >>> >> >> >> That's a high number - I wonder if it's something transient that didn't >> start. >> >> I worry about the 127.0.1.1 number in the hostname -try editing >> /etc/hosts to 127.0.0.1 to see if that helps; it could just be some >> listening on a different port >> >> >>> >>> We can run hadoop 1.0.3 version without any issues on this cluster and I >>> am new to 2.0.0 version. I followed the document >>> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to >>> setup the config files. Is there any other good document that describes >>> how to set up version 2.0.0. >>> >>> Thanks >>> Arjun >>> >>> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <[EMAIL PROTECTED] >>> > wrote: >>> >>>> >>>> >>>> On 8 August 2012 14:39, Arjun Reddy <[EMAIL PROTECTED]> wrote: >>>> >>>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI >>>>> example to validate the setup. When I have 1 master and 1 slave the >>>>> example works fine. I am getting exceptions with the PI example when >>>>> additional slave nodes are added to the cluster. The syslogs for failed >>>>> tasks are as follows. Any ideas why this is happening. >>>>> >>>>> >>>>> . >>>>> 2012-08-08 15:41:30,492 WARN [main] >>>>> org.apache.hadoop.mapred.YarnChild: Exception running child : >>>>> java.net.ConnectException: Call From node2/127.0.1.1 to node2:45965 >>>>> failed on connection exception: java.net.ConnectException: Connection >>>>> refused; For more details see: >>>>> http://wiki.apache.org/hadoop/ConnectionRefused >>>>> at >>>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727) >>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165) >>>>> at >>>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224) >>>>> at $Proxy6.getTask(Unknown Source) >>>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123) >>>>> >>>>> did you follow the link in this message? >>>> >>>> If so, did you follow all the actions? >>>> >>>> >>> >> >> >> -- >> Steve Loughran >> Hortonworks Inc >> [EMAIL PROTECTED] >> skype: steve_loughran >> tel: +1 408 400 3721 >> >> <http://hortonworks.com/download/> >> >> >> > -- Steve Loughran Hortonworks Inc [EMAIL PROTECTED] skype: steve_loughran tel: +1 408 400 3721 <http://hortonworks.com/download/>
-
Re: Problem running PI example in Hadoop 2.0.0anil gupta 2012-08-11, 04:50
Hi Sathya,
It wont help you even little bit in unsubscribing from Hadoop list even if you send a zillion unsubscribe email to [EMAIL PROTECTED] address. Unsubscribe instructions are present on the Apache Hadoop website. Here's what you are supposed to do for unsubscribing: To unsubscribe, you have to send a mail to user-unsubscribe @hadoop.apache.org with subject line as unsubscribe. ~Anil On Fri, Aug 10, 2012 at 9:09 PM, sathyavageeswaran <[EMAIL PROTECTED]>wrote: > unsubscribe**** > > ** ** > > *From:* Arjun Reddy [mailto:[EMAIL PROTECTED]] > *Sent:* 10 August 2012 23:25 > *To:* [EMAIL PROTECTED] > *Subject:* Re: Problem running PI example in Hadoop 2.0.0**** > > ** ** > > I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the > cluster is working fine now. Thank you very much for your help.**** > > On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <[EMAIL PROTECTED]> > wrote:**** > > ** ** > > On 9 August 2012 10:31, Arjun Reddy <[EMAIL PROTECTED]> wrote:**** > > I did go through the 6 steps listed at the link and telnet to port > 45965 is failing. Netstat -a shows nothing is listing on port 45965. **** > > ** ** > > ** ** > > That's a high number - I wonder if it's something transient that didn't > start.**** > > ** ** > > I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts > to 127.0.0.1 to see if that helps; it could just be some listening on a > different port**** > > **** > > **** > > We can run hadoop 1.0.3 version without any issues on this cluster and I > am new to 2.0.0 version. I followed the document > http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to > setup the config files. Is there any other good document that describes > how to set up version 2.0.0. **** > > **** > > Thanks**** > > Arjun**** > > On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <[EMAIL PROTECTED]> > wrote:**** > > ** ** > > On 8 August 2012 14:39, Arjun Reddy <[EMAIL PROTECTED]> wrote:**** > > I am trying to setup a small cluster using hadoop 2.0.0 and using PI > example to validate the setup. When I have 1 master and 1 slave the > example works fine. I am getting exceptions with the PI example when > additional slave nodes are added to the cluster. The syslogs for failed > tasks are as follows. Any ideas why this is happening.**** > > ** ** > > . > 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: > Exception running child : java.net.ConnectException: Call From node2/ > 127.0.1.1 to node2:45965 failed on connection exception: > java.net.ConnectException: Connection refused; For more details see: > http://wiki.apache.org/hadoop/ConnectionRefused > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727) > at org.apache.hadoop.ipc.Client.call(Client.java:1165) > at > org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224) > at $Proxy6.getTask(Unknown Source) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)**** > > did you follow the link in this message? **** > > ** ** > > If so, did you follow all the actions?**** > > ** ** > > ** ** > > > > **** > > ** ** > > -- **** > > Steve Loughran**** > > Hortonworks Inc**** > > [EMAIL PROTECTED]**** > > skype: steve_loughran**** > > tel: +1 408 400 3721**** > > ** ** > > **** <http://hortonworks.com/download/> > > ** ** <http://hortonworks.com/download/> > > ** ** <http://hortonworks.com/download/> > > ** ** <http://hortonworks.com/download/> > ------------------------------ > <http://hortonworks.com/download/> > > No virus found in this message. > Checked by AVG - *www.avg.com* > Version: 2012.0.2197 / Virus Database: 2437/5190 - Release Date: 08/09/12* > *** <http://hortonworks.com/download/> > -- Thanks & Regards, Anil Gupta |