Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Error running the pi program on a 4 node cluster


Copy link to this message
-
Error running the pi program on a 4 node cluster
Hi, all

Could some help looking at this problem? I am setting up a four node
cluster on EC2 and seems that the cluster is set up fine until I start
testing.

I have tried password-less ssh from each node to all the nodes and there is
no problem connecting. Any advice will be greatly appreciated!

[hduser@ip-XX-XX-XXX-XXX hadoop]$ bin/hadoop jar
share/hadoop/mapreduce/hadoop-mapreduce-examples-0.23.4.jar pi -
Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory
-libjars
share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-0.23.4.jar 16 10000
Number of Maps  = 16
Samples per Map = 10000
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Wrote input for Map #10
Wrote input for Map #11
Wrote input for Map #12
Wrote input for Map #13
Wrote input for Map #14
Wrote input for Map #15
Starting Job
12/11/09 12:02:59 INFO input.FileInputFormat: Total input paths to process
: 16
12/11/09 12:02:59 INFO mapreduce.JobSubmitter: number of splits:16
12/11/09 12:02:59 WARN conf.Configuration: mapred.job.classpath.files is
deprecated. Instead, use mapreduce.job.classpath.files
12/11/09 12:02:59 WARN conf.Configuration: mapred.jar is deprecated.
Instead, use mapreduce.job.jar
12/11/09 12:02:59 WARN conf.Configuration: mapred.cache.files is
deprecated. Instead, use mapreduce.job.cache.files
12/11/09 12:02:59 WARN conf.Configuration:
mapred.map.tasks.speculative.execution is deprecated. Instead, use
mapreduce.map.speculative
12/11/09 12:02:59 WARN conf.Configuration: mapred.reduce.tasks is
deprecated. Instead, use mapreduce.job.reduces
12/11/09 12:02:59 WARN conf.Configuration: mapred.output.value.class is
deprecated. Instead, use mapreduce.job.output.value.class
12/11/09 12:02:59 WARN conf.Configuration:
mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
mapreduce.reduce.speculative
12/11/09 12:02:59 WARN conf.Configuration: mapred.used.genericoptionsparser
is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
12/11/09 12:02:59 WARN conf.Configuration: mapreduce.map.class is
deprecated. Instead, use mapreduce.job.map.class
12/11/09 12:02:59 WARN conf.Configuration: mapred.job.name is deprecated.
Instead, use mapreduce.job.name
12/11/09 12:02:59 WARN conf.Configuration: mapreduce.reduce.class is
deprecated. Instead, use mapreduce.job.reduce.class
12/11/09 12:02:59 WARN conf.Configuration: mapreduce.inputformat.class is
deprecated. Instead, use mapreduce.job.inputformat.class
12/11/09 12:02:59 WARN conf.Configuration: mapred.input.dir is deprecated.
Instead, use mapreduce.input.fileinputformat.inputdir
12/11/09 12:02:59 WARN conf.Configuration: mapred.output.dir is deprecated.
Instead, use mapreduce.output.fileoutputformat.outputdir
12/11/09 12:02:59 WARN conf.Configuration: mapreduce.outputformat.class is
deprecated. Instead, use mapreduce.job.outputformat.class
12/11/09 12:02:59 WARN conf.Configuration: mapred.map.tasks is deprecated.
Instead, use mapreduce.job.maps
12/11/09 12:02:59 WARN conf.Configuration: mapred.cache.files.timestamps is
deprecated. Instead, use mapreduce.job.cache.files.timestamps
12/11/09 12:02:59 WARN conf.Configuration: mapred.output.key.class is
deprecated. Instead, use mapreduce.job.output.key.class
12/11/09 12:02:59 WARN conf.Configuration: mapred.working.dir is
deprecated. Instead, use mapreduce.job.working.dir
12/11/09 12:03:00 INFO mapred.ResourceMgrDelegate: Submitted application
application_1352478937343_0002 to ResourceManager at master/
10.12.181.233:60400
12/11/09 12:03:00 INFO mapreduce.Job: The url to track the job:
http://master:8088/proxy/application_1352478937343_0002/
12/11/09 12:03:00 INFO mapreduce.Job: Running job: job_1352478937343_0002
12/11/09 12:03:01 INFO mapreduce.Job: Job job_1352478937343_0002 running in
uber mode : false
12/11/09 12:03:01 INFO mapreduce.Job:  map 0% reduce 0%
12/11/09 12:03:01 INFO mapreduce.Job: Job job_1352478937343_0002 failed
with state FAILED due to: Application application_1352478937343_0002 failed
1 times due to Error launching appattempt_1352478937343_0002_000001. Got
exception: java.lang.reflect.UndeclaredThrowableException
        at
org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagerPBClientImpl.startContainer(ContainerManagerPBClientImpl.java:111)
        at
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:115)
        at
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:258)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:722)
Caused by: com.google.protobuf.ServiceException:
java.net.UnknownHostException: Invalid host name: local host is: (unknown);
destination host is: ""domU-12-31-39-16-45-3B":46641;
java.net.UnknownHostException; For more details see:
http://wiki.apache.org/hadoop/UnknownHost
        at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:144)
        at $Proxy24.startContainer(Unknown Source)
        at
org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagerPBClientImpl.startContainer(ContainerManagerPBClientImpl.java:104)
        ... 5 more
Caused by: java.net.UnknownHostException: Invalid host name: local host is:
(unknown); destination host is: ""domU-12-31-39-16-45-3B":46641;
java.net.UnknownHostException; For more details see:
http://wiki.apache.org/hadoop/UnknownHost
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:713)
        at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:236)
        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1188)
        at org.apache.hadoop.ipc.Client.call(Client.java:1068)
      
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB