Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Error running the pi program on a 4 node cluster


Copy link to this message
-
Error running the pi program on a 4 node cluster
Hi, all

Could some help looking at this problem? I am setting up a four node
cluster on EC2 and seems that the cluster is set up fine until I start
testing.

I have tried password-less ssh from each node to all the nodes and there is
no problem connecting. Any advice will be greatly appreciated!

[hduser@ip-XX-XX-XXX-XXX hadoop]$ bin/hadoop jar
share/hadoop/mapreduce/hadoop-mapreduce-examples-0.23.4.jar pi -
Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory
-libjars
share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-0.23.4.jar 16 10000
Number of Maps  = 16
Samples per Map = 10000
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Wrote input for Map #10
Wrote input for Map #11
Wrote input for Map #12
Wrote input for Map #13
Wrote input for Map #14
Wrote input for Map #15
Starting Job
12/11/09 12:02:59 INFO input.FileInputFormat: Total input paths to process
: 16
12/11/09 12:02:59 INFO mapreduce.JobSubmitter: number of splits:16
12/11/09 12:02:59 WARN conf.Configuration: mapred.job.classpath.files is
deprecated. Instead, use mapreduce.job.classpath.files
12/11/09 12:02:59 WARN conf.Configuration: mapred.jar is deprecated.
Instead, use mapreduce.job.jar
12/11/09 12:02:59 WARN conf.Configuration: mapred.cache.files is
deprecated. Instead, use mapreduce.job.cache.files
12/11/09 12:02:59 WARN conf.Configuration:
mapred.map.tasks.speculative.execution is deprecated. Instead, use
mapreduce.map.speculative
12/11/09 12:02:59 WARN conf.Configuration: mapred.reduce.tasks is
deprecated. Instead, use mapreduce.job.reduces
12/11/09 12:02:59 WARN conf.Configuration: mapred.output.value.class is
deprecated. Instead, use mapreduce.job.output.value.class
12/11/09 12:02:59 WARN conf.Configuration:
mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
mapreduce.reduce.speculative
12/11/09 12:02:59 WARN conf.Configuration: mapred.used.genericoptionsparser
is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
12/11/09 12:02:59 WARN conf.Configuration: mapreduce.map.class is
deprecated. Instead, use mapreduce.job.map.class
12/11/09 12:02:59 WARN conf.Configuration: mapred.job.name is deprecated.
Instead, use mapreduce.job.name
12/11/09 12:02:59 WARN conf.Configuration: mapreduce.reduce.class is
deprecated. Instead, use mapreduce.job.reduce.class
12/11/09 12:02:59 WARN conf.Configuration: mapreduce.inputformat.class is
deprecated. Instead, use mapreduce.job.inputformat.class
12/11/09 12:02:59 WARN conf.Configuration: mapred.input.dir is deprecated.
Instead, use mapreduce.input.fileinputformat.inputdir
12/11/09 12:02:59 WARN conf.Configuration: mapred.output.dir is deprecated.
Instead, use mapreduce.output.fileoutputformat.outputdir
12/11/09 12:02:59 WARN conf.Configuration: mapreduce.outputformat.class is
deprecated. Instead, use mapreduce.job.outputformat.class
12/11/09 12:02:59 WARN conf.Configuration: mapred.map.tasks is deprecated.
Instead, use mapreduce.job.maps
12/11/09 12:02:59 WARN conf.Configuration: mapred.cache.files.timestamps is
deprecated. Instead, use mapreduce.job.cache.files.timestamps
12/11/09 12:02:59 WARN conf.Configuration: mapred.output.key.class is
deprecated. Instead, use mapreduce.job.output.key.class
12/11/09 12:02:59 WARN conf.Configuration: mapred.working.dir is
deprecated. Instead, use mapreduce.job.working.dir
12/11/09 12:03:00 INFO mapred.ResourceMgrDelegate: Submitted application
application_1352478937343_0002 to ResourceManager at master/
10.12.181.233:60400
12/11/09 12:03:00 INFO mapreduce.Job: The url to track the job:
http://master:8088/proxy/application_1352478937343_0002/
12/11/09 12:03:00 INFO mapreduce.Job: Running job: job_1352478937343_0002
12/11/09 12:03:01 INFO mapreduce.Job: Job job_1352478937343_0002 running in
uber mode : false
12/11/09 12:03:01 INFO mapreduce.Job:  map 0% reduce 0%
12/11/09 12:03:01 INFO mapreduce.Job: Job job_1352478937343_0002 failed
with state FAILED due to: Application application_1352478937343_0002 failed
1 times due to Error launching appattempt_1352478937343_0002_000001. Got
exception: java.lang.reflect.UndeclaredThrowableException
        at
org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagerPBClientImpl.startContainer(ContainerManagerPBClientImpl.java:111)
        at
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:115)
        at
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:258)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:722)
Caused by: com.google.protobuf.ServiceException:
java.net.UnknownHostException: Invalid host name: local host is: (unknown);
destination host is: ""domU-12-31-39-16-45-3B":46641;
java.net.UnknownHostException; For more details see:
http://wiki.apache.org/hadoop/UnknownHost
        at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:144)
        at $Proxy24.startContainer(Unknown Source)
        at
org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagerPBClientImpl.startContainer(ContainerManagerPBClientImpl.java:104)
        ... 5 more
Caused by: java.net.UnknownHostException: Invalid host name: local host is:
(unknown); destination host is: ""domU-12-31-39-16-45-3B":46641;
java.net.UnknownHostException; For more details see:
http://wiki.apache.org/hadoop/UnknownHost
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:713)
        at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:236)
        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1188)
        at org.apache.hadoop.ipc.Client.call(Client.java:1068)