Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Erro running pi programm


+
Kartashov, Andy 2012-11-09, 17:36
Copy link to this message
-
Re: Erro running pi programm
Hi, Andy

Thanks for suggestions!

I am running it on a four node cluster on EC2. All the services started
fine, Namenode, Datanode, ResourceManager, NodeManager and
JobHistoryServer. Each node can ssh to all the nodes without problem.

But problem appears when trying to run any job.
On Fri, Nov 9, 2012 at 9:36 AM, Kartashov, Andy <[EMAIL PROTECTED]>wrote:

>   Yinghua,
>
>
>
> What mode are you running your hadoop in: Local/Pseud/Fully...?
>
>
>
> Your hostname is not recognised
>
>
>
> Your configuration setting seems to be wrong.
>
>
>
>
>
>
>
>
>
>
>
> Hi, all
>
>
>
> Could some help looking at this problem? I am setting up a four node
> cluster on EC2 and seems that the cluster is set up fine until I start
> testing.
>
>
>
> I have tried password-less ssh from each node to all the nodes and there
> is no problem connecting. Any advice will be greatly appreciated!
>
>
>
> [hduser@ip-XX-XX-XXX-XXX hadoop]$ bin/hadoop jar
> share/hadoop/mapreduce/hadoop-mapreduce-examples-0.23.4.jar pi -
> Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory
> -libjars
> share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-0.23.4.jar 16 10000
>
> Number of Maps  = 16
>
> Samples per Map = 10000
>
> Wrote input for Map #0
>
> Wrote input for Map #1
>
> Wrote input for Map #2
>
> Wrote input for Map #3
>
> Wrote input for Map #4
>
> Wrote input for Map #5
>
> Wrote input for Map #6
>
> Wrote input for Map #7
>
> Wrote input for Map #8
>
> Wrote input for Map #9
>
> Wrote input for Map #10
>
> Wrote input for Map #11
>
> Wrote input for Map #12
>
> Wrote input for Map #13
>
> Wrote input for Map #14
>
> Wrote input for Map #15
>
> Starting Job
>
> 12/11/09 12:02:59 INFO input.FileInputFormat: Total input paths to process
> : 16
>
> 12/11/09 12:02:59 INFO mapreduce.JobSubmitter: number of splits:16
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapred.job.classpath.files is
> deprecated. Instead, use mapreduce.job.classpath.files
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapred.cache.files is
> deprecated. Instead, use mapreduce.job.cache.files
>
> 12/11/09 12:02:59 WARN conf.Configuration:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapred.output.value.class is
> deprecated. Instead, use mapreduce.job.output.value.class
>
> 12/11/09 12:02:59 WARN conf.Configuration:
> mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.reduce.speculative
>
> 12/11/09 12:02:59 WARN conf.Configuration:
> mapred.used.genericoptionsparser is deprecated. Instead, use
> mapreduce.client.genericoptionsparser.used
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapred.job.name is deprecated.
> Instead, use mapreduce.job.name
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapreduce.reduce.class is
> deprecated. Instead, use mapreduce.job.reduce.class
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapreduce.inputformat.class is
> deprecated. Instead, use mapreduce.job.inputformat.class
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapred.input.dir is deprecated.
> Instead, use mapreduce.input.fileinputformat.inputdir
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapred.output.dir is
> deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapreduce.outputformat.class is
> deprecated. Instead, use mapreduce.job.outputformat.class
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapred.map.tasks is deprecated.
> Instead, use mapreduce.job.maps
>
> 12/11/09 12:02:59 WARN conf.Configuration: mapred.cache.files.timestamps

Regards,

Yinghua
+
yinghua hu 2012-11-09, 18:20
+
Kartashov, Andy 2012-11-09, 19:31
+
Ted Reynolds 2012-11-09, 20:43
+
yinghua hu 2012-11-10, 00:10