Sai Sai 2013-03-14, 09:24
-Re: Increase the number of mappers in PM mode
YouPeng Yang 2013-03-14, 14:27
I have done some tests in my Pseudo Mode(CDH4.1.2)with MV2 yarn,and :
According to the doc:
*mapreduce.jobtracker.address :*The host and port that the MapReduce job
tracker runs at. If "local", then jobs are run in-process as a single map
and reduce task.
*mapreduce.job.maps (default value is 2)* :The default number of map
tasks per job. Ignored when mapreduce.jobtracker.address is "local".
I changed the mapreduce.jobtracker.address = Hadoop:50031.
And then run the wordcount examples:
hadoop jar hadoop-mapreduce-examples-2.0.0-cdh4.1.2.jar wordcount
the output logs are as follows:
Launched map tasks=1
Launched reduce tasks=1
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=60336
Total time spent by all reduces in occupied slots (ms)=63264
Map input records=5
Map output records=7
Map output bytes=56
Map output materialized bytes=76
i seem to does not work.
I thought maybe my input file is small-just 5 records . is it right?
2013/3/14 Sai Sai <[EMAIL PROTECTED]>
> In Pseudo Mode where is the setting to increase the number of mappers or
> is this not possible.
yypvsxf19870706 2013-03-16, 04:46