Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - How to lower the total number of map tasks


Copy link to this message
-
How to lower the total number of map tasks
Shing Hing Man 2012-10-02, 16:34

I am running Hadoop 1.0.3 in Pseudo  distributed mode.
When I  submit a map/reduce job to process a file of  size about 16 GB, in job.xml, I have the following
mapred.map.tasks =242
mapred.min.split.size =0
dfs.block.size = 67108864
I would like to reduce   mapred.map.tasks to see if it improves performance.
I have tried doubling  the size of  dfs.block.size. But the    mapred.map.tasks remains unchanged.
Is there a way to reduce  mapred.map.tasks  ?
Thanks in advance for any assistance !  
Shing
+
Bejoy Ks 2012-10-02, 17:01
+
Bejoy Ks 2012-10-02, 17:03
+
Shing Hing Man 2012-10-02, 17:38
+
Bejoy KS 2012-10-02, 17:46
+
Chris Nauroth 2012-10-02, 17:00
+
Shing Hing Man 2012-10-02, 17:33
+
Bejoy KS 2012-10-02, 17:37
+
Shing Hing Man 2012-10-02, 18:17