Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Set reducer capacity for a specific M/R job


Copy link to this message
-
Set reducer capacity for a specific M/R job
Han JU 2013-04-30, 10:00
Hi,

I want to change the cluster's capacity of reduce slots on a per job basis.
Originally I have 8 reduce slots for a tasktracker.
I did:

conf.set("mapred.tasktracker.reduce.tasks.maximum", "4");
...
Job job = new Job(conf, ...)
And in the web UI I can see that for this job, the max reduce tasks is
exactly at 4, like I set. However hadoop still launches 8 reducer per
datanode ... why is this?

How could I achieve this?
--
*JU Han*

Software Engineer Intern @ KXEN Inc.
UTC   -  Université de Technologie de Compiègne
*     **GI06 - Fouille de Données et Décisionnel*

+33 0619608888
+
Nitin Pawar 2013-04-30, 10:26
+
Han JU 2013-04-30, 10:32
+
Nitin Pawar 2013-04-30, 10:35
+
Han JU 2013-04-30, 10:38
+
Nitin Pawar 2013-04-30, 10:45
+
Nitin Pawar 2013-04-30, 10:28