Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Set reducer capacity for a specific M/R job


Copy link to this message
-
Set reducer capacity for a specific M/R job
Hi,

I want to change the cluster's capacity of reduce slots on a per job basis.
Originally I have 8 reduce slots for a tasktracker.
I did:

conf.set("mapred.tasktracker.reduce.tasks.maximum", "4");
...
Job job = new Job(conf, ...)
And in the web UI I can see that for this job, the max reduce tasks is
exactly at 4, like I set. However hadoop still launches 8 reducer per
datanode ... why is this?

How could I achieve this?
--
*JU Han*

Software Engineer Intern @ KXEN Inc.
UTC   -  Université de Technologie de Compiègne
*     **GI06 - Fouille de Données et Décisionnel*

+33 0619608888
+
Nitin Pawar 2013-04-30, 10:26
+
Han JU 2013-04-30, 10:32
+
Nitin Pawar 2013-04-30, 10:35
+
Han JU 2013-04-30, 10:38
+
Nitin Pawar 2013-04-30, 10:45
+
Nitin Pawar 2013-04-30, 10:28
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB