Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: Set the number of maps

Harsh J 2012-11-01, 16:55
Marcos Ortiz 2012-11-01, 18:03
Ted Dunning 2012-11-01, 16:50
Copy link to this message
Re: Set the number of maps
Hi Peter

'mapred.tasktracker.map.tasks.maximum' is not for setting an upper cap on the map tasks spawned by a job. This property is used to set the map slots on each TaskTracker. It is TaskTracker level property and cannot be overriden on a per job basis.

To control the number of map tasks for your job you need to increase the value for min and max split sizes.

Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Ted Dunning <[EMAIL PROTECTED]>
Date: Thu, 1 Nov 2012 09:50:10
Subject: Re: Set the number of maps

Is the spelling of the option correct?

On Thu, Nov 1, 2012 at 6:43 AM, Cogan, Peter (Peter) <

> Hi
> I understand that the maximum number of concurrent map tasks is set
> by mapred.tasktracker.map.tasks.maximum  - however I wish to run with a
> smaller number of maps (am testing disk IO). I thought that I could set
> that within the main program using
> conf.set("mapred.tasktracker.map.tasks.maximuma", "4");
> to run with 4 maps – but that seems to have no impact. I know I could just
> change the mapred-site.xml and restart map reduce but that's kind of a
> pain. Can it be set from within the code?
> Thanks
> Peter