-Re: Set the number of maps
Bejoy KS 2012-11-01, 16:59
'mapred.tasktracker.map.tasks.maximum' is not for setting an upper cap on the map tasks spawned by a job. This property is used to set the map slots on each TaskTracker. It is TaskTracker level property and cannot be overriden on a per job basis.
To control the number of map tasks for your job you need to increase the value for min and max split sizes.
Sent from handheld, please excuse typos.
From: Ted Dunning <[EMAIL PROTECTED]>
Date: Thu, 1 Nov 2012 09:50:10
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Set the number of maps
Is the spelling of the option correct?
On Thu, Nov 1, 2012 at 6:43 AM, Cogan, Peter (Peter) <
[EMAIL PROTECTED]> wrote:
> I understand that the maximum number of concurrent map tasks is set
> by mapred.tasktracker.map.tasks.maximum - however I wish to run with a
> smaller number of maps (am testing disk IO). I thought that I could set
> that within the main program using
> conf.set("mapred.tasktracker.map.tasks.maximuma", "4");
> to run with 4 maps – but that seems to have no impact. I know I could just
> change the mapred-site.xml and restart map reduce but that's kind of a
> pain. Can it be set from within the code?