Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: Set the number of maps


Copy link to this message
-
Re: Set the number of maps
Bejoy KS 2012-11-01, 16:59
Hi Peter

'mapred.tasktracker.map.tasks.maximum' is not for setting an upper cap on the map tasks spawned by a job. This property is used to set the map slots on each TaskTracker. It is TaskTracker level property and cannot be overriden on a per job basis.

To control the number of map tasks for your job you need to increase the value for min and max split sizes.

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Ted Dunning <[EMAIL PROTECTED]>
Date: Thu, 1 Nov 2012 09:50:10
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Set the number of maps

Is the spelling of the option correct?

On Thu, Nov 1, 2012 at 6:43 AM, Cogan, Peter (Peter) <
[EMAIL PROTECTED]> wrote:

> Hi
>
> I understand that the maximum number of concurrent map tasks is set
> by mapred.tasktracker.map.tasks.maximum  - however I wish to run with a
> smaller number of maps (am testing disk IO). I thought that I could set
> that within the main program using
>
> conf.set("mapred.tasktracker.map.tasks.maximuma", "4");
>
>
> to run with 4 maps – but that seems to have no impact. I know I could just
> change the mapred-site.xml and restart map reduce but that's kind of a
> pain. Can it be set from within the code?
>
>
> Thanks
>
> Peter
>