Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: Set the number of maps


+
Harsh J 2012-11-01, 16:55
+
Marcos Ortiz 2012-11-01, 18:03
+
Ted Dunning 2012-11-01, 16:50
Copy link to this message
-
Re: Set the number of maps
Hi Peter

'mapred.tasktracker.map.tasks.maximum' is not for setting an upper cap on the map tasks spawned by a job. This property is used to set the map slots on each TaskTracker. It is TaskTracker level property and cannot be overriden on a per job basis.

To control the number of map tasks for your job you need to increase the value for min and max split sizes.

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Ted Dunning <[EMAIL PROTECTED]>
Date: Thu, 1 Nov 2012 09:50:10
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Set the number of maps

Is the spelling of the option correct?

On Thu, Nov 1, 2012 at 6:43 AM, Cogan, Peter (Peter) <
[EMAIL PROTECTED]> wrote:

> Hi
>
> I understand that the maximum number of concurrent map tasks is set
> by mapred.tasktracker.map.tasks.maximum  - however I wish to run with a
> smaller number of maps (am testing disk IO). I thought that I could set
> that within the main program using
>
> conf.set("mapred.tasktracker.map.tasks.maximuma", "4");
>
>
> to run with 4 maps – but that seems to have no impact. I know I could just
> change the mapred-site.xml and restart map reduce but that's kind of a
> pain. Can it be set from within the code?
>
>
> Thanks
>
> Peter
>

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB