Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Set the number of maps


Copy link to this message
-
Re: Set the number of maps
The option since 0.21 was renamed to
mapreduce.tasktracker.map.tasks.maximum, and like
Harsh said to you, is is a TaskTracker service level option.

Another thing is that this option is very united to the
mapreduce.child.java.opts, so , make sure
to monitor constantly the effect of these changes in your cluster.

On 11/01/2012 11:55 AM, Harsh J wrote:
> It can't be set from the code this way - the slot property is applied
> at the TaskTracker service level (as the name goes).
>
> Since you're just testing at the moment, try to set these values,
> restart TTs, and run your jobs again. You do not need to restart JT at
> any point for tweaking these values.
>
> On Thu, Nov 1, 2012 at 7:13 PM, Cogan, Peter (Peter)
> <[EMAIL PROTECTED]> wrote:
>> Hi
>>
>> I understand that the maximum number of concurrent map tasks is set by
>> mapred.tasktracker.map.tasks.maximum  - however I wish to run with a smaller
>> number of maps (am testing disk IO). I thought that I could set that within
>> the main program using
>>
>> conf.set("mapred.tasktracker.map.tasks.maximuma", "4");
>>
>>
>> to run with 4 maps � but that seems to have no impact. I know I could just
>> change the mapred-site.xml and restart map reduce but that's kind of a pain.
>> Can it be set from within the code?
>>
>>
>> Thanks
>>
>> Peter
>
>

--

Marcos Luis Ort�z Valmaseda
about.me/marcosortiz <http://about.me/marcosortiz>
@marcosluis2186 <http://twitter.com/marcosluis2186>

10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB