Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - How to set the number of map tasks? (ver 0.20.2)


Copy link to this message
-
Re: How to set the number of map tasks? (ver 0.20.2)
Hemanth Yamijala 2010-06-22, 01:43
Felix,

> I'm using the new Job class:
>
> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html
>
> There is a way to set the number of reduce tasks:
>
> setNumReduceTasks(int tasks)
>
> However, I don't see how to set the number of MAP tasks?
>
> I tried to set it through mapred-site.xml :
>
>        <property>
>                <name>mapred.map.tasks</name>
>                <value>500</value>
>        </property>
>
> It doesn't work either (launched map task is still small).
>
> I'm wondering, do I have to rename the prefix from "mapred" to
> "mapreduce"? like this (for all configurations?):
>
>        <property>
>                <name>mapreduce.map.tasks</name>
>                <value>500</value>
>        </property>
>
> I added both, and it still doesn't work.

As documented in the mapreduce tutorial as well as the Java
documentation (http://bit.ly/9HKclu), the number of map tasks is
primarily determined by the number of input splits generated for the
input data.

Thanks
Hemanth