Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> How to set the number of map tasks? (ver 0.20.2)


Copy link to this message
-
Re: How to set the number of map tasks? (ver 0.20.2)
Felix,

> I'm using the new Job class:
>
> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html
>
> There is a way to set the number of reduce tasks:
>
> setNumReduceTasks(int tasks)
>
> However, I don't see how to set the number of MAP tasks?
>
> I tried to set it through mapred-site.xml :
>
>        <property>
>                <name>mapred.map.tasks</name>
>                <value>500</value>
>        </property>
>
> It doesn't work either (launched map task is still small).
>
> I'm wondering, do I have to rename the prefix from "mapred" to
> "mapreduce"? like this (for all configurations?):
>
>        <property>
>                <name>mapreduce.map.tasks</name>
>                <value>500</value>
>        </property>
>
> I added both, and it still doesn't work.

As documented in the mapreduce tutorial as well as the Java
documentation (http://bit.ly/9HKclu), the number of map tasks is
primarily determined by the number of input splits generated for the
input data.

Thanks
Hemanth
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB