Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> mapred.map.tasks


Copy link to this message
-
Re: mapred.map.tasks
then what's the purpose of mapred.map.tasks? I thought that controls the
number of map tasks that will be spawned for the job.

Can I control the number of mappers like setNumMapTasks() in mapreduce?

On Fri, Mar 9, 2012 at 4:44 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:

> Number of maps depends on the input splits. If your dataset is too big (and
> not gzipped) there will be a map created for each split (which equals block
> size).
>
> On Fri, Mar 9, 2012 at 4:39 PM, Mohit Anchlia <[EMAIL PROTECTED]
> >wrote:
>
> > I have "set mapred.map.tasks  5" in the pig job and still I am seeing
> > around 214 map tasks and around 30 actively running jobs. I was expecting
> > only 5 map tasks.
> >
> > My cluster has 5 nodes.
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB