Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - mapred.map.tasks


Copy link to this message
-
Re: mapred.map.tasks
Mohit Anchlia 2012-03-10, 01:16
then what's the purpose of mapred.map.tasks? I thought that controls the
number of map tasks that will be spawned for the job.

Can I control the number of mappers like setNumMapTasks() in mapreduce?

On Fri, Mar 9, 2012 at 4:44 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:

> Number of maps depends on the input splits. If your dataset is too big (and
> not gzipped) there will be a map created for each split (which equals block
> size).
>
> On Fri, Mar 9, 2012 at 4:39 PM, Mohit Anchlia <[EMAIL PROTECTED]
> >wrote:
>
> > I have "set mapred.map.tasks  5" in the pig job and still I am seeing
> > around 214 map tasks and around 30 actively running jobs. I was expecting
> > only 5 map tasks.
> >
> > My cluster has 5 nodes.
> >
>