I have "set mapred.map.tasks  5" in the pig job and still I am seeing
around 214 map tasks and around 30 actively running jobs. I was expecting
only 5 map tasks.

My cluster has 5 nodes.

Number of maps depends on the input splits. If your dataset is too big (and
not gzipped) there will be a map created for each split (which equals block
size).

On Fri, Mar 9, 2012 at 4:39 PM, Mohit Anchlia <[EMAIL PROTECTED]>wrote:

> I have "set mapred.map.tasks  5" in the pig job and still I am seeing
> around 214 map tasks and around 30 actively running jobs. I was expecting
> only 5 map tasks.
>
> My cluster has 5 nodes.
>

then what's the purpose of mapred.map.tasks? I thought that controls the
number of map tasks that will be spawned for the job.

Can I control the number of mappers like setNumMapTasks() in mapreduce?

On Fri, Mar 9, 2012 at 4:44 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:

> Number of maps depends on the input splits. If your dataset is too big (and
> not gzipped) there will be a map created for each split (which equals block
> size).
>
> On Fri, Mar 9, 2012 at 4:39 PM, Mohit Anchlia <[EMAIL PROTECTED]
> >wrote:
>
> > I have "set mapred.map.tasks  5" in the pig job and still I am seeing
> > around 214 map tasks and around 30 actively running jobs. I was expecting
> > only 5 map tasks.
> >
> > My cluster has 5 nodes.
> >
>