Mohit Anchlia 2012-03-10, 01:16
then what's the purpose of mapred.map.tasks? I thought that controls the
number of map tasks that will be spawned for the job.
Can I control the number of mappers like setNumMapTasks() in mapreduce?
On Fri, Mar 9, 2012 at 4:44 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:
> Number of maps depends on the input splits. If your dataset is too big (and
> not gzipped) there will be a map created for each split (which equals block
> On Fri, Mar 9, 2012 at 4:39 PM, Mohit Anchlia <[EMAIL PROTECTED]
> > I have "set mapred.map.tasks 5" in the pig job and still I am seeing
> > around 214 map tasks and around 30 actively running jobs. I was expecting
> > only 5 map tasks.
> > My cluster has 5 nodes.