then what's the purpose of mapred.map.tasks? I thought that controls the number of map tasks that will be spawned for the job.
Can I control the number of mappers like setNumMapTasks() in mapreduce?
On Fri, Mar 9, 2012 at 4:44 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:
> Number of maps depends on the input splits. If your dataset is too big (and > not gzipped) there will be a map created for each split (which equals block > size). > > On Fri, Mar 9, 2012 at 4:39 PM, Mohit Anchlia <[EMAIL PROTECTED] > >wrote: > > > I have "set mapred.map.tasks 5" in the pig job and still I am seeing > > around 214 map tasks and around 30 actively running jobs. I was expecting > > only 5 map tasks. > > > > My cluster has 5 nodes. > > >
All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by Sematext