Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> mapred.map.tasks vs mapred.tasktracker.map.tasks.maximum


Copy link to this message
-
Re: mapred.map.tasks vs mapred.tasktracker.map.tasks.maximum
Is this system parameter too? Or can I specify as mapred.map.tasks? I am
using pig.

On Fri, Mar 9, 2012 at 6:19 PM, Chen He <[EMAIL PROTECTED]> wrote:

> if you do not specify  setNumMapTasks, by default, system will use the
> number you configured  for "mapred.map.tasks" in the conf/mapred-site.xml
> file.
>
> On Fri, Mar 9, 2012 at 7:19 PM, Mohit Anchlia <[EMAIL PROTECTED]
> >wrote:
>
> > What's the difference between setNumMapTasks and mapred.map.tasks?
> >
> > On Fri, Mar 9, 2012 at 5:00 PM, Chen He <[EMAIL PROTECTED]> wrote:
> >
> > > Hi Mohit
> > >
> > > " mapred.tasktracker.reduce(map).tasks.maximum " means how many
> > reduce(map)
> > > slot(s) you can have on each tasktracker.
> > >
> > > "mapred.job.reduce(maps)" means default number of reduce (map) tasks
> your
> > > job will has.
> > >
> > > To set the number of mappers in your application. You can write like
> > this:
> > >
> > > *configuration.setNumMapTasks(the number you want);*
> > >
> > > Chen
> > >
> > > Actually, you can just use configuration.set()
> > >
> > > On Fri, Mar 9, 2012 at 6:42 PM, Mohit Anchlia <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > What's the difference between mapred.tasktracker.reduce.tasks.maximum
> > and
> > > > mapred.map.tasks
> > > > **
> > >  > I want my data to be split against only 10 mappers in the entire
> > > cluster.
> > > > Can I do that using one of the above parameters?
> > > >
> > >
> >
>