I have mapred.tasktracker.map.tasks.maximum set to 2 in my job and I have 5
nodes. I was expecting this to have only 10 concurrent jobs. But I have 30
mappers running. Does hadoop ignores this setting when supplied from the
job?

you set the " mapred.tasktracker.map.tasks.maximum " in your job means
nothing. Because Hadoop mapreduce platform only checks this parameter when
it starts. This is a system configuration.

  You need to set it in your conf/mapred-site.xml file and restart your
hadoop mapreduce.
On Fri, Mar 9, 2012 at 7:32 PM, Mohit Anchlia <[EMAIL PROTECTED]>wrote:

> I have mapred.tasktracker.map.tasks.maximum set to 2 in my job and I have 5
> nodes. I was expecting this to have only 10 concurrent jobs. But I have 30
> mappers running. Does hadoop ignores this setting when supplied from the
> job?
>

Adding on to Chen's response.

This is a setting meant at Task Tracker level(environment setting based on parameters like your CPU cores, memory etc) and you need to override the same at each task tracker's mapred-site.xml and restart the TT daemon for changes to be in effect.

Regards
Bejoy K S

From handheld, Please excuse typos.

-----Original Message-----
From: Chen He <[EMAIL PROTECTED]>
Date: Fri, 9 Mar 2012 20:16:23
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: mapred.tasktracker.map.tasks.maximum not working

you set the " mapred.tasktracker.map.tasks.maximum " in your job means
nothing. Because Hadoop mapreduce platform only checks this parameter when
it starts. This is a system configuration.

  You need to set it in your conf/mapred-site.xml file and restart your
hadoop mapreduce.
On Fri, Mar 9, 2012 at 7:32 PM, Mohit Anchlia <[EMAIL PROTECTED]>wrote:

> I have mapred.tasktracker.map.tasks.maximum set to 2 in my job and I have 5
> nodes. I was expecting this to have only 10 concurrent jobs. But I have 30
> mappers running. Does hadoop ignores this setting when supplied from the
> job?
>

Thanks. Looks like there are some parameters that I can use at client level
and others need cluster wide setting. Is there a place where I can see all
the config parameters with description of level of changes that can be done
at client level vs at cluster level?

On Fri, Mar 9, 2012 at 10:39 PM, <[EMAIL PROTECTED]> wrote:

> Adding on to Chen's response.
>
> This is a setting meant at Task Tracker level(environment setting based on
> parameters like your CPU cores, memory etc) and you need to override the
> same at each task tracker's mapred-site.xml and restart the TT daemon for
> changes to be in effect.
>
> Regards
> Bejoy K S
>
> From handheld, Please excuse typos.
>
> -----Original Message-----
> From: Chen He <[EMAIL PROTECTED]>
> Date: Fri, 9 Mar 2012 20:16:23
> To: <[EMAIL PROTECTED]>
> Reply-To: [EMAIL PROTECTED]
> Subject: Re: mapred.tasktracker.map.tasks.maximum not working
>
> you set the " mapred.tasktracker.map.tasks.maximum " in your job means
> nothing. Because Hadoop mapreduce platform only checks this parameter when
> it starts. This is a system configuration.
>
>  You need to set it in your conf/mapred-site.xml file and restart your
> hadoop mapreduce.
>
>
> On Fri, Mar 9, 2012 at 7:32 PM, Mohit Anchlia <[EMAIL PROTECTED]
> >wrote:
>
> > I have mapred.tasktracker.map.tasks.maximum set to 2 in my job and I
> have 5
> > nodes. I was expecting this to have only 10 concurrent jobs. But I have
> 30
> > mappers running. Does hadoop ignores this setting when supplied from the
> > job?
> >
>
>