Hi,

I am currently trying to tune a CDH 4.0.1 cluster running HDFS, YARN,
and HBase managed by Cloudera Manager 4.0.3 (Free Edition).

In CM, there are a number of options for setting mapreduce.*
configuration properties on the YARN client page.

Some of the explanations in the GUI still refer to JobTracker and
TaskTracker, e.g.,
- mapreduce.jobtracker.handler.count,
- mapreduce.tasktracker.map.tasks.maximum,
- mapreduce.tasktracker.reduce.tasks.maximum

I wonder whether these and a number of other mapreduce.* (e.g.,
mapreduce.job.reduces) properties are observed by the MR2
ApplicationMaster or not.

Can anyone clarify or point to respective documentation?

Thanks,
Martin

Hi,

AFAIK, these properties are being ignored by YARN:
- mapreduce.tasktracker.map.tasks.maximum,
- mapreduce.tasktracker.reduce.tasks.maximum

Thanks,
Anil Gupta
On Thu, Aug 16, 2012 at 9:28 AM, mg <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha) cluster
> running HDFS, YARN, and HBase managed by Cloudera Manager 4.0.3 (Free
> Edition).
>
>
> In CM, there are a number of options for setting mapreduce.* configuration
> properties on the YARN client page.
>
> Some of the properties and their explanations in the GUI still refer to
> JobTracker and TaskTracker, e.g.,
>
> - mapreduce.jobtracker.handler.**count,
> - mapreduce.tasktracker.map.**tasks.maximum,
> - mapreduce.tasktracker.reduce.**tasks.maximum
>
> I wonder whether these and a number of other mapreduce.* (e.g.,
> mapreduce.job.reduces) properties are observed by the MR2 ApplicationMaster
> (and how), or not.
>
>
> Can anyone clarify or point to respective documentation?
>
> Thanks,
> Martin
>

--
Thanks & Regards,
Anil Gupta

http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html

On Aug 16, 2012, at 10:37 AM, mg wrote:

> Hi,
>
> thanks.
>
> But there are many more mapreduce.* properties. ...
>
> Does anyone have more information on those with respect to whether they are relevant to YARN, and if so, what they exactly mean in the YARN context?
>
> I am trying to optimize settings in order to use as much RAM and CPU as possible. The cluster has 8 node managers each with 96GB RAM and 16 CPUs (cpuinfo: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz)
>
> Cheers,
> Martin
>
> On 16.08.2012 18:40, anil gupta wrote:
>> Hi,
>>
>> AFAIK, these properties are being ignored by YARN:
>> - mapreduce.tasktracker.map.tasks.maximum,
>> - mapreduce.tasktracker.reduce.tasks.maximum
>>
>> Thanks,
>> Anil Gupta
>>
>>
>> On Thu, Aug 16, 2012 at 9:28 AM, mg <[EMAIL PROTECTED]
>> <mailto:[EMAIL PROTECTED]>> wrote:
>>
>>    Hi,
>>
>>    I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>>    cluster running HDFS, YARN, and HBase managed by Cloudera Manager
>>    4.0.3 (Free Edition).
>>
>>
>>    In CM, there are a number of options for setting mapreduce.*
>>    configuration properties on the YARN client page.
>>
>>    Some of the properties and their explanations in the GUI still refer
>>    to JobTracker and TaskTracker, e.g.,
>>
>>    - mapreduce.jobtracker.handler.__count,
>>    - mapreduce.tasktracker.map.__tasks.maximum,
>>    - mapreduce.tasktracker.reduce.__tasks.maximum
>>
>>    I wonder whether these and a number of other mapreduce.* (e.g.,
>>    mapreduce.job.reduces) properties are observed by the MR2
>>    ApplicationMaster (and how), or not.
>>
>>
>>    Can anyone clarify or point to respective documentation?
>>
>>    Thanks,
>>    Martin
>>
>>
>>
>>
>> --
>> Thanks & Regards,
>> Anil Gupta

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/