Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Relevance of mapreduce.* configuration properties for MR V2


Copy link to this message
-
Re: Relevance of mapreduce.* configuration properties for MR V2
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html

On Aug 16, 2012, at 10:37 AM, mg wrote:

> Hi,
>
> thanks.
>
> But there are many more mapreduce.* properties. ...
>
> Does anyone have more information on those with respect to whether they are relevant to YARN, and if so, what they exactly mean in the YARN context?
>
> I am trying to optimize settings in order to use as much RAM and CPU as possible. The cluster has 8 node managers each with 96GB RAM and 16 CPUs (cpuinfo: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz)
>
> Cheers,
> Martin
>
> On 16.08.2012 18:40, anil gupta wrote:
>> Hi,
>>
>> AFAIK, these properties are being ignored by YARN:
>> - mapreduce.tasktracker.map.tasks.maximum,
>> - mapreduce.tasktracker.reduce.tasks.maximum
>>
>> Thanks,
>> Anil Gupta
>>
>>
>> On Thu, Aug 16, 2012 at 9:28 AM, mg <[EMAIL PROTECTED]
>> <mailto:[EMAIL PROTECTED]>> wrote:
>>
>>    Hi,
>>
>>    I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
>>    cluster running HDFS, YARN, and HBase managed by Cloudera Manager
>>    4.0.3 (Free Edition).
>>
>>
>>    In CM, there are a number of options for setting mapreduce.*
>>    configuration properties on the YARN client page.
>>
>>    Some of the properties and their explanations in the GUI still refer
>>    to JobTracker and TaskTracker, e.g.,
>>
>>    - mapreduce.jobtracker.handler.__count,
>>    - mapreduce.tasktracker.map.__tasks.maximum,
>>    - mapreduce.tasktracker.reduce.__tasks.maximum
>>
>>    I wonder whether these and a number of other mapreduce.* (e.g.,
>>    mapreduce.job.reduces) properties are observed by the MR2
>>    ApplicationMaster (and how), or not.
>>
>>
>>    Can anyone clarify or point to respective documentation?
>>
>>    Thanks,
>>    Martin
>>
>>
>>
>>
>> --
>> Thanks & Regards,
>> Anil Gupta

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB