Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Parameter 'yarn.nodemanager.resource.cpu-cores' does not work


+
sam liu 2013-07-03, 03:33
Copy link to this message
-
RE: Parameter 'yarn.nodemanager.resource.cpu-cores' does not work
I think you need to change the following configurations in yarn-site.xml to enable CPU resource limits.

'yarn.nodemanager.container-monitor.resource-calculator.class' 'org.apache.hadoop.yarn.util.resource.DominantResourceCalculator'

'yarn.nodemanager.container-executor.class'
'org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor'

Some LinuxContainerExecutor configurations:
yarn.nodemanager.linux-container-executor.path
yarn.nodemanager.linux-container-executor.resources-handler.class
yarn.nodemanager.linux-container-executor.cgroups.hierarchy
yarn.nodemanager.linux-container-executor.cgroups.mount
yarn.nodemanager.linux-container-executor.cgroups.mount-path

-Chuan

From: sam liu [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, July 02, 2013 8:33 PM
To: [EMAIL PROTECTED]
Subject: Parameter 'yarn.nodemanager.resource.cpu-cores' does not work

Hi,
With Hadoop 2.0.4-alpha, yarn.nodemanager.resource.cpu-cores does not work for me:
1. The performance of running same terasort job do not change, even after increasing or decreasing the value of 'yarn.nodemanager.resource.cpu-cores' in yarn-site.xml and restart the yarn cluster.
2. Even if I set the value of both
'yarn.nodemanager.resource.cpu-cores' and 'yarn.nodemanager.vcores-pcores-ratio' to 0, the MR job still could complete without any exception, but the expected behavior should be that no cpu could be assigned to the container, and then no job could be executed on the cluster. Right?
Thanks!

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB