Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: How is the memory usage of containers controlled?


Copy link to this message
-
Re: How is the memory usage of containers controlled?
Thank you Arun.

I am trying to study how YARN works. Can someone tell me which class(es)
monitors containers w.r.t. memory usage?
On 6/18/2013 4:07 PM, Arun C Murthy wrote:
> NodeManagers monitor containers w.r.t memory usage, and put containers
> in cgroups with cpu limits to restrict CPU usage.
>
> On Jun 18, 2013, at 12:30 PM, Yuzhang Han <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>> Hi,
>> I am curious about how YARN containers control their memory usage.
>> Say, I have a MR job, and I configure that every map task should be
>> assigned a 1 GB container, and every reduce task a 1.5 GB one. So,
>> when YARN runs the containers, how is it ensured that all map
>> containers use less than 1 GB memory and all reduce containers less
>> than 1.5 GB? Is container memory control realized via -Xmx arguments
>> in the scripts generated by YARN to invoke the MR program in the
>> containers?
>>
>> Thank you.
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB