Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: How is the memory usage of containers controlled?


Copy link to this message
-
Re: How is the memory usage of containers controlled?
Start with LinuxResourceCalculatorPlugin.

Arun

On Jun 21, 2013, at 1:30 PM, Yuzhang Han <[EMAIL PROTECTED]> wrote:

> Thank you Arun.
>
> I am trying to study how YARN works. Can someone tell me which class(es) monitors containers w.r.t. memory usage?
>
>
> On 6/18/2013 4:07 PM, Arun C Murthy wrote:
>> NodeManagers monitor containers w.r.t memory usage, and put containers in cgroups with cpu limits to restrict CPU usage.
>>
>> On Jun 18, 2013, at 12:30 PM, Yuzhang Han <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>> I am curious about how YARN containers control their memory usage. Say, I have a MR job, and I configure that every map task should be assigned a 1 GB container, and every reduce task a 1.5 GB one. So, when YARN runs the containers, how is it ensured that all map containers use less than 1 GB memory and all reduce containers less than 1.5 GB? Is container memory control realized via -Xmx arguments in the scripts generated by YARN to invoke the MR program in the containers?
>>>
>>> Thank you.
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB