Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> JVM vs container memory configs


Copy link to this message
-
Re: JVM vs container memory configs
For us we typically leave a 500MB difference between the heap and the
container size.  I think we can make this smaller, but we have not really
tried.

--Bobby

On 5/3/13 11:20 AM, "Karthik Kambatla" <[EMAIL PROTECTED]> wrote:

>Hi
>
>While looking into MAPREDUCE-5207 (adding defaults for
>mapreduce.{map|reduce}.memory.mb), I was wondering how much headroom
>should
>be left on top of mapred.child.java.opts (or other similar JVM opts) for
>the container memory itself?
>
>Currently, mapred.child.java.opts (per mapred-default.xml) is set to 200
>MB
>by default. The default for mapreduce.{map|reduce}.memory.mb is 1024 in
>the
>code, which is significantly higher than the 200MB value.
>
>Do we need more than 100 MB for non-JVM memory per container? If so, does
>it make sense make that a config property in itself and the code to verify
>all 3 values are clear enough?
>
>Thanks
>Karthik
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB