Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> mapred.child.ulimit default value of null?


Copy link to this message
-
mapred.child.ulimit default value of null?
What is the interpretation of a default value for mapred.child.ulimit of null?  Does that mean children are completely unlimited in their permission to use memory?

If I want to insure maximum memory availability to children, should I leave this null (on a cluster where that is the default), or should I set it to some fairly large number, say 8 GBs (bearing in mind that the parameter's units are KB)?  Is null better given my desire here?

Also, what is a "child" in the context of this question (what does this parameter apply to)?  For example, is a map task itself considered a "child" of the overall job or does this parameter not affect the map task itself but only child processes spawned *by* the map task?

In particular, how would this parameter (or any other relevant parameters you can think of, say, mapred.child.java.opts) affect the availability of memory to compiled C++ code (.so files in the distributed cache) called from a map task via JNI?

What I really want to do is insure that the native routines (C++ .so) called via JNI from a (Java) mapper have maximum memory availability/access on the node.  Ostensibly, the native routines would be permitted to use the entire memory available on the machine.

Thank you.

________________________________________________________________________________
Keith Wiley               [EMAIL PROTECTED]               www.keithwiley.com

"And what if we picked the wrong religion?  Every week, we're just making God
madder and madder!"
  -- Homer Simpson
________________________________________________________________________________
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB