Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> ulimit -v changed for child job?


Copy link to this message
-
ulimit -v changed for child job?
the Child of our jobs are launched using user hdfs

we got OOM in some jobs when they try to mmap() a big file.

I manually checked the ulimit -v of hdfs on the slave nodes, they are
unlimited,

but when I launched a fake streaming job, whose mapper is simply bash -c '
ulimit -a ',
I found that it has a 1G virtual memory limit.
why is this??
looks like a Cloudera bug

we are using CDH3U3
Thanks
Yang
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB