Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> "JVM reuse": if I use this option, do setup(), cleanup() get called only once?


Copy link to this message
-
"JVM reuse": if I use this option, do setup(), cleanup() get called only once?
Hi,

This question might sound stupid, but I couldn't find a definite answer on
Google.
My job loads a big file at setup() in Map tasks.
So I would like to use the loaded file again and again.

I came across "JVM reuse", but text book only says that this option enables
multiple use of
tasks on a single JVM. It does not say anything about setup() or cleanup().

Even if I set "mapred.job.reuse.jvm.num.tasks" to -1, do setup() and
cleanup() get called every single time a task is launched?

Best,
Ed
+
Harsh J 2012-11-13, 08:20
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB