Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - "JVM reuse": if I use this option, do setup(), cleanup() get called only once?


Copy link to this message
-
"JVM reuse": if I use this option, do setup(), cleanup() get called only once?
edward choi 2012-11-13, 08:17
Hi,

This question might sound stupid, but I couldn't find a definite answer on
Google.
My job loads a big file at setup() in Map tasks.
So I would like to use the loaded file again and again.

I came across "JVM reuse", but text book only says that this option enables
multiple use of
tasks on a single JVM. It does not say anything about setup() or cleanup().

Even if I set "mapred.job.reuse.jvm.num.tasks" to -1, do setup() and
cleanup() get called every single time a task is launched?

Best,
Ed