edward choi 2012-11-13, 08:17
-Re: "JVM reuse": if I use this option, do setup(), cleanup() get called only once?
Those API hooks are called once per task attempt, and regardless of
JVM reuse they will still be run once per task attempt. So yes,
setup+cleanup for every map split or reduce partition that runs
through the reused JVM.
On Tue, Nov 13, 2012 at 1:47 PM, edward choi <[EMAIL PROTECTED]> wrote:
> This question might sound stupid, but I couldn't find a definite answer on
> My job loads a big file at setup() in Map tasks.
> So I would like to use the loaded file again and again.
> I came across "JVM reuse", but text book only says that this option enables
> multiple use of
> tasks on a single JVM. It does not say anything about setup() or cleanup().
> Even if I set "mapred.job.reuse.jvm.num.tasks" to -1, do setup() and
> cleanup() get called every single time a task is launched?