-Re: "JVM reuse": if I use this option, do setup(), cleanup() get called only once?
Harsh J 2012-11-13, 08:20
Those API hooks are called once per task attempt, and regardless of
JVM reuse they will still be run once per task attempt. So yes,
setup+cleanup for every map split or reduce partition that runs
through the reused JVM.
On Tue, Nov 13, 2012 at 1:47 PM, edward choi <[EMAIL PROTECTED]> wrote:
> This question might sound stupid, but I couldn't find a definite answer on
> My job loads a big file at setup() in Map tasks.
> So I would like to use the loaded file again and again.
> I came across "JVM reuse", but text book only says that this option enables
> multiple use of
> tasks on a single JVM. It does not say anything about setup() or cleanup().
> Even if I set "mapred.job.reuse.jvm.num.tasks" to -1, do setup() and
> cleanup() get called every single time a task is launched?