|
|
-
Task JVM reuse: A Question regarding hadoop 0.20.1Saptarshi Guha 2010-07-09, 11:38
Hello,
I have set mapred.job.reuse.jvm.num.tasks to -1 for re-using the JVM. My intention is to run a helper program at the beginning of the job and then feed the key/value pairs from the tasks to the helper program. Currently am running it in the call to setup below. If JVM Task re-use is -1, will the Mapper class containing the run method below be created once and Hadoop will call the objects run method as many times as there tasks assigned to that JVM? Thank you Saptarshi public void run(Context context) throws IOException, InterruptedException { setup(context); while (context.nextKeyValue()) { map(context.getCurrentKey(), context.getCurrentValue(), context); } cleanup(context); } |