Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Task JVM reuse: A Question regarding hadoop 0.20.1


Copy link to this message
-
Task JVM reuse: A Question regarding hadoop 0.20.1
Hello,
I have set mapred.job.reuse.jvm.num.tasks to -1 for re-using the JVM.
My intention is to run a helper program at the beginning of the job and then
feed the key/value pairs
from the tasks to the helper program.
Currently am running it in the call to setup below.

If JVM Task re-use is -1, will the Mapper class containing the run method
below be created once
and Hadoop will call the objects run method as many times as there tasks
assigned to that JVM?

Thank you
Saptarshi

  public void run(Context context) throws IOException,
 InterruptedException {
 setup(context);
 while (context.nextKeyValue()) {
    map(context.getCurrentKey(), context.getCurrentValue(), context);
 }
 cleanup(context);
    }
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB