Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> FileOutputCommiter behavior doubts


Copy link to this message
-
FileOutputCommiter behavior doubts
Hi,

I have an implementation SampleFileOutputCommiter which extends org.apache.hadoop.mapred.FileOutputCommitter . The implementation has specific code to be executed during cleanupJob() execution.
When the framework(LocalJobRunner) makes a call to commitJob(), the framework never takes care of calling the cleanupJob() instead it calls org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitJob() which internally calls its own cleanupJob().
Though the method cleanupJob is deprecated but, still I feel the framework should take care of executing it as it is being executed from org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.
Currently the framework is not letting the Jobs written with MRV1 to run properly.

Any thoughts….. ??

Cheers,
Subroto Sanyal
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB