Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Fixing a failed reduce task


Copy link to this message
-
Fixing a failed reduce task
I am running a map reduce ob where a few reduce tasks fail with an out of
memory error -
Increasing the memory is not an option. However if a retry had information
that an earlier attempt
failed out of memory and especially it it had access to a few numbers
describing how far the earlier attempt
managed to get, it could defend against the error
I have seen little information about how a retried task might access the
error logs or other information
from previous attempts - is there such a mechanism???
--
Steven M. Lewis PhD
Institute for Systems Biology
Seattle WA
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB