Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Fixing a failed reduce task


Copy link to this message
-
Fixing a failed reduce task
I am running a map reduce ob where a few reduce tasks fail with an out of
memory error -
Increasing the memory is not an option. However if a retry had information
that an earlier attempt
failed out of memory and especially it it had access to a few numbers
describing how far the earlier attempt
managed to get, it could defend against the error
I have seen little information about how a retried task might access the
error logs or other information
from previous attempts - is there such a mechanism???
--
Steven M. Lewis PhD
Institute for Systems Biology
Seattle WA