Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Spill related errors after modifying parameters


Copy link to this message
-
Spill related errors after modifying parameters
ranjith raghunath 2012-10-07, 01:52
This seems a little odd but after modifying the following parameters

io.sort.mb
*io*.*sort*.*spill*.percent
mapred.min.split.size
mapred.inmem.merge.threshold

mapred.child.java.opts

I am seeing the following error messages:

java.io.IOException: Spill failed at org.apache.hadoop.mapred.
MapTask$MapOutputBuffer.collect(MapTask.java:1029) at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
Any help on this topic appreciated would be appreciated

Thanks
+
ranjith raghunath 2012-10-07, 12:56