-oom during shuffle
Radim Kolar 2013-05-06, 02:35
i retested it and oom error is also in 2.0.3, but not in 1.x.
how much memory is needed for reducer during shuffle and sort? Peak
usage is memoryLimit or there have to be additional memory for
mapred.reduce.parallel.copies fetch buffers?
MergerManager: memoryLimit=326264416, maxSingleShuffleLimit=81566104,
mergeThreshold=215334528, ioSortFactor=10, memToMemMergeOutputsThreshold=10