Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Reduce Merge Memtomem Parameter

Copy link to this message
Re: Reduce Merge Memtomem Parameter

This parameter triggers a sort of fetched map outputs on the reducer node when the number of in memory map outputs > memToMemMergeOutputsThreshold . It is disabled by default. I am guessing this was put in on the premise that it might be faster to
sort fewer number of streams even in memory. And also we can sort
perhaps while waiting to fetch.

On Friday, October 25, 2013 3:22 PM, - <[EMAIL PROTECTED]> wrote:
Hi All,

What do "mapreduce.reduce.merge.memtomem.threshold" and "mapreduce.reduce.merge.memtomem.enabled" do and by default is it enabled? There's no documentation for them but I noticed that they exist in hadoop 2.2.0.