Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Reduce Merge Memtomem Parameter


Copy link to this message
-
Re: Reduce Merge Memtomem Parameter
Hi!

This parameter triggers a sort of fetched map outputs on the reducer node when the number of in memory map outputs > memToMemMergeOutputsThreshold . It is disabled by default. I am guessing this was put in on the premise that it might be faster to
sort fewer number of streams even in memory. And also we can sort
perhaps while waiting to fetch.

HTH
Ravi.
On Friday, October 25, 2013 3:22 PM, - <[EMAIL PROTECTED]> wrote:
 
Hi All,

What do "mapreduce.reduce.merge.memtomem.threshold" and "mapreduce.reduce.merge.memtomem.enabled" do and by default is it enabled? There's no documentation for them but I noticed that they exist in hadoop 2.2.0.

Thanks!