Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - Reg: Map output copy failure


+
Manoj Babu 2012-12-10, 13:09
+
Bharath Mundlapudi 2012-12-11, 02:09
Copy link to this message
-
Re: Reg: Map output copy failure
Manoj Babu 2012-12-10, 13:29
In the ReduceTask.java having the below code

maxSize = (int)(conf.getInt("mapred.job.reduce.total.mem.bytes",
            (int)Math.min(Runtime.getRuntime().maxMemory(),
Integer.MAX_VALUE))
          * maxInMemCopyUse);

But in the patch
 maxSize = (long)Math.min(
             Runtime.getRuntime().maxMemory() * maxInMemCopyUse,
             Integer.MAX_VALUE);

Whether cdh3u1 not holding this fix?
Kindly Advice.

Cheers!
Manoj.

On Mon, Dec 10, 2012 at 6:39 PM, Manoj Babu <[EMAIL PROTECTED]> wrote:

> at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:1302)
>
>
+
Manoj Babu 2012-12-10, 13:38