Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Reg: Map output copy failure


+
Manoj Babu 2012-12-10, 13:09
Copy link to this message
-
Re: Reg: Map output copy failure
What was the job or query you were running?

Couple of suggestions:
1. Reduce data set size with job chaining
2. Increase Reduce task heap
3. If you are using Hive/Pig, you may want to tune your query.

-Bharath
________________________________
 From: Manoj Babu <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Monday, December 10, 2012 5:09 AM
Subject: Reg: Map output copy failure
 

Hi All

I got the below exception, Is the issue related to https://issues.apache.org/jira/browse/MAPREDUCE-1182 ?

Am using CDH3U1
2012-12-10 06:22:39,688 FATAL org.apache.hadoop.mapred.Task: attempt_201211120903_9197_r_000024_0 : Map output copy failure : java.lang.OutOfMemoryError: Java heap space at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMemory(ReduceTask.java:1593) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutput(ReduceTask.java:1453) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:1302) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:1234)Cheers!
Manoj.
+
Manoj Babu 2012-12-10, 13:29
+
Manoj Babu 2012-12-10, 13:38