Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Error: org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError Java Heap Space


+
Eduard Skaley 2012-10-31, 15:45
+
Nitin Pawar 2012-10-31, 15:47
+
Eduard Skaley 2012-10-31, 15:58
+
Eduard Skaley 2012-11-05, 09:09
+
Eduard Skaley 2012-11-05, 13:20
+
Kartashov, Andy 2012-11-05, 15:43
Copy link to this message
-
Re: Error: org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError Java Heap Space
Eduard,

Would you try using the following properties in your job invocation?

-D mapreduce.map.java.opts=-Xmx768m -D
mapreduce.reduce.java.opts=-Xmx768m -D mapreduce.map.memory.mb=2000 -D
mapreduce.reduce.memory.mb=3000

Thx
On Mon, Nov 5, 2012 at 7:43 AM, Kartashov, Andy <[EMAIL PROTECTED]> wrote:
> Your error takes place during reduce task, when temporary files are written
> to memory/disk. You are clearly running low on resources. Check your memory
> “$ free –m” and disk space “$ df –H” as well as “$hadoop fs -df”
>
>
>
> I remember it took me a couple of days to figure out why I was getting heap
> size error and nothing wporked!  Becaue, I tried to write 7Gb output file
> onto a disk (in pseudo distr mode) that only had 4Gb of free space.
>
>
>
> p.s. Always test your jobs on small input first (few lines of inputs) .
>
>
>
> p.p.s. follow your job execution through web:
> http://<fully-qualified-hostan-name of your job tracker>:50030
>
>
>
>
>
> From: Eduard Skaley [mailto:[EMAIL PROTECTED]]
> Sent: Monday, November 05, 2012 4:10 AM
> To: [EMAIL PROTECTED]
> Cc: Nitin Pawar
> Subject: Re: Error:
> org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError Java Heap Space
>
>
>
> By the way it happens on Yarn not on MRv1
>
> each container gets 1GB at the moment.
>
> can you try increasing memory per reducer  ?
>
>
>
> On Wed, Oct 31, 2012 at 9:15 PM, Eduard Skaley <[EMAIL PROTECTED]> wrote:
>
> Hello,
>
> I'm getting this Error through job execution:
>
> 16:20:26 INFO  [main]                     Job -  map 100% reduce 46%
> 16:20:27 INFO  [main]                     Job -  map 100% reduce 51%
> 16:20:29 INFO  [main]                     Job -  map 100% reduce 62%
> 16:20:30 INFO  [main]                     Job -  map 100% reduce 64%
> 16:20:32 INFO  [main]                     Job - Task Id :
> attempt_1351680008718_0018_r_000006_0, Status : FAILED
> Error: org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error
> in shuffle in fetcher#2
>     at org.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:123)
>     at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:371)
>     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:396)
>     at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
>     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
> Caused by: java.lang.OutOfMemoryError: Java heap space
>     at
> org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:58)
>     at
> org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:45)
>     at
> org.apache.hadoop.mapreduce.task.reduce.MapOutput.<init>(MapOutput.java:97)
>     at
> org.apache.hadoop.mapreduce.task.reduce.MergeManager.unconditionalReserve(MergeManager.java:286)
>     at
> org.apache.hadoop.mapreduce.task.reduce.MergeManager.reserve(MergeManager.java:276)
>     at
> org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:384)
>     at
> org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:319)
>     at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:179)
>
> 16:20:33 INFO  [main]                     Job -  map 100% reduce 65%
> 16:20:36 INFO  [main]                     Job -  map 100% reduce 67%
> 16:20:39 INFO  [main]                     Job -  map 100% reduce 69%
> 16:20:41 INFO  [main]                     Job -  map 100% reduce 70%
> 16:20:43 INFO  [main]                     Job -  map 100% reduce 71%
>
> I have no clue what the issue could be for this. I googled this issue and
> checked several sources of possible solutions but nothing does fit.
>
> I saw this jira entry which could fit:
> https://issues.apache.org/jira/browse/MAPREDUCE-4655.
>
> Here somebody recommends to increase the value for the property

Alejandro
+
Eduard Skaley 2012-10-31, 15:54
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB