Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Memory exception in the mapper


+
Mark Kerzner 2012-05-23, 19:16
+
Arun C Murthy 2012-05-24, 02:29
Copy link to this message
-
Re: Memory exception in the mapper
Arun,

I am running the latest CDH3, which I re-installed yesterday, so I believe
it is Hadoop 0.21.

I have about 6000 maps emitted, and 16 spills, and then I see Mapper
cleanup() being called, after which I get this error

2012-05-23 20:22:58,108 FATAL org.apache.hadoop.mapred.Child: Error running
child : java.lang.OutOfMemoryError: Java heap space
    at org.apache.hadoop.mapred.IFile$Reader.readNextBlock(IFile.java:355)

Thank you,
Mark

On Wed, May 23, 2012 at 9:29 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> What version of hadoop are you running?
>
> On May 23, 2012, at 12:16 PM, Mark Kerzner wrote:
>
> > Hi, all,
> >
> > I got the exception below in the mapper. I already have my global Hadoop
> > heap at 5 GB, but is there a specific other setting? Or maybe I should
> > troubleshoot for memory?
> >
> > But the same application works in the IDE.
> >
> > Thank you!
> >
> > Mark
> >
> > *stderr logs*
> >
> > Exception in thread "Thread for syncLogs" java.lang.OutOfMemoryError:
> > Java heap space
> >       at
> java.io.BufferedOutputStream.<init>(BufferedOutputStream.java:76)
> >       at
> java.io.BufferedOutputStream.<init>(BufferedOutputStream.java:59)
> >       at
> org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:292)
> >       at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:365)
> >       at org.apache.hadoop.mapred.Child$3.run(Child.java:157)
> > Exception in thread "communication thread" java.lang.OutOfMemoryError:
> > Java heap space
> >
> > Exception: java.lang.OutOfMemoryError thrown from the
> > UncaughtExceptionHandler in thread "communication thread"
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
+
Mark Kerzner 2012-05-24, 02:48
+
Joey Krabacher 2012-05-23, 19:47
+
Mark Kerzner 2012-05-24, 02:35
+
Joey Krabacher 2012-05-24, 02:43
+
Mark Kerzner 2012-05-24, 03:21
+
Joey Krabacher 2012-05-24, 03:39
+
Mark Kerzner 2012-05-24, 03:44
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB