Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Memory exception in the mapper


Copy link to this message
-
Re: Memory exception in the mapper
Arun,

Actually CDH3 is Hadoop 0.20, but with .21 backported, so I am using 0.21
API whenever I can.

Mark

On Wed, May 23, 2012 at 9:40 PM, Mark Kerzner <[EMAIL PROTECTED]>wrote:

> Arun,
>
> I am running the latest CDH3, which I re-installed yesterday, so I believe
> it is Hadoop 0.21.
>
> I have about 6000 maps emitted, and 16 spills, and then I see Mapper
> cleanup() being called, after which I get this error
>
> 2012-05-23 20:22:58,108 FATAL org.apache.hadoop.mapred.Child: Error
> running child : java.lang.OutOfMemoryError: Java heap space
>     at org.apache.hadoop.mapred.IFile$Reader.readNextBlock(IFile.java:355)
>
> Thank you,
> Mark
>
>
> On Wed, May 23, 2012 at 9:29 PM, Arun C Murthy <[EMAIL PROTECTED]>wrote:
>
>> What version of hadoop are you running?
>>
>> On May 23, 2012, at 12:16 PM, Mark Kerzner wrote:
>>
>> > Hi, all,
>> >
>> > I got the exception below in the mapper. I already have my global Hadoop
>> > heap at 5 GB, but is there a specific other setting? Or maybe I should
>> > troubleshoot for memory?
>> >
>> > But the same application works in the IDE.
>> >
>> > Thank you!
>> >
>> > Mark
>> >
>> > *stderr logs*
>> >
>> > Exception in thread "Thread for syncLogs" java.lang.OutOfMemoryError:
>> > Java heap space
>> >       at
>> java.io.BufferedOutputStream.<init>(BufferedOutputStream.java:76)
>> >       at
>> java.io.BufferedOutputStream.<init>(BufferedOutputStream.java:59)
>> >       at
>> org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:292)
>> >       at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:365)
>> >       at org.apache.hadoop.mapred.Child$3.run(Child.java:157)
>> > Exception in thread "communication thread" java.lang.OutOfMemoryError:
>> > Java heap space
>> >
>> > Exception: java.lang.OutOfMemoryError thrown from the
>> > UncaughtExceptionHandler in thread "communication thread"
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB