Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Pig out of memory error


Copy link to this message
-
Re: Pig out of memory error
Kind of curious what it's using all that RAM for. Don't suppose you profiled it?

On Mon, Jun 18, 2012 at 1:37 PM, Pankaj Gupta <[EMAIL PROTECTED]> wrote:
> Awesome, that worked. Thanks.
> To summarize the root cause and resolution as I understand it:
> At the end of first job, PIG did a list on the job and apparently that shot its Heap size requirement up to 1.7G. The existing, default, heapsize for the PIG JVM didn't allow that and gave an out of memory error. Setting HADOOP_HEAPSIZE to 8192 set the max heap size of the PIG process to 8196m and the problem went away.
>
> Thanks,
> Pankaj
> On Jun 18, 2012, at 1:23 AM, Aniket Mokashi wrote:
>
>> export HADOOP_HEAPSIZE=<something more than what it is right now>
>>
>> Thanks,
>> Aniket
>>
>> On Sun, Jun 17, 2012 at 11:16 PM, Pankaj Gupta <[EMAIL PROTECTED]>wrote:
>>
>>> Hi,
>>>
>>> I am getting an out of memory error while running Pig. I am running a
>>> pretty big job with one master node and over 100 worker nodes. Pig divides
>>> the execution in two map-reduce jobs. Both the jobs succeed with a small
>>> data set. With a large data set I get an out of memory error at the end of
>>> the first job. This happens right after the all the mappers and reducers of
>>> the first job are done and the second job hasn't started. Here is the error:
>>>
>>> 2012-06-18 03:15:29,565 [Low Memory Detector] INFO
>>> org.apache.pig.impl.util.SpillableMemoryManager - first memory handler
>>> call - Collection threshold init = 187039744(182656K) used >>> 390873656(381712K) committed = 613744640(599360K) max = 699072512(682688K)
>>> 2012-06-18 03:15:31,137 [Low Memory Detector] INFO
>>> org.apache.pig.impl.util.SpillableMemoryManager - first memory handler
>>> call- Usage threshold init = 187039744(182656K) used = 510001720(498048K)
>>> committed = 613744640(599360K) max = 699072512(682688K)
>>> Exception in thread "IPC Client (47) connection to /10.217.23.253:9001from hadoop" java.lang.RuntimeException:
>>> java.lang.reflect.InvocationTargetException
>>> Caused by: java.lang.reflect.InvocationTargetException
>>> Caused by: java.lang.OutOfMemoryError: Java heap space
>>>       at org.apache.hadoop.mapred.TaskReport.<init>(TaskReport.java:46)
>>>       at sun.reflect.GeneratedConstructorAccessor31.newInstance(Unknown
>>> Source)
>>>       at
>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>>>       at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>>>       at
>>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
>>>       at
>>> org.apache.hadoop.io.WritableFactories.newInstance(WritableFactories.java:53)
>>>       at
>>> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:236)
>>>       at
>>> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:171)
>>>       at
>>> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:219)
>>>       at
>>> org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:66)
>>>       at
>>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:807)
>>>       at org.apache.hadoop.ipc.Client$Connection.run(Client.java:742)
>>> Exception in thread "Low Memory Detector" java.lang.OutOfMemoryError: Java
>>> heap space
>>>       at
>>> sun.management.MemoryUsageCompositeData.getCompositeData(MemoryUsageCompositeData.java:40)
>>>       at
>>> sun.management.MemoryUsageCompositeData.toCompositeData(MemoryUsageCompositeData.java:34)
>>>       at
>>> sun.management.MemoryNotifInfoCompositeData.getCompositeData(MemoryNotifInfoCompositeData.java:42)
>>>       at
>>> sun.management.MemoryNotifInfoCompositeData.toCompositeData(MemoryNotifInfoCompositeData.java:36)
>>>       at sun.management.MemoryImpl.createNotification(MemoryImpl.java:168)
>>>       at
>>> sun.management.MemoryPoolImpl$CollectionSensor.triggerAction(MemoryPoolImpl.java:300)
>>>       at sun.management.Sensor.trigger(Sensor.java:120)
>>>
>>> I will really appreciate and suggestions on how to go about debugging and
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB