Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - GC overhead limit exceeded


+
Camp, Roy 2012-10-11, 01:08
+
Harish Mandala 2012-10-11, 01:15
+
Brock Noland 2012-10-11, 01:17
Copy link to this message
-
Re: GC overhead limit exceeded
Denny Ye 2012-10-11, 02:25
It might be caused by :
1. Less heap memory. Increasing '-Xms -Xmx' options
2. Disable that prompt. It's not recommended. Using
'-XX:-UseGCOverheadLimit' option
3. There always have too many GC limitation with memory channel. I have met
some case with you before. I have tried to use direct memory for storing
event body out of heap in the past, that's effective method and no any GC
happened. I think you can test it by yourself.

-Regards
Denny Ye

2012/10/11 Brock Noland <[EMAIL PROTECTED]>

> Yep, sounds like: agent heap size < (capacity * avg event size)
>
> Brock
>
> On Wed, Oct 10, 2012 at 8:15 PM, Harish Mandala <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> >
> > In flume-env.sh, please add
> >
> > JAVA_OPTS="-Xms128m -Xmx256m"
> >
> > (Or whatever amount of memory works for you. I have Xmx set to 4g)
> >
> > Regards,
> > Harish
> >
> >
> > On Wed, Oct 10, 2012 at 9:08 PM, Camp, Roy <[EMAIL PROTECTED]> wrote:
> >>
> >> I ran into the following error – I had to restart flume to recover.  Do
> I
> >> just need to adjust a GC setting of some sort or is there a larger issue
> >> here?
> >>
> >>
> >>
> >> Source: thriftLegacy
> >>
> >> Channel: memory (capacity: 1,000,000)
> >>
> >> Sink: avro
> >>
> >>
> >>
> >>
> >>
> >> Exception in thread "pool-4-thread-2" java.lang.OutOfMemoryError: GC
> >> overhead limit exceeded
> >>
> >>         at
> >>
> java.util.concurrent.LinkedBlockingDeque.<init>(LinkedBlockingDeque.java:108)
> >>
> >>         at
> >>
> org.apache.flume.channel.MemoryChannel$MemoryTransaction.<init>(MemoryChannel.java:49)
> >>
> >>         at
> >>
> org.apache.flume.channel.MemoryChannel.createTransaction(MemoryChannel.java:264)
> >>
> >>         at
> >>
> org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:118)
> >>
> >>         at
> >>
> org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:260)
> >>
> >>         at
> >>
> org.apache.flume.source.thriftLegacy.ThriftLegacySource$ThriftFlumeEventServerImpl.append(ThriftLegacySource.java:96)
> >>
> >>         at
> >>
> com.cloudera.flume.handlers.thrift.ThriftFlumeEventServer$Processor$append.process(ThriftFlumeEventServer.java:276)
> >>
> >>         at
> >>
> com.cloudera.flume.handlers.thrift.ThriftFlumeEventServer$Processor.process(ThriftFlumeEventServer.java:256)
> >>
> >>         at
> >>
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176)
> >>
> >>         at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> >>
> >>         at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> >>
> >>         at java.lang.Thread.run(Thread.java:679)
> >>
> >> Exception in thread
> "SinkRunner-PollingRunner-LoadBalancingSinkProcessor"
> >> java.lang.OutOfMemoryError: GC overhead limit exceeded
> >>
> >> Exception in thread "Avro NettyTransceiver I/O Worker 1"
> >> java.lang.OutOfMemoryError: GC overhead limit exceeded
> >>
> >> Oct 10, 2012 5:51:39 PM
> >> org.jboss.netty.channel.socket.nio.AbstractNioWorker
> >>
> >> WARNING: Unexpected exception in the selector loop.
> >>
> >> java.lang.OutOfMemoryError: GC overhead limit exceeded
> >>
> >> Oct 10, 2012 5:52:35 PM
> >> org.jboss.netty.channel.socket.nio.AbstractNioWorker
> >>
> >> WARNING: Unexpected exception in the selector loop.
> >>
> >> java.lang.OutOfMemoryError: GC overhead limit exceeded
> >>
> >> Exception in thread "lifecycleSupervisor-1-2"
> java.lang.OutOfMemoryError:
> >> GC overhead limit exceeded
> >>
> >> Exception in thread "pool-4-thread-4" java.lang.OutOfMemoryError: GC
> >> overhead limit exceeded
> >>
> >> Exception in thread "lifecycleSupervisor-1-0"
> java.lang.OutOfMemoryError:
> >> GC overhead limit exceeded
> >>
> >> Exception in thread "lifecycleSupervisor-1-10"
> java.lang.OutOfMemoryError:
> >> GC overhead limit exceeded
> >>
> >> Oct 10, 2012 5:52:39 PM
> >> org.jboss.netty.channel.socket.nio.AbstractNioWorker
> >>
> >> WARNING: Unexpected exception in the selector loop.
+
Senthilvel Rangaswamy 2012-10-11, 02:27
+
Denny Ye 2012-10-11, 02:40
+
Brock Noland 2012-10-11, 11:38
+
Denny Ye 2012-10-11, 12:15