Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Take list for MemoryTransaction, capacity 100 full ?


+
周梦想 2013-02-26, 05:57
Copy link to this message
-
Re: Take list for MemoryTransaction, capacity 100 full?
more logs:

2013-02-26 14:37:00,380 (SinkRunner-PollingRunner-DefaultSinkProcessor)
[ERROR -
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)] Unable
to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: org.apache.flume.ChannelException:
Take list for MemoryTransaction, capacity 100 full, consider committing
more frequently, increasing capacity, or increasing thread count
        at
org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:464)
        at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.flume.ChannelException: Take list for
MemoryTransaction, capacity 100 full, consider committing more frequently,
increasing capacity, or increasing thread count
        at
org.apache.flume.channel.MemoryChannel$MemoryTransaction.doTake(MemoryChannel.java:100)
        at
org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:113)
        at
org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:95)
        at
org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:391)
        ... 3 more
2013-02-26 14:37:02,854 (pool-7-thread-1) [ERROR -
org.apache.flume.source.AvroSource.appendBatch(AvroSource.java:261)] Avro
source userlogsrc: Unable to process event batch. Exception follows.
org.apache.flume.ChannelException: Unable to put batch on required channel:
org.apache.flume.channel.MemoryChannel{name: memch1}
        at
org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:200)
        at
org.apache.flume.source.AvroSource.appendBatch(AvroSource.java:259)
        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at
org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:88)
        at org.apache.avro.ipc.Responder.respond(Responder.java:149)
        at
org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:188)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75)
        at
org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:173)
        at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:792)
        at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at
org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:321)
        at
org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:303)
        at
org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:208)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75)
        at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
        at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at
org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
        at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
        at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
        at
org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
        at
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.flume.ChannelException: Space for commit to queue
couldn't be acquired Sinks are likely not keeping up with sources, or the
buffer size is too tight
        at
org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:126)
        at
org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
        at
org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:192)
        ... 28 more
2013/2/26 周梦想 <[EMAIL PROTECTED]>

+
周梦想 2013-02-26, 07:41
+
Hari Shreedharan 2013-02-26, 07:56
+
周梦想 2013-02-27, 02:29
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB