Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Avro and HDFS


+
Mohit Anchlia 2012-09-18, 21:15
Copy link to this message
-
Re: Avro and HDFS
Your channel is full because HDFS Sink is not clearing the channel fast enough.

Thanks,
Hari

--
Hari Shreedharan
On Tuesday, September 18, 2012 at 2:15 PM, Mohit Anchlia wrote:

> I have a 6 node cluster and client machines where we run flume. We use Netty RPC client that comes with flume-ng. When we shutdown one of the nodes in Hadoop cluster we start get this error and it never stops until the node comes back up. Could someone suggest what might be wrong here?
>  
>  12/09/17 14:19:48 WARN source.AvroSource: Avro source avroSrc: Unable to process event. Exception follows.
> org.apache.flume.ChannelException: Unable to put event on required channel: org.apache.flume.channel.MemoryChannel@49938039 (mailto:org.apache.flume.channel.MemoryChannel@49938039)
>         at org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:275)
>         at org.apache.flume.source.AvroSource.append(AvroSource.java:207)
>         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:88)
>         at org.apache.avro.ipc.Responder.respond(Responder.java:149)
>         at org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:178)
>         at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75)
>         at org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:163)
>         at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>         at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:792)
>         at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
>         at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:321)
>         at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:303)
>         at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:208)
>         at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75)
>         at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>         at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
>         at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
>         at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
>         at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
>         at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
>         at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
>         at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
>         at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.flume.ChannelException: Space for commit to queue couldn't be acquired Sinks are likely not keeping up with sources, or the buffer size is too tight
>         at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:91)
>         at org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
>         at org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:267)