Hello,
I am trying to send the events to the Avro Source through external RPC
client..
While sending the events, the following error is thrown

org.apache.flume.ChannelException: Space for commit to queue couldn't be
acquired Sinks are likely not keeping up with sources, or the buffer size
is too tight
 at
org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:85)
        at
org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
        at
org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:252)
        at org.apache.flume.source.AvroSource.append(AvroSource.java:203)
        at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at
org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:88)
        at org.apache.avro.ipc.Responder.respond(Responder.java:149)
        at
org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:158)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
        at
org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:143)

I have also changed the heap size to 1024m

And my host file is as follows:

host1.sources = avroSource
host1.channels = memoryChannel
host1.sinks = esper
#avroSource configuration
host1.sources.avroSource.type = avro
host1.sources.avroSource.bind = localhost
host1.sources.avroSource.port = 41414
host1.sources.avroSource.channels = memoryChannel
#Channels
host1.channels.memoryChannel.type = memory

#Sinks
host1.sinks.esper.type = com.gsshop.bigdata.poc.flume.EsperSink
host1.sinks.esper.channel = memoryChannel

What could be the reason for this error?

Regards,
SOm

You could try increasing the channel capacity. Add the following to your config file:

host1.channels.memoryChannel.capacity = 500

________________________________________
From: shekhar sharma [[EMAIL PROTECTED]]
Sent: Tuesday, July 24, 2012 9:54 AM
To: flume-user
Subject: Channel Exception

Hello,
I am trying to send the events to the Avro Source through external RPC client..
While sending the events, the following error is thrown

org.apache.flume.ChannelException: Space for commit to queue couldn't be acquired Sinks are likely not keeping up with sources, or the buffer size is too tight
 at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:85)
        at org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
        at org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:252)
        at org.apache.flume.source.AvroSource.append(AvroSource.java:203)
        at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:88)
        at org.apache.avro.ipc.Responder.respond(Responder.java:149)
        at org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:158)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
        at org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:143)

I have also changed the heap size to 1024m

And my host file is as follows:

host1.sources = avroSource
host1.channels = memoryChannel
host1.sinks = esper
#avroSource configuration
host1.sources.avroSource.type = avro
host1.sources.avroSource.bind = localhost
host1.sources.avroSource.port = 41414
host1.sources.avroSource.channels = memoryChannel
#Channels
host1.channels.memoryChannel.type = memory

#Sinks
host1.sinks.esper.type = com.gsshop.bigdata.poc.flume.EsperSink
host1.sinks.esper.channel = memoryChannel

What could be the reason for this error?

Regards,
SOm

This is one possibility. It is also possible that your sink is not
keeping up with your source. If It can't take items out faster than the
source can put them in, events will pile up and the channel will fill up.

On 07/24/2012 04:59 PM, Stern, Mark wrote:
> You could try increasing the channel capacity. Add the following to your config file:
>
> host1.channels.memoryChannel.capacity = 500
>
> ________________________________________
> From: shekhar sharma [[EMAIL PROTECTED]]
> Sent: Tuesday, July 24, 2012 9:54 AM
> To: flume-user
> Subject: Channel Exception
>
> Hello,
> I am trying to send the events to the Avro Source through external RPC client..
> While sending the events, the following error is thrown
>
> org.apache.flume.ChannelException: Space for commit to queue couldn't be acquired Sinks are likely not keeping up with sources, or the buffer size is too tight
>   at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:85)
>          at org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
>          at org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:252)
>          at org.apache.flume.source.AvroSource.append(AvroSource.java:203)
>          at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
>          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>          at java.lang.reflect.Method.invoke(Method.java:597)
>          at org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:88)
>          at org.apache.avro.ipc.Responder.respond(Responder.java:149)
>          at org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:158)
>          at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
>          at org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:143)
>
> I have also changed the heap size to 1024m
>
> And my host file is as follows:
>
> host1.sources = avroSource
> host1.channels = memoryChannel
> host1.sinks = esper
> #avroSource configuration
> host1.sources.avroSource.type = avro
> host1.sources.avroSource.bind = localhost
> host1.sources.avroSource.port = 41414
> host1.sources.avroSource.channels = memoryChannel
> #Channels
> host1.channels.memoryChannel.type = memory
>
> #Sinks
> host1.sinks.esper.type = com.gsshop.bigdata.poc.flume.EsperSink
> host1.sinks.esper.channel = memoryChannel
>
> What could be the reason for this error?
>
> Regards,
> SOm
>

Thanks. Actually in my case threads were blocked inside the sink, due to
which it was unable to commit the transaction.

Regards,
Som Shekhar

On Tue, Jul 24, 2012 at 2:21 PM, Juhani Connolly <
[EMAIL PROTECTED]> wrote:

> This is one possibility. It is also possible that your sink is not keeping
> up with your source. If It can't take items out faster than the source can
> put them in, events will pile up and the channel will fill up.
>
>
> On 07/24/2012 04:59 PM, Stern, Mark wrote:
>
>> You could try increasing the channel capacity. Add the following to your
>> config file:
>>
>> host1.channels.memoryChannel.**capacity = 500
>>
>> ______________________________**__________
>> From: shekhar sharma [[EMAIL PROTECTED]]
>> Sent: Tuesday, July 24, 2012 9:54 AM
>> To: flume-user
>> Subject: Channel Exception
>>
>> Hello,
>> I am trying to send the events to the Avro Source through external RPC
>> client..
>> While sending the events, the following error is thrown
>>
>> org.apache.flume.**ChannelException: Space for commit to queue couldn't
>> be acquired Sinks are likely not keeping up with sources, or the buffer
>> size is too tight
>>   at org.apache.flume.channel.**MemoryChannel$**
>> MemoryTransaction.doCommit(**MemoryChannel.java:85)
>>          at org.apache.flume.channel.**BasicTransactionSemantics.**
>> commit(**BasicTransactionSemantics.**java:151)
>>          at org.apache.flume.channel.**ChannelProcessor.processEvent(**
>> ChannelProcessor.java:252)
>>          at org.apache.flume.source.**AvroSource.append(AvroSource.**
>> java:203)
>>          at sun.reflect.**GeneratedMethodAccessor1.**invoke(Unknown
>> Source)
>>          at sun.reflect.**DelegatingMethodAccessorImpl.**invoke(**
>> DelegatingMethodAccessorImpl.**java:25)
>>          at java.lang.reflect.Method.**invoke(Method.java:597)
>>          at org.apache.avro.ipc.specific.**SpecificResponder.respond(**
>> SpecificResponder.java:88)
>>          at org.apache.avro.ipc.Responder.**respond(Responder.java:149)
>>          at org.apache.avro.ipc.**NettyServer$**NettyServerAvroHandler.**
>> messageReceived(NettyServer.**java:158)
>>          at org.jboss.netty.channel.**SimpleChannelUpstreamHandler.**
>> handleUpstream(**SimpleChannelUpstreamHandler.**java:80)
>>          at org.apache.avro.ipc.**NettyServer$**NettyServerAvroHandler.**
>> handleUpstream(NettyServer.**java:143)
>>
>> I have also changed the heap size to 1024m
>>
>> And my host file is as follows:
>>
>> host1.sources = avroSource
>> host1.channels = memoryChannel
>> host1.sinks = esper
>> #avroSource configuration
>> host1.sources.avroSource.type = avro
>> host1.sources.avroSource.bind = localhost
>> host1.sources.avroSource.port = 41414
>> host1.sources.avroSource.**channels = memoryChannel
>> #Channels
>> host1.channels.memoryChannel.**type = memory
>>
>> #Sinks
>> host1.sinks.esper.type = com.gsshop.bigdata.poc.flume.**EsperSink
>> host1.sinks.esper.channel = memoryChannel
>>
>> What could be the reason for this error?
>>
>> Regards,
>> SOm
>>
>>
>
>

You could try increasing the capacity of the channel. Add the following
to your config file:

host1.channels.memoryChannel.capacity = 500

On Tue, 2012-07-24 at 12:24 +0530, shekhar sharma wrote:
> Hello,
> I am trying to send the events to the Avro Source through external RPC
> client..
> While sending the events, the following error is thrown
>  
> org.apache.flume.ChannelException: Space for commit to queue couldn't
> be acquired Sinks are likely not keeping up with sources, or the
> buffer size is too tight
>  at org.apache.flume.channel.MemoryChannel
> $MemoryTransaction.doCommit(MemoryChannel.java:85)
>         at
> org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
>         at
> org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:252)
>         at
> org.apache.flume.source.AvroSource.append(AvroSource.java:203)
>         at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at
> org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:88)
>         at org.apache.avro.ipc.Responder.respond(Responder.java:149)
>         at org.apache.avro.ipc.NettyServer
> $NettyServerAvroHandler.messageReceived(NettyServer.java:158)
>         at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
>         at org.apache.avro.ipc.NettyServer
> $NettyServerAvroHandler.handleUpstream(NettyServer.java:143)
>
>  
> I have also changed the heap size to 1024m
>  
> And my host file is as follows:
>  
> host1.sources = avroSource
> host1.channels = memoryChannel
> host1.sinks = esper
> #avroSource configuration
> host1.sources.avroSource.type = avro
> host1.sources.avroSource.bind = localhost
> host1.sources.avroSource.port = 41414
> host1.sources.avroSource.channels = memoryChannel
> #Channels
> host1.channels.memoryChannel.type = memory
>
> #Sinks
> host1.sinks.esper.type = com.gsshop.bigdata.poc.flume.EsperSink
> host1.sinks.esper.channel = memoryChannel
>
>  
> What could be the reason for this error?
>  
> Regards,
> SOm