Vikram Kulkarni 2013-03-12, 18:03
Hari Shreedharan 2013-03-12, 18:05
Vikram Kulkarni 2013-03-12, 20:33
Hari, I still have the same problem even after setting the transaction capacity.
However, does it have anything to do with Batching as I am not using 'batchSize' currently. I just wanted to keep it simple like the Logger sink.
From: Vikram Kulkarni [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, March 12, 2013 1:33 PM
To: [EMAIL PROTECTED]
Subject: RE: Dropped events
I had it at 10 originally and the capacity at 100.
Yes, I am committing the transaction and closing it (and backoff) even if the Event is null.
From: Hari Shreedharan [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, March 12, 2013 11:06 AM
To: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
Subject: Re: Dropped events
Your channel capacity is set to 1. Are you sure you really want that? If a null event is returned by the channel, then you should commit and close the transaction, optionally backoff (usually, this is when a batch was empty), and then try again. You should probably also use a higher transaction capacity.
On Tuesday, March 12, 2013 at 11:03 AM, Vikram Kulkarni wrote:
I have my custom Source and Sink that I have hooked with a memory channel but I am noticing that it is not very consistent. Even after sending many events to the Source the Sink's event is still null. It works for about 1 out 4 events. I do see the events going to the Logger Sink so I know the Source is doing its job. However, for the custom Sink I simply get Event is null messages. I tried adjusting the channel capacity from 10 to 1 but no difference.
Here's my conf file
# flume-httpxmlhttp.conf: A single-node Flume with Http Source and Http sink configuration
# Name the components on this agent
agent1.sources = r1
agent1.channels = c1
# Describe/configure the source
agent1.sources.r1.type = org.apache.flume.source.http.HTTPSource
agent1.sources.r1.port = 5140
agent1.sources.r1.handler = main.java.org.apache.flume.source.http.XMLHandler
agent1.sources.r1.handler.nickname = random props
# Setup the sinks
agent1.sinks = httpsink logsink
# Describe the sink
agent1.sinks.logsink.type = logger
# Describe the sink
agent1.sinks.httpsink.type = main.java.org.apache.flume.sink.http.HttpPostSink
agent1.sinks.httpsink.serverAddress = <my http server URL>
# Use a channel which buffers events in memory
agent1.channels.c1.type = memory
agent1.channels.c1.capacity = 1
agent1.channels.c1.transactionCapacity = 1
# Bind the source and sink to the channel
agent1.sources.r1.channels = c1
agent1.sinks.logsink.channel = c1
agent1.sinks.httpsink.channel = c1
Connor Woodson 2013-03-15, 19:22