Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Dropped events


I have my custom Source and Sink that I have hooked with a memory channel but I am noticing that it is not very consistent. Even after sending many events to the Source the Sink’s event is still null. It works for about 1 out 4 events. I do see the events going to the Logger Sink so I know the Source is doing its job. However, for the custom Sink I simply get Event is null messages. I tried adjusting the channel capacity from 10 to 1 but no difference.

Thanks.

Here’s my conf file

# flume-httpxmlhttp.conf: A single-node Flume with Http Source and Http sink configuration

# Name the components on this agent
agent1.sources = r1
agent1.channels = c1

# Describe/configure the source
agent1.sources.r1.type = org.apache.flume.source.http.HTTPSource
agent1.sources.r1.port = 5140
agent1.sources.r1.handler = main.java.org.apache.flume.source.http.XMLHandler
agent1.sources.r1.handler.nickname = random props

# Setup the sinks
agent1.sinks = httpsink logsink

# Describe the sink
agent1.sinks.logsink.type = logger

# Describe the sink
agent1.sinks.httpsink.type = main.java.org.apache.flume.sink.http.HttpPostSink
agent1.sinks.httpsink.serverAddress = <my http server URL>

# Use a channel which buffers events in memory
agent1.channels.c1.type = memory
agent1.channels.c1.capacity = 1
agent1.channels.c1.transactionCapacity = 1

# Bind the source and sink to the channel
agent1.sources.r1.channels = c1
agent1.sinks.logsink.channel = c1
agent1.sinks.httpsink.channel = c1
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB