Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Blocking Source if Channel is full?


Copy link to this message
-
Blocking Source if Channel is full?
Otis Gospodnetic 2014-02-28, 04:52
Hi,

Is it possible to configure Flume to block on reading from the Sink when
the Channel is full?
For example, if the source is a file being tailed, and the sink is HDFS,
and writing to HDFS is not working for whatever reason, then the Channel
will fill up.... right?

Once the Channel is full, what happens with new data that comes from the
file being tailed?  Can it be blocked?  I doubt it, because tailing is done
with exec + tail -F, and I think that can't be blocked.... but who knows!

Thanks,
Otis
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/