Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> The size of data folder continue grow


Copy link to this message
-
The size of data folder continue grow
Dear all,

I use flume and custom base sink to put data to HBase. In flume, I use file channel.

the file channel data put in the following folder.

razor.channels.c_error.dataDirs = /var/lib/flume-ng/data/error

But I see the channel data folder size continue grow until to 2gb.

Does flume remove the data in channel data folder after flush to HBase? Or still store in data folder after flush to HBase ?

Or the speed sink to HBase is too slower than source ?

And how to speed up the sink to HBase ?
Thanks

Brad