Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> The size of data folder continue grow


Copy link to this message
-
The size of data folder continue grow
Dear all,

I use flume and custom base sink to put data to HBase. In flume, I use file channel.

the file channel data put in the following folder.

razor.channels.c_error.dataDirs = /var/lib/flume-ng/data/error

But I see the channel data folder size continue grow until to 2gb.

Does flume remove the data in channel data folder after flush to HBase? Or still store in data folder after flush to HBase ?

Or the speed sink to HBase is too slower than source ?

And how to speed up the sink to HBase ?
Thanks

Brad
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB