Flume, mail # user - fileChannel always reached it's capacity - 2014-03-21, 09:03
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
fileChannel always reached it's capacity
I use hdfsSink and fileChannel, it seems hdfsSink consume too slow.here is
my configuration:
...
agent1.channels.thrift_ch2.capacity = 200000000
agent1.channels.thrift_ch2.transactionCapacity = 100000
...
agent1.sinks.hdfsSink.hdfs.batchSize = 100000
agent1.sinks.hdfsSink.hdfs.threadsPoolSize = 40

very appreciate for guiding !!!

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB