Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Flume performance tuning


Copy link to this message
-
Flume performance tuning
Hi.

I did benchmark of the flume. 3~8ms in one time....
Fluentd is 0.1ms~1ms in one time. So, I want to performance tuning.
Please give me some advise!
[conf/flume-conf.properties]-------
agent.sources = r1
agent.sinks = k1
agent.channels = c1

agent.sources.r1.type = avro
agent.sources.r1.bind = 0.0.0.0
agent.sources.r1.port = 44444

agent.sinks.k1.type = avro
agent.sinks.k1.hostname = xxx.xxx.xxx.xxx
agent.sinks.k1.port = 44444

agent.channels.c1.type = memory
agent.channels.c1.capacity = 1000
agent.channels.c1.transactionCapacity = 100

agent.sources.r1.channels = c1
agent.sinks.k1.channel = c1
---------------------------------------------------
+
Brock Noland 2013-02-21, 13:22
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB