Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Of BatchSize / Channel Capacity / Transaction Capacity


Copy link to this message
-
Of BatchSize / Channel Capacity / Transaction Capacity
Can some one explain the importance of the following
1) Batch Size
  1.a) When configured by client code using the flume-core-sdk , to send
events to flume avro source.
  1.b) When set as a parameter on HDFS sink (or other sinks which support
BatchSize parameter)
2)
  2.a) Channel Capacity
  2.b) Channel Transaction Capacity.
Under which conditions should these params be set to high values, and under
which conditions should they be set to low values.
How will setting these parameters to different values, affect throughput,
latency in event flow.
Specifically if we have clients with varying frequency of event generation,
i.e. some clients generating thousands of events/sec, while
others at a much slower rate, what effect will different values of these
params have on these clients ?

thanks
Bhaskar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB