Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Of BatchSize / Channel Capacity / Transaction Capacity


Copy link to this message
-
Of BatchSize / Channel Capacity / Transaction Capacity
Can some one explain the importance of the following
1) Batch Size
  1.a) When configured by client code using the flume-core-sdk , to send
events to flume avro source.
  1.b) When set as a parameter on HDFS sink (or other sinks which support
BatchSize parameter)
2)
  2.a) Channel Capacity
  2.b) Channel Transaction Capacity.
Under which conditions should these params be set to high values, and under
which conditions should they be set to low values.
How will setting these parameters to different values, affect throughput,
latency in event flow.
Specifically if we have clients with varying frequency of event generation,
i.e. some clients generating thousands of events/sec, while
others at a much slower rate, what effect will different values of these
params have on these clients ?

thanks
Bhaskar
+
Jeff Lord 2013-01-09, 02:40
+
Bhaskar V. Karambelkar 2013-01-11, 14:48
+
Jeff Lord 2013-01-11, 17:03
+
Alexander Alten-Lorenz 2013-01-12, 09:05
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB