Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Of BatchSize / Channel Capacity / Transaction Capacity


Copy link to this message
-
Of BatchSize / Channel Capacity / Transaction Capacity
Bhaskar V. Karambelkar 2013-01-08, 18:46
Can some one explain the importance of the following
1) Batch Size
  1.a) When configured by client code using the flume-core-sdk , to send
events to flume avro source.
  1.b) When set as a parameter on HDFS sink (or other sinks which support
BatchSize parameter)
2)
  2.a) Channel Capacity
  2.b) Channel Transaction Capacity.
Under which conditions should these params be set to high values, and under
which conditions should they be set to low values.
How will setting these parameters to different values, affect throughput,
latency in event flow.
Specifically if we have clients with varying frequency of event generation,
i.e. some clients generating thousands of events/sec, while
others at a much slower rate, what effect will different values of these
params have on these clients ?

thanks
Bhaskar