Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # dev - Does Flume need a workload generator?

Copy link to this message
Does Flume need a workload generator?
Patrick Wendell 2012-07-17, 04:28
Hi All,

I was trying to debug a memory leak and it doesn't seem like there is
a good way to generate workload for Flume at a configurable rate, or
with configurable characteristics like payload size and header size.
The sequence source is something like this, but you can't control the
rate of event generation at all.

What do people do now when benchmarking or debugging Flume? I was
thinking of writing a source that generates event streams with
configurable properties. Would that be useful? Or is this need already
met somehow else?

- Patrick