-Re: Does Flume need a workload generator?
Mike Percy 2012-07-17, 04:41
Will and I have put together a couple of classes that do much (not all) of
what you're describing for a Syslog TCP client and an Avro RPC client. I've
been meaning to commit them to Flume SVN but I haven't gotten around to it
yet. Here is the syslog one:
I can give you an Avro RPC one which is similar.
On Mon, Jul 16, 2012 at 9:28 PM, Patrick Wendell <[EMAIL PROTECTED]> wrote:
> Hi All,
> I was trying to debug a memory leak and it doesn't seem like there is
> a good way to generate workload for Flume at a configurable rate, or
> with configurable characteristics like payload size and header size.
> The sequence source is something like this, but you can't control the
> rate of event generation at all.
> What do people do now when benchmarking or debugging Flume? I was
> thinking of writing a source that generates event streams with
> configurable properties. Would that be useful? Or is this need already
> met somehow else?
> - Patrick