Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - long running (continous) benchmark test


Copy link to this message
-
Re: long running (continous) benchmark test
Jay Kreps 2013-02-07, 03:26
For our long running testing at LinkedIn we actually just use the mirror
maker tool and fork off a copy of the full production streams and throw it
at the test cluster. This is, of course, only useful if you have a
high-load cluster to replay...

-Jay
On Wed, Feb 6, 2013 at 11:27 AM, S Ahmed <[EMAIL PROTECTED]> wrote:

> Another reason is to test kafka will potentially larger payloads (10-100K)
>
>
> On Wed, Feb 6, 2013 at 2:26 PM, S Ahmed <[EMAIL PROTECTED]> wrote:
>
> > I plan on creating some sort of a long running test with kafka, curiuos
> if
> > anyone has already done this and released the code.
> >
> > I want to setup some ec2 instances, and mimick what I think will be close
> > to real life usage, and monitor how things work.  I want this to run for
> 24
> > hours or more.
> >
> > I know there are system tests, but I believe those are short lived tests.
> >
> > It is always interesting to see how things work like if consumers fall
> > behind, etc.
> >
>