Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> long running (continous) benchmark test


+
S Ahmed 2013-02-06, 19:27
+
S Ahmed 2013-02-06, 19:27
Copy link to this message
-
Re: long running (continous) benchmark test
For our long running testing at LinkedIn we actually just use the mirror
maker tool and fork off a copy of the full production streams and throw it
at the test cluster. This is, of course, only useful if you have a
high-load cluster to replay...

-Jay
On Wed, Feb 6, 2013 at 11:27 AM, S Ahmed <[EMAIL PROTECTED]> wrote:

> Another reason is to test kafka will potentially larger payloads (10-100K)
>
>
> On Wed, Feb 6, 2013 at 2:26 PM, S Ahmed <[EMAIL PROTECTED]> wrote:
>
> > I plan on creating some sort of a long running test with kafka, curiuos
> if
> > anyone has already done this and released the code.
> >
> > I want to setup some ec2 instances, and mimick what I think will be close
> > to real life usage, and monitor how things work.  I want this to run for
> 24
> > hours or more.
> >
> > I know there are system tests, but I believe those are short lived tests.
> >
> > It is always interesting to see how things work like if consumers fall
> > behind, etc.
> >
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB