Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: Unit test failure


Copy link to this message
-
Re: Unit test failure
On 11/30/2011 01:02 AM, Jay Kreps wrote:
> I think this is one of the core problems of this type of system--how to do
> good, realistic distributed testing. I have not seen anyone really succeed
> except via the "throw production load at it and see" approach so I would be
> interested to hear people's thoughts.

Cassandra has the same problem and has gone through several iterations
of distributed test systems.  The most recent is here:
https://github.com/riptano/cassandra-dtest  Time will tell if this
attempt is more successful.  The obvious prerequisite for something like
this is a set of scripts that make it easy to set up Kafka clusters of
different topologies for testing.

Another thing they do well is having a solid stress/perf tool
distributed with releases, everyone agrees it gives decent results, easy
to hack if you want to test something slightly different and thus it's
widely used.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB