On 11/30/2011 01:02 AM, Jay Kreps wrote:
> I think this is one of the core problems of this type of system--how to do
> good, realistic distributed testing. I have not seen anyone really succeed
> except via the "throw production load at it and see" approach so I would be
> interested to hear people's thoughts.
Cassandra has the same problem and has gone through several iterations
of distributed test systems. The most recent is here:
https://github.com/riptano/cassandra-dtest Time will tell if this
attempt is more successful. The obvious prerequisite for something like
this is a set of scripts that make it easy to set up Kafka clusters of
different topologies for testing.
Another thing they do well is having a solid stress/perf tool
distributed with releases, everyone agrees it gives decent results, easy
to hack if you want to test something slightly different and thus it's