Kafka, mail # user - Re: Relative cluster sizes and cluster size limits - 2013-08-03, 02:50
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
Re: Relative cluster sizes and cluster size limits
Hi Scott,

What version of Kafka is this?

In general our throughput will scale linearly with the number of machines
or more specifically the number of disks. Our bottleneck will really be
with the number of partitions. With thousands of partitions leader election
can get slower (seconds), and if you have consumers that consume all
partitions the rebalancing in these consumers can get slow (minutes).

We hope to fix these issues but that is the current state up through 0.8.

On Fri, Aug 2, 2013 at 2:27 PM, Scott Arthur <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB