I have questions about the load balancing of kafka high-level consumer Suppose I have 4 partition And the producer throughput to these 4 partitions are like this 0 1 2 3 10MB/s 10MB/s 1MB/s 1MB/s 1kMsg/s, 10kMsg/s 1kMsg/s 10kMsg/s
And I have only 2 consumers, what partitions will the 2 consumers pick up?
If the throughput is changing dynamically, will the consumer re-balancing accordingly?
The consumer load balancing logic today is pretty simple. It just tries to divide the partitions evenly among the consumers. It doesn't try to balance by load.
Jun On Thu, Nov 14, 2013 at 3:34 PM, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext