I have a cluster of 12 brokers receiving 10,000 msg/s from producers where each message is roughly 2.5KB. We also have 12 ZooKeepers and everything is on AWS. Under these conditions, top (the Linux utility) reports around 10-15 out of 32 for system load, so we’re at less than half capacity.
When under this load consumers take a very long time, often more than 30 minutes, to connect to the brokers. When under no load they connect immediately. Why is this happening?
Tim Kellogg Sr. Software Engineer, Protocols 2lemetry @kellogh
Tim Kellogg 2014-01-11, 02:03
Re: Consumers can't connect while broker is under load
12 zookeepers seems like a lot......and you should always, by default, prefer an odd number of zookeepers. Consumers negotiate with each other for partition ownership, via zookeeper.....
On Fri, Jan 10, 2014 at 9:20 PM, Guozhang Wang <[EMAIL PROTECTED]> wrote:
Jason Rosenberg 2014-01-14, 10:48
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation projects and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext