Do you guys have any stats on what kind of load was reduced on ZK? Just trying to understand if this changes the type of servers required to host ZK. On Wed, Jun 4, 2014 at 1:10 PM, Guozhang Wang <[EMAIL PROTECTED]> wrote:
At LinkedIn, we are just beginning to move our applications to Kafka based offset management. I'm not sure if Todd/Clark/Joel have any numbers to share right away, but depending on the number of consumers you have and the number of topics those consumers pull, it will likely make a sizable impact on the performance of your zookeeper cluster. It will not, however, influence the decision of sizing the zookeeper cluster since zookeeper writes do not scale with the number of servers in a zookeeper cluster. In general, it is recommended to keep a zookeeper cluster small, of the size of 3 or 5 servers. i.e. whether or not your consumers checkpoint offsets to zookeeper or Kafka. It is true that the writes will be faster on a 3 node zookeeper cluster vs a 5 node zookeeper cluster.
Thanks Neha On Wed, Jun 4, 2014 at 11:15 AM, S Ahmed <[EMAIL PROTECTED]> wrote:
This will force a rewrite of those monitoring tools and UI tools that read offsets from Zookeeper in order directly to get lag information for reporting on consumer clients. It seems a good thing to know this is coming down the pipe.
Dennis On Wed, Jun 4, 2014 at 6:50 PM, Neha Narkhede <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext