I am reading the design doc for Kafka. But I can not find the algorithm which illustrates such scenario. When a new broker is registered in zookeeper, how do all the brokers negotiates with all the others? How does a new broker decide which partitions should be stored ?
By the way, I find num.partitions in server configure, but I can not find any parameter defining the topics of this broker.
Could you tell me how to configure brokers in the following example. For example, I want to set "broker1" to store "topic1" and "topic2", each topic has 3 partitions? I think setting topic.partition.count.map as "topic1:3, topic2:3" should works. Is there any simpler configuration? Thanks.
Every topic in one broker has num.partitions partitions if we only set num.partitions
When a new broker is added, if there is topic data folder under kafka-logs folder, then the broker will register num.partitions of this topic in zk, else the broker doesn't know which topic should be registered. If no brokers register the topic(topic data folder is not existed in any broker), consumer will fall in bootstrap, assuming each broker has 1 partition. The broker will register num.partitions of this topic in zk, after it receive this topic message .
If only a few brokers haven't register the topic in zk, I think these brokers may not actually work for this topic.
Please correct me if I am wrong.
On 12-12-28 上午10:30, "王国栋" <[EMAIL PROTECTED]> wrote:
Thanks yonghui. That makes sense. On Mon, Dec 31, 2012 at 1:45 AM, 永辉 赵 <[EMAIL PROTECTED]> wrote:
Guodong Wang 王国栋
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext