I am exploring kafka 0.8 beta release to understand data flow, replication features. While testing i found that, the partition data for data for deleted topic is preseved in kafka-logs, why this behavior? suppose below case,
A topic (suppose test1) is created with partition 6 and replication 3 on a system with 4 brokers, respective log and index files will be prepared per partition in kafka-logs. If I delete the topic and recreate the same topic ‘test1’ after some time with partition 2 and replication 2. The kafka-logs directory seems to be confusing to understand why the partitions for previous topic are present. *Please help to understand this scenario*.
Also, while testing the replication and leader selection feature observed leader -1 status,
As per kafka protocol guide, *leader: -1 means If no leader exists because we are in the middle of a leader election this id will be -1.*
*Does it mean that, the data from partition 0 will be unavailable due no leader (leader selection in progress)?* As per my understanding, can we have auto re-balancer facility to re-balance the partition replications to available brokers if one of the broker is down, as in above case of (if leader 4 goes down), we can replicate the partition 0 to broker 0/1 to re-balance the replication.
Please correct me for any wrong understanding as those are my initial observations.
Thanks in advance.
Thanks, Yogesh Sangvikar
Yogesh Sangvikar 2013-06-18, 13:26
Re: Partition data for deleted topic found in kafka-logs, also, found leader: -1
We never tried manipulating ZK directly. So it may or may not work. Our plan is to have tools to change # partitions, replication factor, etc in the future.
Jun On Wed, Jun 19, 2013 at 6:18 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
Jun Rao 2013-06-20, 04:02
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation projects and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext