When I delete a topic, the topic is deleted from zookeeper but its log files are not deleted from Brokers.
When I restart a broker, the broker will try to sync the log files whose topic has been deleted. Manually deleting the log files will resolve the issue. Should broker ignore log files whose topic has been deleted?
The JIRA tracking this feature is https://issues.apache.org/jira/browse/KAFKA-330. Until we have this feature, we should remove any scripts that give the impression of deleting a topic, as Jay suggested. I tried looking under bin/ but couldn't find any delete topic or partition script.
Can you tell us which script you are using?
Thanks, Neha On Thu, Aug 22, 2013 at 11:40 AM, Vadim Keylis <[EMAIL PROTECTED]>wrote:
In production, we are not deleting topics yet. In test environments, if we have to delete topics, we wipe out the cluster. If this is a feature that most users are blocked on, I think it makes sense to prioritize https://issues.apache.org/jira/browse/KAFKA-330.
Thanks, Neha On Thu, Aug 22, 2013 at 1:01 PM, Vadim Keylis <[EMAIL PROTECTED]> wrote:
By that, I meant bringing down all brokers, deleting zookeeper data and kafka data and restarting the brokers. I suspect attempting a delete topic might have caused something bad to happen on the ISR side. It will be great if you can start clean and then see if the ISR is still an issue.
Without the proper functionality of delete topic, I'm not sure if deleting the zookeeper and kafka data just for that topic is enough or not. On the surface, this approach seems sufficient, but I'm not sure of all the consequences of doing that. Also, remember to bounce the entire cluster once you've deleted the zookeeper and kafka data for the topic in question.
Can you give it a try and let us know how it went?
Thanks, Neha On Fri, Aug 23, 2013 at 10:15 AM, Yu, Libo <[EMAIL PROTECTED]> wrote:
Here is the list of paths that should be deleted: /brokers/topics/my_topic /consumers/[groupId]/owners/my_topic /consumers/[groupId]/offsets/my_topic (if the topic has been consumed) Let me know if I miss anything. Thanks.
I mean I think the meta point Neha is making, is that clearly we need tooling support for topic deletion. We just haven't done it yet. This is kind of silly but works okay for us temporarily (we don't really delete topics).
-Jay On Fri, Aug 23, 2013 at 2:22 PM, Yu, Libo <[EMAIL PROTECTED]> wrote:
I maintain a kafka wrapper shell script wrapper for our Kafka packaging. I've noticed that 0.8 has several bin/*topic.sh scripts, while trunk has as single bin/kafka-topics.sh, which uses the TopicCommand class.
If you are discussing support for delete-topics in 0.8, perhaps the TopicCommand class and script that trunk has should be merged into 0.8 sooner rather than later? (I'm really only mentioning this because TopicCommand makes my wrapper script a bit simpler. :) )
On Aug 23, 2013, at 5:28 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:
Search for *-ConsumerLag under kafka.server. There is a jmx bean per partition in the follower.
Jun On Fri, Aug 23, 2013 at 7:59 PM, Vadim Keylis <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation projects and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext