Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - 0.8 best practices for migrating / electing leaders in failure situations?


Copy link to this message
-
Re: 0.8 best practices for migrating / electing leaders in failure situations?
Scott Clasen 2013-03-22, 21:17
Thanks Neha-

To Clarify...

*In scenario => 1 will the new broker get all messages on the other brokers
replicated to it?

*In Scenario 2 => it is clear that the data is gone, but I still need
producers to be able to send and consumers to receive on the same topic. In
my testing today I was unable to do that as I kept getting errors...so if i
was doing the correct steps it seems there is a bug here, basically the
"second-cluster-topic" topic is unusable after all 3 brokers crash, and 3
more are booted to replace them.  Something not quite correct in zookeeper?

Like so

./bin/kafka-reassign-partitions.sh --zookeeper ... --path-to-json-file
reassign.json

kafka.common.LeaderNotAvailableException: Leader not available for topic
second-cluster-topic partition 0
at kafka.admin.AdminUtils$$anonfun$3.apply(AdminUtils.scala:120)
at kafka.admin.AdminUtils$$anonfun$3.apply(AdminUtils.scala:103)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
at scala.collection.immutable.List.foreach(List.scala:45)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
at scala.collection.immutable.List.map(List.scala:45)
at
kafka.admin.AdminUtils$.kafka$admin$AdminUtils$$fetchTopicMetadataFromZk(AdminUtils.scala:103)
at kafka.admin.AdminUtils$.fetchTopicMetadataFromZk(AdminUtils.scala:92)
at kafka.admin.ListTopicCommand$.showTopic(ListTopicCommand.scala:80)
at
kafka.admin.ListTopicCommand$$anonfun$main$2.apply(ListTopicCommand.scala:66)
at
kafka.admin.ListTopicCommand$$anonfun$main$2.apply(ListTopicCommand.scala:65)
at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
at scala.collection.immutable.List.foreach(List.scala:45)
at kafka.admin.ListTopicCommand$.main(ListTopicCommand.scala:65)
at kafka.admin.ListTopicCommand.main(ListTopicCommand.scala)
Caused by: kafka.common.LeaderNotAvailableException: No leader exists for
partition 0
at kafka.admin.AdminUtils$$anonfun$3.apply(AdminUtils.scala:117)
... 16 more
topic: second-cluster-topic

./bin/kafka-preferred-replica-election.sh  --zookeeper...
--path-to-json-file elect.json
....[2013-03-22 10:24:20,706] INFO Created preferred replica election path
with { "partitions":[ { "partition":0, "topic":"first-cluster-topic" }, {
"partition":0, "topic":"second-cluster-topic" } ], "version":1 }
(kafka.admin.PreferredReplicaLeaderElectionCommand$)

./bin/kafka-list-topic.sh  --zookeeper ... --topic second-cluster-topic

2013-03-22 10:24:30,869] ERROR Error while fetching metadata for partition
[second-cluster-topic,0] (kafka.admin.AdminUtils$)
kafka.common.LeaderNotAvailableException: Leader not available for topic
second-cluster-topic partition 0
at kafka.admin.AdminUtils$$anonfun$3.apply(AdminUtils.scala:120)
at kafka.admin.AdminUtils$$anonfun$3.apply(AdminUtils.scala:103)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
at scala.collection.immutable.List.foreach(List.scala:45)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
at scala.collection.immutable.List.map(List.scala:45)
at
kafka.admin.AdminUtils$.kafka$admin$AdminUtils$$fetchTopicMetadataFromZk(AdminUtils.scala:103)
at kafka.admin.AdminUtils$.fetchTopicMetadataFromZk(AdminUtils.scala:92)
at kafka.admin.ListTopicCommand$.showTopic(ListTopicCommand.scala:80)
at
kafka.admin.ListTopicCommand$$anonfun$main$2.apply(ListTopicCommand.scala:66)
at
kafka.admin.ListTopicCommand$$anonfun$main$2.apply(ListTopicCommand.scala:65)
at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
at scala.collection.immutable.List.foreach(List.scala:45)
at kafka.admin.ListTopicCommand$.main(ListTopicCommand.scala:65)
at kafka.admin.ListTopicCommand.main(ListTopicCommand.scala)
Caused by: kafka.common.LeaderNotAvailableException: No leader exists for
partition 0
at kafka.admin.AdminUtils$$anonfun$3.apply(AdminUtils.scala:117)
... 16 more

On Fri, Mar 22, 2013 at 1:12 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote: