Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> 0.8 best practices for migrating / electing leaders in failure situations?


Copy link to this message
-
Re: 0.8 best practices for migrating / electing leaders in failure situations?
Jun Thanks. To clarify, do you mean that clients will have cached broker
lists or some other data that will make them ignore the new brokers?

Like so

topic-1 replication factor 3, on broker-ids 1,2,3
all brokers 1,2,3 die, and are never coming back.
delete all kafka data in zookeeper.
boot 4,5,6, create new topic called topic-1 repl factor 3, brokers 4,5,6

clients will/will not start sending to topic-1 on 4,5,6?

On Sun, Mar 24, 2013 at 4:01 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> If you bring up 3 new brokers with different broker ids, you won't be able
> to use them on existing topics until after you have run the partition
> reassignment tool.
>
> Thanks,
>
> Jun
>
> On Fri, Mar 22, 2013 at 9:23 PM, Scott Clasen <[EMAIL PROTECTED]> wrote:
>
> > Thanks!
> >
> >  Would there be any difference if I instead  deleted all the Kafka data
> > from zookeeper and booted 3 instances  with different broker id? clients
> > with cached broker id lists or any other issue?
> >
> > Sent from my iPhone
> >
> > On Mar 22, 2013, at 9:15 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> > > In scenario 2, you can bring up 3 new brokers with the same broker id.
> > You
> > > won't get the data back. However, new data can be published to and
> > consumed
> > > from the new brokers.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Fri, Mar 22, 2013 at 2:17 PM, Scott Clasen <[EMAIL PROTECTED]>
> wrote:
> > >
> > >> Thanks Neha-
> > >>
> > >> To Clarify...
> > >>
> > >> *In scenario => 1 will the new broker get all messages on the other
> > brokers
> > >> replicated to it?
> > >>
> > >> *In Scenario 2 => it is clear that the data is gone, but I still need
> > >> producers to be able to send and consumers to receive on the same
> > topic. In
> > >> my testing today I was unable to do that as I kept getting errors...so
> > if i
> > >> was doing the correct steps it seems there is a bug here, basically
> the
> > >> "second-cluster-topic" topic is unusable after all 3 brokers crash,
> and
> > 3
> > >> more are booted to replace them.  Something not quite correct in
> > zookeeper?
> > >>
> > >> Like so
> > >>
> > >> ./bin/kafka-reassign-partitions.sh --zookeeper ... --path-to-json-file
> > >> reassign.json
> > >>
> > >> kafka.common.LeaderNotAvailableException: Leader not available for
> topic
> > >> second-cluster-topic partition 0
> > >> at kafka.admin.AdminUtils$$anonfun$3.apply(AdminUtils.scala:120)
> > >> at kafka.admin.AdminUtils$$anonfun$3.apply(AdminUtils.scala:103)
> > >> at
> > >>
> > >>
> >
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
> > >> at
> > >>
> > >>
> >
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
> > >> at
> > >>
> > >>
> >
> scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
> > >> at scala.collection.immutable.List.foreach(List.scala:45)
> > >> at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
> > >> at scala.collection.immutable.List.map(List.scala:45)
> > >> at
> > >>
> > >>
> >
> kafka.admin.AdminUtils$.kafka$admin$AdminUtils$$fetchTopicMetadataFromZk(AdminUtils.scala:103)
> > >> at
> kafka.admin.AdminUtils$.fetchTopicMetadataFromZk(AdminUtils.scala:92)
> > >> at kafka.admin.ListTopicCommand$.showTopic(ListTopicCommand.scala:80)
> > >> at
> > >>
> > >>
> >
> kafka.admin.ListTopicCommand$$anonfun$main$2.apply(ListTopicCommand.scala:66)
> > >> at
> > >>
> > >>
> >
> kafka.admin.ListTopicCommand$$anonfun$main$2.apply(ListTopicCommand.scala:65)
> > >> at
> > >>
> > >>
> >
> scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
> > >> at scala.collection.immutable.List.foreach(List.scala:45)
> > >> at kafka.admin.ListTopicCommand$.main(ListTopicCommand.scala:65)
> > >> at kafka.admin.ListTopicCommand.main(ListTopicCommand.scala)
> > >> Caused by: kafka.common.LeaderNotAvailableException: No leader exists
> > for
> > >> partition 0
> > >> at kafka.admin.AdminUtils$$anonfun$3.apply(AdminUtils.scala:117)

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB