Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> 0.8 best practices for migrating / electing leaders in failure situations?


Copy link to this message
-
Re: 0.8 best practices for migrating / electing leaders in failure situations?
You can't create the topic with the same name before deleting it first.

Thanks,

Jun

On Mon, Mar 25, 2013 at 9:58 AM, Scott Clasen <[EMAIL PROTECTED]> wrote:

> Jun Thanks. To clarify, do you mean that clients will have cached broker
> lists or some other data that will make them ignore the new brokers?
>
> Like so
>
> topic-1 replication factor 3, on broker-ids 1,2,3
> all brokers 1,2,3 die, and are never coming back.
> delete all kafka data in zookeeper.
> boot 4,5,6, create new topic called topic-1 repl factor 3, brokers 4,5,6
>
> clients will/will not start sending to topic-1 on 4,5,6?
>
>
>
> On Sun, Mar 24, 2013 at 4:01 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > If you bring up 3 new brokers with different broker ids, you won't be
> able
> > to use them on existing topics until after you have run the partition
> > reassignment tool.
> >
> > Thanks,
> >
> > Jun
> >
> > On Fri, Mar 22, 2013 at 9:23 PM, Scott Clasen <[EMAIL PROTECTED]> wrote:
> >
> > > Thanks!
> > >
> > >  Would there be any difference if I instead  deleted all the Kafka data
> > > from zookeeper and booted 3 instances  with different broker id?
> clients
> > > with cached broker id lists or any other issue?
> > >
> > > Sent from my iPhone
> > >
> > > On Mar 22, 2013, at 9:15 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > >
> > > > In scenario 2, you can bring up 3 new brokers with the same broker
> id.
> > > You
> > > > won't get the data back. However, new data can be published to and
> > > consumed
> > > > from the new brokers.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Fri, Mar 22, 2013 at 2:17 PM, Scott Clasen <[EMAIL PROTECTED]>
> > wrote:
> > > >
> > > >> Thanks Neha-
> > > >>
> > > >> To Clarify...
> > > >>
> > > >> *In scenario => 1 will the new broker get all messages on the other
> > > brokers
> > > >> replicated to it?
> > > >>
> > > >> *In Scenario 2 => it is clear that the data is gone, but I still
> need
> > > >> producers to be able to send and consumers to receive on the same
> > > topic. In
> > > >> my testing today I was unable to do that as I kept getting
> errors...so
> > > if i
> > > >> was doing the correct steps it seems there is a bug here, basically
> > the
> > > >> "second-cluster-topic" topic is unusable after all 3 brokers crash,
> > and
> > > 3
> > > >> more are booted to replace them.  Something not quite correct in
> > > zookeeper?
> > > >>
> > > >> Like so
> > > >>
> > > >> ./bin/kafka-reassign-partitions.sh --zookeeper ...
> --path-to-json-file
> > > >> reassign.json
> > > >>
> > > >> kafka.common.LeaderNotAvailableException: Leader not available for
> > topic
> > > >> second-cluster-topic partition 0
> > > >> at kafka.admin.AdminUtils$$anonfun$3.apply(AdminUtils.scala:120)
> > > >> at kafka.admin.AdminUtils$$anonfun$3.apply(AdminUtils.scala:103)
> > > >> at
> > > >>
> > > >>
> > >
> >
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
> > > >> at
> > > >>
> > > >>
> > >
> >
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
> > > >> at
> > > >>
> > > >>
> > >
> >
> scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
> > > >> at scala.collection.immutable.List.foreach(List.scala:45)
> > > >> at
> > scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
> > > >> at scala.collection.immutable.List.map(List.scala:45)
> > > >> at
> > > >>
> > > >>
> > >
> >
> kafka.admin.AdminUtils$.kafka$admin$AdminUtils$$fetchTopicMetadataFromZk(AdminUtils.scala:103)
> > > >> at
> > kafka.admin.AdminUtils$.fetchTopicMetadataFromZk(AdminUtils.scala:92)
> > > >> at
> kafka.admin.ListTopicCommand$.showTopic(ListTopicCommand.scala:80)
> > > >> at
> > > >>
> > > >>
> > >
> >
> kafka.admin.ListTopicCommand$$anonfun$main$2.apply(ListTopicCommand.scala:66)
> > > >> at
> > > >>
> > > >>
> > >
> >
> kafka.admin.ListTopicCommand$$anonfun$main$2.apply(ListTopicCommand.scala:65)
> > > >> at
> > > >>
> > > >>
> > >
> >
> scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)