Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Questions about the leader election


+
James Wu 2013-08-23, 03:51
+
Neha Narkhede 2013-08-23, 04:00
+
James Wu 2013-08-23, 10:38
Copy link to this message
-
Re: Questions about the leader election
ZK paths for 0.8 are documented in
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper

If a controller fails, any live broker can become the leader. This is
coordinated through an ephemeral path in ZK.

Thanks,

Jun
On Fri, Aug 23, 2013 at 3:37 AM, James Wu <[EMAIL PROTECTED]> wrote:

> Thank you Neha, it's very helpful information!
>
> I also read this article
>
> http://engineering.linkedin.com/kafka/intra-cluster-replication-apache-kafka
> For the section "Handling Failures", I am wondering some questions:
>
> 1. "The leader and the ISR for each partition are also stored in Zookeeper
> and are used during the failover of the controller." What path does it be
> stored ? Is there any way to see the information on zookeeper ?
>
> 2. If the controller fail, how does the new controller be elected ? Does it
> be elected by Zookeeper ? How does Zookeeper decide which node should be
> the controller ?
>
> Many thanks!
>
>
>
>
> On Fri, Aug 23, 2013 at 11:59 AM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
>
> > The replication state machine and leader election mechanism is described
> > here - http://kafka.apache.org/documentation.html#replication
> >
> > Let us know how the docs can be improved.
> >
> > Thanks,
> > Neha
> >
> >
> > On Thu, Aug 22, 2013 at 8:51 PM, James Wu <[EMAIL PROTECTED]> wrote:
> >
> > > Hi,
> > >
> > > I am wondering what is the mechanism that Kafka elects the leader of
> > > partitions ?
> > >
> > > Does it handle by the controller process or ?
> > > If the leader crashed, who will decide the new leader ? and the process
> > is
> > > running on Zookeeper or Kafka?
> > >
> > >
> > > Thanks.
> > >
> > > --
> > > --
> > > Friendly regards,
> > >
> > > *James Wu <http://www.facebook.com/jameswu629>*
> > >
> >
>
>
>
> --
>
> Friendly regards,
>
> *James Wu <https://plus.google.com/u/0/100829801349304669533>
> *
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB