Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Disqualify a node from leader election


+
Owen Kim 2013-11-22, 02:10
+
FPJ 2013-11-22, 10:18
+
Owen Kim 2013-11-25, 01:09
+
Benjamin Reed 2013-11-25, 01:20
Copy link to this message
-
RE: Disqualify a node from leader election
D and E can only become a leader if they can form a quorum with B and/or C
(assuming you've taken A down). In the case A, D, E are the only 3 servers
able to talk to each other, killing A is going to make you lose quorum, and
if you bring A back up, it might be elected again, so you might end up in
this cycle until the net partition is healed or B and/or C are up.

I suppose we could have an option that tells servers to ignore a given
server when electing a leader, but it is not entirely trivial because A in
the example Ben gave might be the only available server that has the latest
committed txn. We would need a mechanism to transfer state from a follower
to a prospective leader.

-Flavio

> -----Original Message-----
> From: Benjamin Reed [mailto:[EMAIL PROTECTED]]
> Sent: 25 November 2013 01:21
> To: [EMAIL PROTECTED]
> Subject: Re: Disqualify a node from leader election
>
> camille really has the right solution. you have to let it become the
leader and
> then kill it. here is why:
>
> lets says you have servers: A, B, C, D, and E and A is the node that you
don't
> want to be the leader. let's also say that C is a leader and commits
transaction
> x on A, B, and C but before D and E get x B and C fail. now A is the only
> surviving node with x, so unless A can become the leader, at least
> temporarily, D and E will never get x. if you follow Camille's suggestion,
you
> let A become the leader (since it has the most recent
> transaction) and D and E will sync with A and get x. now if you restart A,
> leader election will run again and D or E can be elected leader. this does
also
> show that you want the less desirable nodes to have lower server ids so
that
> there will be less chance of them becoming leader if there is a "tie"
(other
> nodes are just as uptodate as them).
>
> ben
>
>
> On Sun, Nov 24, 2013 at 5:09 PM, Owen Kim <[EMAIL PROTECTED]> wrote:
>
> > Observers can't vote in leader election, though right? I'm not sure
> > the loss of fault tolerance would be worth it.
> >
> > The scenario is that I have a 5-node cluster but one node is in an
> > network partition that gets DOSed around it. When this happens and
> > it's leader, I see huge performance degradation. The real solution is
> > obviously to move the node off that network but I wondered if there
> > was an easy way to keep it from being leader in its configuration.
> >
+
Kuba Lekstan 2013-11-25, 15:54
+
FPJ 2013-11-25, 16:12
+
Kuba Lekstan 2013-11-25, 18:28
+
Camille Fournier 2013-11-22, 02:39
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB