Hi,

I read the docs and presentations online
https://cwiki.apache.org/ZOOKEEPER/zookeeperpresentations.html about the
internals of zab. I am confused about what happens to uncommitted proposals
when new leader is elected after current leader fails. To be precise, these
are the scenarios i am thinking

a) new leader has got a proposal that no other follower has seen, will it
commit this proposal or let it go
b) new leader has got a proposal that majority of nodes have it, will it
commit this or let it go. What happens to other followers, will new leader
ask followers to commit or rollback based on what it does.
c)new leader does not have proposal but one of the followers has it, is
this possible?

>From what i understood the new leader simply commits all proposals that he
has seen irrespective of  what other followers have seen.

Can some one explain what exactly happens, i am more interested in what
happens in the code

thanks,
Kishore G

Hi Kishore,

Here's more or less what happens.

Basically, if an operation is uncommitted but reached a quorum, it is
guaranteed to survive. If it reached less than a quorum it may survive
but may get lost depending on whether the server that has it
participates in a timely manner in leader election phases.

First, FastLeaderElection tries to optimistically choose a leader
candidate that has the most up to date log - a quorum must say that it
does. This phase is an optimization to avoid having the leader fetch
the latest state (this way the leader already has it).

Then, servers connect to the candidate leader (LearnerHandler). If
during this phase the leader discovers someone that is more up-to-date
than itself, it gives up and we go back to the preliminary phase
above. Otherwise, once it has a quorum of followers connected to it,
it basically tries to commit its own log (NEWLEADER is like a propose
and UPTODATE is like a commit of the log).
The NEWLEADER message has a zxid which is higher than any zxid existed before.

It is possible that a minority of the servers didn't participate in
this process, but once NEWLEADER reaches a quorum, servers in this
quorum will be "more up-to-date" than anyone that didn't participate.
So any servers that were lagging behind do not stand a chance becoming
leader unless they get updated from a server that did participate. Any
uncommitted op known to these slow servers will be lost forever.
Later, if such a server connects to the new leader, the uncommitted
ops will be truncated from the log.

Hope this helps.

Alex
On Wed, Dec 12, 2012 at 10:31 PM, kishore g <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I read the docs and presentations online
> https://cwiki.apache.org/ZOOKEEPER/zookeeperpresentations.html about the
> internals of zab. I am confused about what happens to uncommitted proposals
> when new leader is elected after current leader fails. To be precise, these
> are the scenarios i am thinking
>
> a) new leader has got a proposal that no other follower has seen, will it
> commit this proposal or let it go
> b) new leader has got a proposal that majority of nodes have it, will it
> commit this or let it go. What happens to other followers, will new leader
> ask followers to commit or rollback based on what it does.
> c)new leader does not have proposal but one of the followers has it, is
> this possible?
>
> From what i understood the new leader simply commits all proposals that he
> has seen irrespective of  what other followers have seen.
>
> Can some one explain what exactly happens, i am more interested in what
> happens in the code
>
> thanks,
> Kishore G

Thanks Alex, that helped a lot.
On Thu, Dec 13, 2012 at 10:56 AM, Alexander Shraer <[EMAIL PROTECTED]>wrote:

> Hi Kishore,
>
> Here's more or less what happens.
>
> Basically, if an operation is uncommitted but reached a quorum, it is
> guaranteed to survive. If it reached less than a quorum it may survive
> but may get lost depending on whether the server that has it
> participates in a timely manner in leader election phases.
>
> First, FastLeaderElection tries to optimistically choose a leader
> candidate that has the most up to date log - a quorum must say that it
> does. This phase is an optimization to avoid having the leader fetch
> the latest state (this way the leader already has it).
>
> Then, servers connect to the candidate leader (LearnerHandler). If
> during this phase the leader discovers someone that is more up-to-date
> than itself, it gives up and we go back to the preliminary phase
> above. Otherwise, once it has a quorum of followers connected to it,
> it basically tries to commit its own log (NEWLEADER is like a propose
> and UPTODATE is like a commit of the log).
> The NEWLEADER message has a zxid which is higher than any zxid existed
> before.
>
> It is possible that a minority of the servers didn't participate in
> this process, but once NEWLEADER reaches a quorum, servers in this
> quorum will be "more up-to-date" than anyone that didn't participate.
> So any servers that were lagging behind do not stand a chance becoming
> leader unless they get updated from a server that did participate. Any
> uncommitted op known to these slow servers will be lost forever.
> Later, if such a server connects to the new leader, the uncommitted
> ops will be truncated from the log.
>
> Hope this helps.
>
> Alex
>
>
>
>
> On Wed, Dec 12, 2012 at 10:31 PM, kishore g <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > I read the docs and presentations online
> > https://cwiki.apache.org/ZOOKEEPER/zookeeperpresentations.html about the
> > internals of zab. I am confused about what happens to uncommitted
> proposals
> > when new leader is elected after current leader fails. To be precise,
> these
> > are the scenarios i am thinking
> >
> > a) new leader has got a proposal that no other follower has seen, will it
> > commit this proposal or let it go
> > b) new leader has got a proposal that majority of nodes have it, will it
> > commit this or let it go. What happens to other followers, will new
> leader
> > ask followers to commit or rollback based on what it does.
> > c)new leader does not have proposal but one of the followers has it, is
> > this possible?
> >
> > From what i understood the new leader simply commits all proposals that
> he
> > has seen irrespective of  what other followers have seen.
> >
> > Can some one explain what exactly happens, i am more interested in what
> > happens in the code
> >
> > thanks,
> > Kishore G
>

Here is a sketch that used as a reference for the code:

https://cwiki.apache.org/confluence/display/ZOOKEEPER/Zab1.0

-Flavio

On Dec 13, 2012, at 9:31 PM, kishore g wrote:

> Thanks Alex, that helped a lot.
>
>
> On Thu, Dec 13, 2012 at 10:56 AM, Alexander Shraer <[EMAIL PROTECTED]>wrote:
>
>> Hi Kishore,
>>
>> Here's more or less what happens.
>>
>> Basically, if an operation is uncommitted but reached a quorum, it is
>> guaranteed to survive. If it reached less than a quorum it may survive
>> but may get lost depending on whether the server that has it
>> participates in a timely manner in leader election phases.
>>
>> First, FastLeaderElection tries to optimistically choose a leader
>> candidate that has the most up to date log - a quorum must say that it
>> does. This phase is an optimization to avoid having the leader fetch
>> the latest state (this way the leader already has it).
>>
>> Then, servers connect to the candidate leader (LearnerHandler). If
>> during this phase the leader discovers someone that is more up-to-date
>> than itself, it gives up and we go back to the preliminary phase
>> above. Otherwise, once it has a quorum of followers connected to it,
>> it basically tries to commit its own log (NEWLEADER is like a propose
>> and UPTODATE is like a commit of the log).
>> The NEWLEADER message has a zxid which is higher than any zxid existed
>> before.
>>
>> It is possible that a minority of the servers didn't participate in
>> this process, but once NEWLEADER reaches a quorum, servers in this
>> quorum will be "more up-to-date" than anyone that didn't participate.
>> So any servers that were lagging behind do not stand a chance becoming
>> leader unless they get updated from a server that did participate. Any
>> uncommitted op known to these slow servers will be lost forever.
>> Later, if such a server connects to the new leader, the uncommitted
>> ops will be truncated from the log.
>>
>> Hope this helps.
>>
>> Alex
>>
>>
>>
>>
>> On Wed, Dec 12, 2012 at 10:31 PM, kishore g <[EMAIL PROTECTED]> wrote:
>>> Hi,
>>>
>>> I read the docs and presentations online
>>> https://cwiki.apache.org/ZOOKEEPER/zookeeperpresentations.html about the
>>> internals of zab. I am confused about what happens to uncommitted
>> proposals
>>> when new leader is elected after current leader fails. To be precise,
>> these
>>> are the scenarios i am thinking
>>>
>>> a) new leader has got a proposal that no other follower has seen, will it
>>> commit this proposal or let it go
>>> b) new leader has got a proposal that majority of nodes have it, will it
>>> commit this or let it go. What happens to other followers, will new
>> leader
>>> ask followers to commit or rollback based on what it does.
>>> c)new leader does not have proposal but one of the followers has it, is
>>> this possible?
>>>
>>> From what i understood the new leader simply commits all proposals that
>> he
>>> has seen irrespective of  what other followers have seen.
>>>
>>> Can some one explain what exactly happens, i am more interested in what
>>> happens in the code
>>>
>>> thanks,
>>> Kishore G
>>