Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # dev >> discarded messages seems will exists in some follower, when the leader failed.


Copy link to this message
-
Re: discarded messages seems will exists in some follower, when the leader failed.
Looking at the code, I don't think this can happen.

> > the latest committed proposals in the New Leader. the following is the
> > latest zxid list with  format: *<epoch>,<lower bits>*
> >  1,7
> >  1,8
> >  2,0
> >  2,1
> >
> > the last zxid in a follower 'Z'
> > 1,7
> > 1,8
> > 1,9

when can follower Z have 1,9 in its log?

case 1:  Z was the leader and proposed 1,9. But died before sending it to
other peers.
In this case a new leader (say Y) will have to be elected. Prior to
election,Y will call follower.shutdown() which cleanup everything and  reset
minCommittedLog to 0 (ZKDatabase.clear()).
Later, when Z attempts to join Y, Y will send TRUNC to Z, because
minCommitedLog of Y > 1,9.

case 2: Z was follower and died after logging 1,9.
In this case the leader will have commited 1,9 to the quorum.
On Thu, Apr 7, 2011 at 4:46 AM, jiangwen w <[EMAIL PROTECTED]> wrote:

> Hi Mahadev,
>
> thanks for your answer!
> the leader send TRUNC when the last ZXID in the follower is larger than the
> max committed ZXID in the leader.
> but in this case the last ZXID in the follower is 1,9 the max committed
> ZXID
> in the leader is 2,1. So the leader will not send the TRUNC.
> the following is the code in LeanerHandler:
>
>  if (logTxns && (peerLastZxid >
> leader.zk.getZKDatabase().getmaxCommittedLog())) {
>                // this is the only case that we are sure that
>                // we can ask the peer to truncate the log
>                packetToSend = Leader.TRUNC;
>                zxidToSend = leader.zk.getZKDatabase().getmaxCommittedLog();
>                updates = zxidToSend;
>            }
>
> Sincerely
>
>
>
> On Thu, Apr 7, 2011 at 11:21 AM, Mahadev Konar <[EMAIL PROTECTED]> wrote:
>
> > Hi Jiangwen,
> >  Take a look at :
> >
> >  syncWithLeader(newLeaderZxid);
> >
> > This method calls truncate to take care of the issue you mention.
> >
> > thanks
> > mahadev
> >
> > On Tue, Apr 5, 2011 at 9:31 PM, jiangwen w <[EMAIL PROTECTED]> wrote:
> > > discarded messages seems will exists in some follower, when the leader
> > > failed. considering the following corner case:
> > >
> > > the latest committed proposals in the New Leader. the following is the
> > > latest zxid list with  format: *<epoch>,<lower bits>*
> > >  1,7
> > >  1,8
> > >  2,0
> > >  2,1
> > >
> > > the last zxid in a follower
> > > 1,7
> > > 1,8
> > > 1,9
> > >
> > > when the follower sync with the new leader, the transaction *1,9
> *should
> > be
> > > truncated, but I couldn't found the code that can handle this case.
> > > In this case the following code in LeanerHandler seems will be
> executed.
> > >
> > >   if (proposals.size() != 0) {
> > >                    if ((leader.zk.getZKDatabase().getmaxCommittedLog()
> >> > > peerLastZxid)
> > >                            &&
> > > (leader.zk.getZKDatabase().getminCommittedLog() <= peerLastZxid)) {
> > >                        packetToSend = Leader.DIFF;
> > >
> > > Sincerely
> > >
> >
> >
> >
> > --
> > thanks
> > mahadev
> > @mahadevkonar
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB