Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Re: Where are we in ZOOKEEPER-1416


Copy link to this message
-
Re: Where are we in ZOOKEEPER-1416
Stack 2014-01-18, 04:06
On Fri, Jan 17, 2014 at 2:53 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> What is going on with this thread over on dev@zookeeper? Bringing it to
> the
> attention of people over here.
>
>
>

Ted?
St.Ack
> ---------- Forwarded message ----------
> From: Ted Dunning <[EMAIL PROTECTED]>
> Date: Fri, Jan 17, 2014 at 2:41 PM
> Subject: Re: Where are we in ZOOKEEPER-1416
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>
>
> My reference here is to the comments a ways up thread.  Kishore and I
> clearly agree completely that idempotency and dealing with the state as it
> is right now are the keys to correct design.
>
>
> On Fri, Jan 17, 2014 at 2:14 PM, Ted Dunning <[EMAIL PROTECTED]>
> wrote:
>
> >
> > That comment indicates a lack of understanding of ZK, not a bug in ZK.
> >
> > You don't lose state transitions if you read new state at the same time
> > you set the new watch.
> >
> > Likewise, it is simply a product of bad design to have a problem with
> > asynchronous notification.  Changes on other machines *are* asynchronous
> so
> > anybody who can't handle that is inherently denying reality.  If you want
> > to inject the notifications into a sequential view of an event stream,
> that
> > is trivial to do.
> >
> > Systems that depend on transition notification are generally not as
> robust
> > as systems that depend on current state.  Building a cluster manager
> works
> > better if the master is notified that a change has happened, but then
> > simply deals with the situation as it stands.
> >
> > As an analog, imagine that you have a system that shows a number x and a
> > second system that is supposed to show an echo of that number.
> >
> > Design A is notified of changes to x in the form of deltas.  If there is
> > ever an error in handling events, the echo will be off forever.  The
> error
> > that causes the delta to be dropped could be notification or a coding
> error
> > or a misunderstanding of how parallel systems work.  For instance, the
> > InterruptedException might not be handled right.
> >
> > Design B is notified of changes to x and whenever a change happens, the
> > second system simply goes and reads the new state.  Errors will be
> quickly
> > corrected.
> >
> > It sounds like the original poster is trying to build something like
> > Design A when they should be building Design B.
> >
> >
> >
> >
> >
> > On Fri, Jan 17, 2014 at 12:34 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> >> HBASE-5487 is also related.
> >>
> >> The discussion there is very long. Below is an excerpt from Honghua:
> >>
> >> too many tricky scenarios/bugs due to ZK watch is one-time(which can
> >> result
> >> in missed state transition) and the notification/process is
> >> asyncronous(which can lead to delayed/non-update-to-date state in master
> >> memory).
> >>
> >> Cheers
> >>
> >>
> >> On Fri, Jan 17, 2014 at 11:25 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >>
> >> > Hi, Flavio:
> >> > HBASE-8365 is one such case.
> >> >
> >> > Let me search around for other related discussion.
> >> >
> >> >
> >> > On Fri, Jan 17, 2014 at 11:17 AM, Flavio Junqueira <
> >> [EMAIL PROTECTED]>wrote:
> >> >
> >> >> Hi Ted,
> >> >>
> >> >> Can you provide more detail on how the precise deltas could make it
> >> more
> >> >> robust?
> >> >>
> >> >> -Flavio
> >> >>
> >> >> -----Original Message-----
> >> >> From: "Ted Yu" <[EMAIL PROTECTED]>
> >> >> Sent: 17/01/2014 17:25
> >> >> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> >> >> Subject: Re: Where are we in ZOOKEEPER-1416
> >> >>
> >> >> Having the ability to know exact deltas would help make HBase region
> >> >> assignment more robust.
> >> >>
> >> >> Cheers
> >> >>
> >> >>
> >> >>
> >> >> On Fri, Jan 17, 2014 at 9:13 AM, kishore g <[EMAIL PROTECTED]>
> >> wrote:
> >> >>
> >> >> > I agree with you, I like the side effect and in fact I would prefer
> >> to
> >> >> have
> >> >> > one notification for all changes under a parent node.
> >> >> >
> >> >> > However, Hao is probably asking for ability to know exact deltas.