Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Fwd: Where are we in ZOOKEEPER-1416


Copy link to this message
-
Fwd: Where are we in ZOOKEEPER-1416
Andrew Purtell 2014-01-17, 22:53
What is going on with this thread over on dev@zookeeper? Bringing it to the
attention of people over here.
---------- Forwarded message ----------
From: Ted Dunning <[EMAIL PROTECTED]>
Date: Fri, Jan 17, 2014 at 2:41 PM
Subject: Re: Where are we in ZOOKEEPER-1416
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
My reference here is to the comments a ways up thread.  Kishore and I
clearly agree completely that idempotency and dealing with the state as it
is right now are the keys to correct design.
On Fri, Jan 17, 2014 at 2:14 PM, Ted Dunning <[EMAIL PROTECTED]> wrote:

>
> That comment indicates a lack of understanding of ZK, not a bug in ZK.
>
> You don't lose state transitions if you read new state at the same time
> you set the new watch.
>
> Likewise, it is simply a product of bad design to have a problem with
> asynchronous notification.  Changes on other machines *are* asynchronous
so
> anybody who can't handle that is inherently denying reality.  If you want
> to inject the notifications into a sequential view of an event stream,
that
> is trivial to do.
>
> Systems that depend on transition notification are generally not as robust
> as systems that depend on current state.  Building a cluster manager works
> better if the master is notified that a change has happened, but then
> simply deals with the situation as it stands.
>
> As an analog, imagine that you have a system that shows a number x and a
> second system that is supposed to show an echo of that number.
>
> Design A is notified of changes to x in the form of deltas.  If there is
> ever an error in handling events, the echo will be off forever.  The error
> that causes the delta to be dropped could be notification or a coding
error
> or a misunderstanding of how parallel systems work.  For instance, the
> InterruptedException might not be handled right.
>
> Design B is notified of changes to x and whenever a change happens, the
> second system simply goes and reads the new state.  Errors will be quickly
> corrected.
>
> It sounds like the original poster is trying to build something like
> Design A when they should be building Design B.
>
>
>
>
>
> On Fri, Jan 17, 2014 at 12:34 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> HBASE-5487 is also related.
>>
>> The discussion there is very long. Below is an excerpt from Honghua:
>>
>> too many tricky scenarios/bugs due to ZK watch is one-time(which can
>> result
>> in missed state transition) and the notification/process is
>> asyncronous(which can lead to delayed/non-update-to-date state in master
>> memory).
>>
>> Cheers
>>
>>
>> On Fri, Jan 17, 2014 at 11:25 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> > Hi, Flavio:
>> > HBASE-8365 is one such case.
>> >
>> > Let me search around for other related discussion.
>> >
>> >
>> > On Fri, Jan 17, 2014 at 11:17 AM, Flavio Junqueira <
>> [EMAIL PROTECTED]>wrote:
>> >
>> >> Hi Ted,
>> >>
>> >> Can you provide more detail on how the precise deltas could make it
>> more
>> >> robust?
>> >>
>> >> -Flavio
>> >>
>> >> -----Original Message-----
>> >> From: "Ted Yu" <[EMAIL PROTECTED]>
>> >> Sent: 17/01/2014 17:25
>> >> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> >> Subject: Re: Where are we in ZOOKEEPER-1416
>> >>
>> >> Having the ability to know exact deltas would help make HBase region
>> >> assignment more robust.
>> >>
>> >> Cheers
>> >>
>> >>
>> >>
>> >> On Fri, Jan 17, 2014 at 9:13 AM, kishore g <[EMAIL PROTECTED]>
>> wrote:
>> >>
>> >> > I agree with you, I like the side effect and in fact I would prefer
>> to
>> >> have
>> >> > one notification for all changes under a parent node.
>> >> >
>> >> > However, Hao is probably asking for ability to know exact deltas.
>> >> >
>> >> >
>> >> > On Fri, Jan 17, 2014 at 8:15 AM, FPJ <[EMAIL PROTECTED]> wrote:
>> >> >
>> >> > > We don't need to have a mapping between every change and a
>> >> notification.
>> >> > If
>> >> > > there are 2+ changes between notifications, you'll be able to
quick
the
read
case

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)