Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Fwd: Where are we in ZOOKEEPER-1416


Copy link to this message
-
Fwd: Where are we in ZOOKEEPER-1416
What is going on with this thread over on dev@zookeeper? Bringing it to the
attention of people over here.
---------- Forwarded message ----------
From: Ted Dunning <[EMAIL PROTECTED]>
Date: Fri, Jan 17, 2014 at 2:41 PM
Subject: Re: Where are we in ZOOKEEPER-1416
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
My reference here is to the comments a ways up thread.  Kishore and I
clearly agree completely that idempotency and dealing with the state as it
is right now are the keys to correct design.
On Fri, Jan 17, 2014 at 2:14 PM, Ted Dunning <[EMAIL PROTECTED]> wrote:

>
> That comment indicates a lack of understanding of ZK, not a bug in ZK.
>
> You don't lose state transitions if you read new state at the same time
> you set the new watch.
>
> Likewise, it is simply a product of bad design to have a problem with
> asynchronous notification.  Changes on other machines *are* asynchronous
so
> anybody who can't handle that is inherently denying reality.  If you want
> to inject the notifications into a sequential view of an event stream,
that
> is trivial to do.
>
> Systems that depend on transition notification are generally not as robust
> as systems that depend on current state.  Building a cluster manager works
> better if the master is notified that a change has happened, but then
> simply deals with the situation as it stands.
>
> As an analog, imagine that you have a system that shows a number x and a
> second system that is supposed to show an echo of that number.
>
> Design A is notified of changes to x in the form of deltas.  If there is
> ever an error in handling events, the echo will be off forever.  The error
> that causes the delta to be dropped could be notification or a coding
error
> or a misunderstanding of how parallel systems work.  For instance, the
> InterruptedException might not be handled right.
>
> Design B is notified of changes to x and whenever a change happens, the
> second system simply goes and reads the new state.  Errors will be quickly
> corrected.
>
> It sounds like the original poster is trying to build something like
> Design A when they should be building Design B.
>
>
>
>
>
> On Fri, Jan 17, 2014 at 12:34 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> HBASE-5487 is also related.
>>
>> The discussion there is very long. Below is an excerpt from Honghua:
>>
>> too many tricky scenarios/bugs due to ZK watch is one-time(which can
>> result
>> in missed state transition) and the notification/process is
>> asyncronous(which can lead to delayed/non-update-to-date state in master
>> memory).
>>
>> Cheers
>>
>>
>> On Fri, Jan 17, 2014 at 11:25 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> > Hi, Flavio:
>> > HBASE-8365 is one such case.
>> >
>> > Let me search around for other related discussion.
>> >
>> >
>> > On Fri, Jan 17, 2014 at 11:17 AM, Flavio Junqueira <
>> [EMAIL PROTECTED]>wrote:
>> >
>> >> Hi Ted,
>> >>
>> >> Can you provide more detail on how the precise deltas could make it
>> more
>> >> robust?
>> >>
>> >> -Flavio
>> >>
>> >> -----Original Message-----
>> >> From: "Ted Yu" <[EMAIL PROTECTED]>
>> >> Sent: 17/01/2014 17:25
>> >> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> >> Subject: Re: Where are we in ZOOKEEPER-1416
>> >>
>> >> Having the ability to know exact deltas would help make HBase region
>> >> assignment more robust.
>> >>
>> >> Cheers
>> >>
>> >>
>> >>
>> >> On Fri, Jan 17, 2014 at 9:13 AM, kishore g <[EMAIL PROTECTED]>
>> wrote:
>> >>
>> >> > I agree with you, I like the side effect and in fact I would prefer
>> to
>> >> have
>> >> > one notification for all changes under a parent node.
>> >> >
>> >> > However, Hao is probably asking for ability to know exact deltas.
>> >> >
>> >> >
>> >> > On Fri, Jan 17, 2014 at 8:15 AM, FPJ <[EMAIL PROTECTED]> wrote:
>> >> >
>> >> > > We don't need to have a mapping between every change and a
>> >> notification.
>> >> > If
>> >> > > there are 2+ changes between notifications, you'll be able to
quick
the
read
case

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB