Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper, mail # user - How client can do if watch notify missing?

Copy link to this message
How client can do if watch notify missing?
nileader 2011-12-19, 09:09
 Hi, Ted and Chang.

Based on this situation, you know, zookeeper does not support the
persistent watcher and once watcher triggered, ZK server will never
guarantee if client receive the notify.
So, what i am puzzled that if one notify missing, client will never receive
another notify?
Thank you, Ted.

As far as getChildren goes, you are right.
Since you will get the latest children change and at the same time set a
you will definitely not lose any children node change.

Thank you.

Nov 13, 2010, 12:51 PM, Ted Dunning 작성:

 Persistent watches were omitted from ZK on purpose because of the perceived

 danger of not have a load shedding mechanism.

Note that when you get a notification, the query you do to get the next
 state typically sets the next watch.  This guarantees that you don't lose
 anything, but it may mean that you effectively get notified of multiple
 changes in one notification.

If you really, really want all notifications in order, then what you are
 really looking for is a kind of a distributed transaction log.  For small
 applications, you can implement this by writing logs into ZK znodes.  Your
 client should remember where they were and as they are notified, they read
 the current file to catch up.  This has the downside that to update a file
 you have to completely rewrite it which makes it inconvenient to put a
 of stuff into a single chunk of the log.  You would also need a watcher on
 the directory to notify you when new log files are created.  Aside from the

 slightly quadratic update problem, this does what you need.

You can also check out Bookkeeper which is a more scalable distributed
 transaction logging system.  It addresses the complications of (mis) using
 ZK as a log system and attempts to give you a reliable and robust
 transaction log.

On Fri, Nov 12, 2010 at 7:16 PM, Chang Song <[hidden

 It's been a couple of weeks since we started integrating zookeeper into our

Great piece of software. Cut down LOC by more than half.
Thank you for open sourcing Zookeeper.

I have a couple of questions, though.
 I was wondering you have any plan to support persistent watch stream

One-time watch is very inconvenient for us.
 We need to get a stream of notifications in order. We are talking thousands

 of clients.
Since notification can happen in bulk, we need to set a watch first, and
 once we get a callback,
 we need to check periodically what happens to the children of the watch

You can consider this as a automatic server-side watch registration
 feature: I'd call this sticky watch.
I think it is easier to implement this as a sticky watch.

 Another question is if it is possible to coalesce predefined number of
 changes for watch callbacks.
 We have observed that if there are many changes in children node, clients
 gets different number of
messages in bulks. This makes the architecture of our application from
 event-driven to polling for a while.
 Thus pretty much the same reasoning behind the first question.

 Any comments welcome.

Thank you.