Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # dev >> Where are we in ZOOKEEPER-1416


Copy link to this message
-
Re: Where are we in ZOOKEEPER-1416
Sorry for responding very late to this thread. Seem like it is already
settle in some way, so feel free to ignore. I just need add some more
detail about this JIRA given that I am its owner now.

The main motivation behind ZK-1416 is to reduce memory footprint required
to maintain large number of watches on both the client and the server-side
for our internal service discovery. The secondary goal is to simplify the
user cases where client have to subscribe too all the data under a given
subtree.  We have a working implementation in our branch but we haven't
used in production yet. The memory issue was partially solved by
ZOOKEEPER-1177. And for the second part, existing watch API can be used
but it take us a while to get rid of all corner cases.
Our service discovery system use deltas to make it efficient to publish
information to clients. So we store delta as a new znode and periodically
merging into a snapshot. If our client fall to far behind, it can always
read snapshot. If you are required to see all the deltas you may
purge/merge them only when they are consumed.