Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Using ZK for real-time group membership notification

Copy link to this message
Using ZK for real-time group membership notification

Short version:
How can ZK be used to make sure that all group members know *the exact number of
members at all times*?

I have an app that can be run on 1 or more servers.  New instances of the app
come and go, may die, etc. -- the number of the app instances is completely
dynamic.  At any one time, as these apps come and go, each live instance of the
app needs to know how many instances are there total.  If a new instance of the
app is started, all instances need to know the new total number of instances.  
If an app is stopped or if it dies, the remaining apps need to know the new
number of app instances.

Also, and this is critical, they need to know about these additions/removals of
apps right away and they all need to find out them at the same time. Basically,
all members of some group need to know *the exact number of members at all

This sounds almost like we need to watch a "parent group znode" and monitor the
number of its ephemeral children, which represent each app instance that is
watching the "parent groups znode".  Is that right?  If so, then all I'd need to
know is the answer to "How many watchers are watching this znode?" of "How many
kids does this znode have?". And I'd need ZK to notify all watchers whenever the
answer to this question changes.  Ideally it would send/push the answer (the
number of watchers) to all watchers, but if not, I assume any watcher that is
notified about the change would go poll ZK to get the number of ephemeral kids.

I think the above is essentially what's described on
http://eng.wealthfront.com/2010/01/actually-implementing-group-management.html ,
but doesn't answer the part that's critical for me (the very first Q up above).

Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/