Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Configuration management based on load


Copy link to this message
-
Re: Configuration management based on load
No. ZooKeeper merely reports on state of things, it does not execute
custom logic. You need to have your own servers that translate the
information provided by ZooKeeper and cause system state to change.

You could do what you want via a set of two types of processes and client logic.
Process type 1 is the servers you're talking about, that report via
writing to zk nodes their current busy level, and watch a set of data
nodes that contain configuration information directed at them.

Process type 2 watches the nodes written by type 1, and will update
configuration nodes based on that information and its belief about the
state of the whole system.

Note a few things about this:
Everything here is a few beats behind. If type 1 is totally
overloaded, it could possibly lack the ability to update zk to
indicate this overload. Type 2 may see temporary overloads that
resolve themselves quickly but still reconfigure systems based on
temporary information. So to do this you'll have to be careful to
identify intelligently long-term bottlenecks, and ensure that you can
always update ZK even when you are very busy.

C

On Tue, Jan 31, 2012 at 1:59 AM, Mohan <[EMAIL PROTECTED]> wrote:
> It is like a load balancer for socket listeners using Zookeeper.  I want to
> send some information to Zookeeper from each of the socket listeners. This
> could be the number of bytes waiting to be processed. Zookeeper could then
> decide based on some configuration data that one or more of the listeners is
> getting overloaded.
>
> It could send back an event to increase the threads in one or more of the
> listeners.
>
> Can I post and receive custom events from zookeeper ?
>
> Thanks.
>
> --
> View this message in context: http://zookeeper-user.578899.n2.nabble.com/Configuration-management-based-on-load-tp7238974p7239127.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.