Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Zookeeper Load Balancing


Copy link to this message
-
Re: Zookeeper Load Balancing
Is this an internally or externally exposed service? If its an external service (ie, your website), I think you need to look into redundant load balancing services (haproxy, nginx, etc, etc). For an internal service though ...

We use our nd_service_registry module in our app to look for lists of servers that provide specific backend services (search, databases, caches, etc) and then do application level logic for load balancing when its appropriate. For example, DBs we probably don't want to do that with (we may have different DBs for different databases). For our memcache servers, we let the pylibmc-memcached module handle hashing across the servers  but we get the list with our module. For ElasticSearch though, we dynamically get our list of servers, and then use a random server for each query.

https://github.com/Nextdoor/ndserviceregistry

Since load balancing backend API services can be handled in so many different ways, depending on your needs, we leave it up to the individual service client to make the decisions.

--Matt

On Jan 29, 2013, at 10:42 PM, Saurabh Dutta <[EMAIL PROTECTED]> wrote:

> I'm worried about a single point of failure of these products themselves.
>
> On Wed, Jan 30, 2013 at 8:02 AM, kyletu <[EMAIL PROTECTED]> wrote:
>
>> Why not use nginx,or other load balance product?
>>
>>
>>
>>
>> kyletu
>>
>> From: Saurabh Dutta
>> Date: 2013-01-30 06:02
>> To: user
>> Subject: Zookeeper Load Balancing
>> I've a tcp based service which i want to run on three nodes.
>>
>> I want incoming requests to be distributed among these three nodes
>> uniformly. Can zookeeper help me achieve this?
>>
>> I've gone through leader election and believe it is a very good recipe to
>> make the services fail safe but at this point i want to solve the load
>> balancing problem using zookeeper.
>>
>> Any help will be greatly appreciated.
>>
>> thanks
>>