Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> High availability backend services via zookeeper or TCP load balancer


Copy link to this message
-
Re: High availability backend services via zookeeper or TCP load balancer
Service Discovery is a good use-case for ZooKeeper. FYI - Curator has an implementation of this already:

https://github.com/Netflix/curator/wiki/Service-Discovery

-Jordan

On Feb 26, 2013, at 9:36 AM, howard chen <[EMAIL PROTECTED]> wrote:

> Hi, I am new to ZK and pls forgive me my question below is stupid :)
>
> We have custom written servers (not public facing, only called by our
> internal system) which is distributed (TCP based, share nothing) that is
> currently in AWS and with the help of ELB TCP based load balancing, it is
> somehow fault-tolerant and we are happy with that.
>
> Now, we need to move off from AWS to save cost as our traffic grow.
>
> The problem is, now we need to maintain our own load balancers and we need
> to make it fault-tolerant (unlike ELB is built-in), the
> expected technologies would be haproxy, keepalived.
>
> While I am thinking this setup, I am thinking why not use ZK instead? Why
> not maintain the currently available servers list in ZK, my initial
> algorithms for the internal clients would be:
>
> 1. Get the latest server list from ZK
> 2. Hash the server list and pick one of the backend (load balancing part)
> 3. Call it
> 4. If it fail, update the ZK and increment the error count
> 5. If the error count reached a threshold and remove the backend from the
> server list
> 6. So the other clients would not see the backend with error
> 7. Flush the error count so the backend would have a chance to active again
>
> Is my algorithm above valid? Any caveat when using with ZK?
>
> Looking for your comment, thanks.