Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> How dose zookeeper handle fault-detect in distributed storage system


Copy link to this message
-
Re: How dose zookeeper handle fault-detect in distributed storage system
I am not sure that I get your point.
You need a ZooKeeper ensemble with several servers (minimun recommended
number is 3). One of them will be selected as a leader of the ensemble when
required, but normally you don't have to worry about that.
Normally you will need to run the servers in the ZooKeeper ensemble as
independent processes, and each of them should run in a different machine
in order to increase redundancy. These processes could be running in any
machine in your network, including I assume, the machines that host the
dispatch nodes, store nodes and middleware. On top of that, ZooKeeper
clients must be somehow linked with your dispatch/store/middleware nodes
and manage the information stored in the ZooKeeper ensemble.

On Wed, Sep 11, 2013 at 10:11 AM, fengguang gong
<[EMAIL PROTECTED]>wrote:

> Thanks very much German,
>         The second possibility of you answer will be great. But i still
> confused about
>  the servers(Leader) and clients.
> Should i distinguish servers(Leader) and clients between dispatch nodes,
> store nodes and middleware?
> Or should i just ignore all this concepts?
> 在 2013-9-11,下午3:12,German Blanco <[EMAIL PROTECTED]> 写道:
>
> > Hello Fengguang Gong,
> > I think there is more than one answer to your question.
> > One possibility would be to have each of your nodes as zookeeper clients
> > that create an ephemeral node in the zookeeper data, and query and most
> > likely subscribe to changes so that they are notified about the status of
> > the ephemeral zookeeper nodes created by the rest of the nodes. If you
> are
> > only interested in dispatch and middleware nodes knowing about the status
> > of store nodes, then you could have ephemeral zookeeper nodes created
> only
> > by the store nodes, and dispatch and middleware nodes querying and
> > subscribing to the resulting status.
> > You will need to make sure that the events of store nodes going up and
> down
> > are reflected correctly in the creation and deletion of the zookeeper
> node.
> > You will also have to tune the heartbeat between zookeeper client and
> > server so that it fits your requirements.
> > Does that suit you?
> > Any other options?
> > Good luck :-)
> >
> >
> > On Wed, Sep 11, 2013 at 8:17 AM, fengguang gong <
> [EMAIL PROTECTED]>wrote:
> >
> >> Hi all:
> >>
> >>        Recently my lab want to use zk to manager our cluster(Fault
> >> detect). Our cluster
> >> includes three kinds of node:
> >> 1. dispatch node : load balance and dispatch data.
> >> 2. store node: receive data from dispatch node and store.
> >> 3. middleware: query data from all the store node.
> >> Here my question is : How dose zk handle fault-detect in this system(how
> >> dose dispatch node
> >> and middleware know that a store node is down).
>
>