Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Linking two sites via two Zookeeper instances


+
Christian Schuhegger 2013-01-27, 13:56
+
Jordan Zimmerman 2013-01-28, 20:09
Copy link to this message
-
Re: Linking two sites via two Zookeeper instances
Hi Christian,

I don't think this is currently possible. I believe there has been some
work on building a hierarchy of ZooKeeper clusters @ Facebook, but I don't
know the details. I don't believe that this would mean less management
overhead though, since you'd still need several voting servers in each
datacenter.

But I actually wanted to ask you about your usecase. Do you have
consistency requirements among data items mastered in different datacenters
? For example - do you require that all clients (no matter where they are)
see changes to /A/* and /B/* in the same order ? could you share some more
details ? or, lets say you have 3 datacenters, one mastering /A/* another
/B/* and the third /C/*. Suppose that the first datacenter sees a change to
/C/x and afterwards /A/y is updated. Is it possible that someone in
datacenter B sees the new /A/y  before the new /C/x  ?

The reason I'm asking is that some time in the past me and others made this
initial proposal:
http://wiki.apache.org/hadoop/ZooKeeper/MountRemoteZookeeper
which didn't get enough support for lack of a compelling use-case (among
other things).

Thanks,
Alex

On Sun, Jan 27, 2013 at 5:56 AM, Christian Schuhegger <
[EMAIL PROTECTED]> wrote:

> Hello,
>
> up to now I did not work with Zookeeper itself and am only reading
> documentation. From the document "ZooKeeper: Wait-free coordination for
> Internet-scale systems" I understand that ZooKeeper uses a single writer
> (the leader) approach for a ZooKeeper cluster, e.g. all writes go through
> the leader.
>
> From the documentation about Observers:
> http://zookeeper.apache.org/**doc/trunk/zookeeperObservers.**html<http://zookeeper.apache.org/doc/trunk/zookeeperObservers.html>
> I understand that "Observers have other advantages. Because they do not
> vote, they are not a critical part of the ZooKeeper ensemble. Therefore
> they can fail, or be disconnected from the cluster, without harming the
> availability of the ZooKeeper service. The benefit to the user is that
> Observers may connect over less reliable network links than Followers. In
> fact, Observers may be used to talk to a ZooKeeper server from another data
> center."
>
> The use-case I have in mind is to use ZooKeeper within one data-center and
> synchronize data via the Observer mechanism to another data center half way
> around the world. I would need to do the same symmetrically the other way
> round.
>
> I could imagine to do this by setting up two independent ZooKeeper
> clusters, one which has the leader and the other voters in data center A
> and its Observers in data center B and another ZooKeeper cluster which has
> the leader and the other voters in data center B and its Observers in data
> center A.
>
> This would mean an increased maintenance overhead, I believe.
>
> My question is now if it is somehow possible to do this with one ZooKeeper
> cluster only by configuration, e.g. defining that all writes that go to
> znode /A and its children (e.g. /A/a, /A/b, ...) are handled by a group of
> voters in data center A and all writes that go to znode /B and its children
> are handled by a group of voters in data center B. All ZooKeeper servers
> would be at least observers of all znodes in the cluster, e.g. the group of
> ZooKeeper servers that is not voter for a given top-level node would at
> least be observer of that top-level node.
>
> I would be interested to extend this concept to more than two data centers
> with ping times between the data centers in the order of 300ms.
>
> Many thanks and best regards,
> --
> Christian Schuhegger
>
>
>
+
Christian Schuhegger 2013-02-02, 03:57
+
Christian Schuhegger 2013-02-02, 03:39
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB