Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Distributed ZooKeeper cluster design


Copy link to this message
-
Re: Distributed ZooKeeper cluster design
I ran ZK in prod over the WAN and it was fine. If you have a modest read
and write load, I would bet you'lll have no problem. I don't have numbers
in front of me but we tested clients and got totally acceptable
throughputs. It's really hard to generalize because it depends on the link
between your clusters, etc, so as always I recommend running some basic
load testing against the proposed cluster and seeing how it performs.

C

On Wed, Dec 14, 2011 at 6:04 PM, Yongsheng Wu <[EMAIL PROTECTED]>wrote:

> We have a user case that makes sense to have distributed zookeeper across
> regions. That is to use zookeeper to enforce uniqueness across regions, and
> we want to be able to tolerate any single region failure. In our case, both
> reads and writes are modest.
>
> I wonder if anyone on this list actually have zookeeper cluster set up
> across WAN on production, and I'd be very interested in knowing what kind
> of performance you are getting from the cluster, and what
> reliability/availability are like.
>
> Thanks a lot if someone can share information as such.
>
> Yongsheng
>
> On Tue, Dec 13, 2011 at 6:50 PM, Dima Gutzeit
> <[EMAIL PROTECTED]>wrote:
>
> > To summarize the discussion:
> >
> > As I understand the "preferred" approach will be having a quorum in C and
> > at least two observers in other regions (for HA).
> >
> > Each node's clients must talk to its local ZK servers.
> >
> > This approach will decrease the traffic between the regions and increase
> > the speed of ZK clients accessing local nodes.
> >
> > Thanks so much for the suggestions.
> >
> > Regards,
> > Dima Gutzeit.
> >
> > On Wed, Dec 14, 2011 at 2:36 AM, Ted Dunning <[EMAIL PROTECTED]>
> > wrote:
> >
> > > I am happy to agree with everyone that the mirroring isn't a great idea
> > for
> > > most things even if that makes me look like I disagree with myself.
> > >
> > > I do think that mirroring could be made to happen in a reliable way,
> but
> > it
> > > isn't going to be a viable substitute for direct access to the cluster.
> >  By
> > > reliable, I think that you could get a reliable picture of what was in
> > the
> > > master cluster at some time in the past.  Occasionally the mirror would
> > be
> > > further behind than other times and it might be necessary for the
> mirror
> > to
> > > be updated much faster than real-time.  In my vision, the mirror would
> be
> > > read-only since anything else leads to madness in the strict
> consistency
> > > model that ZK maintains.
> > >
> > > On Tue, Dec 13, 2011 at 2:57 PM, Benjamin Reed <[EMAIL PROTECTED]>
> wrote:
> > >
> > > > i agree with camille that mirror breaks a lot of the basic guarantees
> > > > that you use from zookeeper. with that caveat in mind, there is a
> > > > patch that enables mirroring: ZOOKEEPER-892.
> > > >
> > > > ben
> > > >
> > > > On Tue, Dec 13, 2011 at 8:24 AM, Camille Fournier <
> [EMAIL PROTECTED]>
> > > > wrote:
> > > > > I have to strongly disagree with ted on the mirroring idea... I
> think
> > > it
> > > > is
> > > > > likely to be really error-prone and kind of defeats the purpose of
> ZK
> > > in
> > > > my
> > > > > mind. It depends on what you're mirroring but if you're trying to
> > keep
> > > > all
> > > > > the data coherent you can't sensibly do that in two clusters, so
> > unless
> > > > the
> > > > > mirror is for a really small subset of the data I would stay far
> far
> > > away
> > > > > from that.
> > > > >
> > > > > Observers are available in 3.3.3, yes.
> > > > > Unfortunately, we don't have configurable connection logic in ZK
> > client
> > > > (at
> > > > > least java) right now. We have the ability to add it pretty easily,
> > but
> > > > it
> > > > > hasn't been put in yet.
> > > > >
> > > > > You're seeing slow performance for a setup that has all ZK servers
> in
> > > > > region C for clients only in regions A and B and you can't blame
> the
> > > > > network? That's literally the only thing you could blame, unless
> > > clients
> >
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB