Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> ZooKeeper performance


Copy link to this message
-
Re: ZooKeeper performance
It definitely can be done, but what if your application grows by 10x?

On Mon, Oct 3, 2011 at 10:56 AM, Dima Gutzeit
<[EMAIL PROTECTED]>wrote:

> Very nice analysis at the link, thanks.
>
> Assuming my nodes are around the same size ~100 bytes, ensemble of 5
> servers and 10 clients can easily support the required ~20,000
> operations per second, no ?
>
> I will run a smoke-test to prove it ofcourse ...
>
> Regards,
> Dima Gutzeit.
>
> Sent from my iPhone
>
> On 3 באוק 2011, at 19:41, "Fournier, Camille F."
> <[EMAIL PROTECTED]> wrote:
>
> > It's pretty easy to set up a zk-smoketest to simulate what you are doing.
> We can't answer this question without knowing how big the data you're
> writing, etc etc. I would recommend testing it out yourself on realistic
> data sizes.
> >
> >
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOverview
> >
> > -----Original Message-----
> > From: Dima Gutzeit [mailto:[EMAIL PROTECTED]]
> > Sent: Monday, October 03, 2011 1:36 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: ZooKeeper performance
> >
> > Inline.
> >
> > Regards,
> > Dima Gutzeit.
> >
> > Sent from my iPhone
> >
> > On 3 באוק 2011, at 19:31, Ted Dunning <[EMAIL PROTECTED]> wrote:
> >
> >> Questions in-line
> >>
> >> On Mon, Oct 3, 2011 at 10:19 AM, Dima Gutzeit
> >> <[EMAIL PROTECTED]>wrote:
> >>
> >>> Hi,
> >>>
> >>> I have some performance related question.
> >>>
> >>> I am running a cluster of 5 zookeeper machines, each one has dual quad
> >>> core Xeon 2.5 Ghz, 8 gb RAM.
> >>>
> >>> I want achieve the following numbers:
> >>>
> >>> 10 clients producing in total (at peaks):
> >>>
> >>
> >> Each producing this?
> >
> > Total.
> >>
> >>
> >>> 3K add nodes, 3K delete nodes and 10K watches. Per second.
> >>>
> >>
> >> What do you mean by 10K watches?  10,000 watch notifications?
> >
> > Yes.
> >>
> >> Does those numbers make any sense ?
> >>>
> >>
> >> This is a bit high.  It sounds like you are trying to use ZK as a
> message
> >> bus rather than a coordination service.  You can do this, but the
> throughput
> >> you can achieve is limited.  If you want higher throughput, it is better
> to
> >> have ZK coordinate a higher performance messaging system such as Kafka.
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB