Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> ZooKeeper performance


Copy link to this message
-
Re: ZooKeeper performance
It definitely can be done, but what if your application grows by 10x?

On Mon, Oct 3, 2011 at 10:56 AM, Dima Gutzeit
<[EMAIL PROTECTED]>wrote:

> Very nice analysis at the link, thanks.
>
> Assuming my nodes are around the same size ~100 bytes, ensemble of 5
> servers and 10 clients can easily support the required ~20,000
> operations per second, no ?
>
> I will run a smoke-test to prove it ofcourse ...
>
> Regards,
> Dima Gutzeit.
>
> Sent from my iPhone
>
> On 3 באוק 2011, at 19:41, "Fournier, Camille F."
> <[EMAIL PROTECTED]> wrote:
>
> > It's pretty easy to set up a zk-smoketest to simulate what you are doing.
> We can't answer this question without knowing how big the data you're
> writing, etc etc. I would recommend testing it out yourself on realistic
> data sizes.
> >
> >
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOverview
> >
> > -----Original Message-----
> > From: Dima Gutzeit [mailto:[EMAIL PROTECTED]]
> > Sent: Monday, October 03, 2011 1:36 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: ZooKeeper performance
> >
> > Inline.
> >
> > Regards,
> > Dima Gutzeit.
> >
> > Sent from my iPhone
> >
> > On 3 באוק 2011, at 19:31, Ted Dunning <[EMAIL PROTECTED]> wrote:
> >
> >> Questions in-line
> >>
> >> On Mon, Oct 3, 2011 at 10:19 AM, Dima Gutzeit
> >> <[EMAIL PROTECTED]>wrote:
> >>
> >>> Hi,
> >>>
> >>> I have some performance related question.
> >>>
> >>> I am running a cluster of 5 zookeeper machines, each one has dual quad
> >>> core Xeon 2.5 Ghz, 8 gb RAM.
> >>>
> >>> I want achieve the following numbers:
> >>>
> >>> 10 clients producing in total (at peaks):
> >>>
> >>
> >> Each producing this?
> >
> > Total.
> >>
> >>
> >>> 3K add nodes, 3K delete nodes and 10K watches. Per second.
> >>>
> >>
> >> What do you mean by 10K watches?  10,000 watch notifications?
> >
> > Yes.
> >>
> >> Does those numbers make any sense ?
> >>>
> >>
> >> This is a bit high.  It sounds like you are trying to use ZK as a
> message
> >> bus rather than a coordination service.  You can do this, but the
> throughput
> >> you can achieve is limited.  If you want higher throughput, it is better
> to
> >> have ZK coordinate a higher performance messaging system such as Kafka.
>