Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> ZooKeeper performance


Copy link to this message
-
Re: ZooKeeper performance
Correct.

But if your state is changing that fast, pub-sub might well be a useful way
to direct the firehose in the right direction.

On Mon, Oct 3, 2011 at 10:58 AM, Dima Gutzeit
<[EMAIL PROTECTED]>wrote:

> I read about Kafka as you suggested but it looks to me as a
> memory-less pub-sub while I want to keep a state in addition of
> notifying other watchers ...
>
> Regards,
> Dima Gutzeit.
>
> Sent from my iPhone
>
> On 3 באוק 2011, at 19:40, Ted Dunning <[EMAIL PROTECTED]> wrote:
>
> > I think that these levels are achievable with a correctly tuned ZK
> cluster,
> > but you won't have as much head-room as I would like.  I really think
> that
> > using other systems built on top of ZK might be more useful.
> >
> > On Mon, Oct 3, 2011 at 10:35 AM, Dima Gutzeit
> > <[EMAIL PROTECTED]>wrote:
> >
> >> Inline.
> >>
> >> Regards,
> >> Dima Gutzeit.
> >>
> >> Sent from my iPhone
> >>
> >> On 3 באוק 2011, at 19:31, Ted Dunning <[EMAIL PROTECTED]> wrote:
> >>
> >>> Questions in-line
> >>>
> >>> On Mon, Oct 3, 2011 at 10:19 AM, Dima Gutzeit
> >>> <[EMAIL PROTECTED]>wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I have some performance related question.
> >>>>
> >>>> I am running a cluster of 5 zookeeper machines, each one has dual quad
> >>>> core Xeon 2.5 Ghz, 8 gb RAM.
> >>>>
> >>>> I want achieve the following numbers:
> >>>>
> >>>> 10 clients producing in total (at peaks):
> >>>>
> >>>
> >>> Each producing this?
> >>
> >> Total.
> >>>
> >>>
> >>>> 3K add nodes, 3K delete nodes and 10K watches. Per second.
> >>>>
> >>>
> >>> What do you mean by 10K watches?  10,000 watch notifications?
> >>
> >> Yes.
> >>>
> >>> Does those numbers make any sense ?
> >>>>
> >>>
> >>> This is a bit high.  It sounds like you are trying to use ZK as a
> message
> >>> bus rather than a coordination service.  You can do this, but the
> >> throughput
> >>> you can achieve is limited.  If you want higher throughput, it is
> better
> >> to
> >>> have ZK coordinate a higher performance messaging system such as Kafka.
> >>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB