Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> ZooKeeper Scale


+
Rohan Desai 2012-10-16, 01:23
+
Camille Fournier 2012-10-16, 01:31
Copy link to this message
-
Re: ZooKeeper Scale
It's been a while, and it's probably highly dependent on the exact use
cases, but I have tested successfully with 5m znodes and 25m watches.
This was a three server ensemble. In this case 500 clients each
created 10k znodes, then set 5 watches on each of the znodes they
created, then deleted the znodes. The watches were notified on order
of seconds. I did have to tune the amount of memory and GC but otw it
worked fine for me.

Patrick

On Mon, Oct 15, 2012 at 6:31 PM, Camille Fournier <[EMAIL PROTECTED]> wrote:
> I believe that folks at FB have done some very very heavy-load ZK usage and
> could talk more about scale, but I suspect that the current releases of ZK
> would not scale to what you are looking for. Can you tell us more details
> about what you're trying to do?
>
> C
>
> On Mon, Oct 15, 2012 at 9:23 PM, Rohan Desai <[EMAIL PROTECTED]>wrote:
>
>> Has any performance testing been done on !ZooKeeper to determine how many
>> concurrent watches are supported? The application I am considering using ZK
>> for would use ~4000000 znodes and set ~4000000 concurrent watches.  All the
>> performance numbers I found on the ZK docs were reads/write per second. Are
>> there any numbers for concurrent watches?
>>
+
Thawan Kooburat 2012-10-16, 22:55
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB