Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper, mail # user - ZooKeeper Scale


Copy link to this message
-
Re: ZooKeeper Scale
Thawan Kooburat 2012-10-16, 22:55
You can add more observers to scale the number of watches that the
ensemble can handle.

Znode count will depend on how much RAM you have and how much data are in
each znode. Our last benchmark shows that empty znode consume 600 bytes of
memory.  

--
Thawan Kooburat

On 10/15/12 11:43 PM, "Patrick Hunt" <[EMAIL PROTECTED]> wrote:

>It's been a while, and it's probably highly dependent on the exact use
>cases, but I have tested successfully with 5m znodes and 25m watches.
>This was a three server ensemble. In this case 500 clients each
>created 10k znodes, then set 5 watches on each of the znodes they
>created, then deleted the znodes. The watches were notified on order
>of seconds. I did have to tune the amount of memory and GC but otw it
>worked fine for me.
>
>Patrick
>
>On Mon, Oct 15, 2012 at 6:31 PM, Camille Fournier <[EMAIL PROTECTED]>
>wrote:
>> I believe that folks at FB have done some very very heavy-load ZK usage
>>and
>> could talk more about scale, but I suspect that the current releases of
>>ZK
>> would not scale to what you are looking for. Can you tell us more
>>details
>> about what you're trying to do?
>>
>> C
>>
>> On Mon, Oct 15, 2012 at 9:23 PM, Rohan Desai
>><[EMAIL PROTECTED]>wrote:
>>
>>> Has any performance testing been done on !ZooKeeper to determine how
>>>many
>>> concurrent watches are supported? The application I am considering
>>>using ZK
>>> for would use ~4000000 znodes and set ~4000000 concurrent watches.
>>>All the
>>> performance numbers I found on the ZK docs were reads/write per
>>>second. Are
>>> there any numbers for concurrent watches?
>>>