Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Curator and queue recipe warning


Copy link to this message
-
Re: Curator and queue recipe warning
I took a look last night and it sounds like the big problem is the
serialization mechanism. The transportation layer has a limit of 1M in
size, the getChildren method pass all the children name in that chunk, so
if you have a lot of children then you are in trouble.
On Wed, Dec 5, 2012 at 7:09 AM, Bryan Thompson <[EMAIL PROTECTED]> wrote:

> InterestingÅ . I had long ago requested clarification of the zookeeper
> limits.  This is in keeping with my expectations.  Good for relatively
> small numbers of children.
>
> For example, you would be safe trying to coordinate 1000 machines but not
> safe trying to coordinate the locations of 100,000+ shards over 1000
> machines.
>
> Bryan
>
> On 12/4/12 9:42 PM, "Diego Oliveira" <[EMAIL PROTECTED]> wrote:
>
> >Hello folks,
> >
> >    I got to the Curator wiki page to see something about the distributed
> >queue recipe and got surprised with the big  warning telling to not use
> >zookeeper as a distributed queue controller, the reason page described
> >some
> >cases that I passed here... is this *TRUE*, if so, what option do we have.
> >
> >The message:
> >*IMPORTANT* - We recommend that you do NOT use ZooKeeper for Queues.
> >Please
> >see Tech Note 4 <https://github.com/Netflix/curator/wiki/Tech-Note-4> for
> >details. Links
> >
> >https://github.com/Netflix/curator/wiki/Distributed-Queue
> >https://github.com/Netflix/curator/wiki/Tech-Note-4
> >
> >--
> >Att.
> >Diego de Oliveira
> >System Architect
> >[EMAIL PROTECTED]
> >www.diegooliveira.com
> >Never argue with a fool -- people might not be able to tell the difference
>
>
--
Att.
Diego de Oliveira
System Architect
[EMAIL PROTECTED]
www.diegooliveira.com
Never argue with a fool -- people might not be able to tell the difference
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB