Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Curator and queue recipe warning

Copy link to this message
Re: Curator and queue recipe warning
I took a look last night and it sounds like the big problem is the
serialization mechanism. The transportation layer has a limit of 1M in
size, the getChildren method pass all the children name in that chunk, so
if you have a lot of children then you are in trouble.
On Wed, Dec 5, 2012 at 7:09 AM, Bryan Thompson <[EMAIL PROTECTED]> wrote:

> InterestingÅ . I had long ago requested clarification of the zookeeper
> limits.  This is in keeping with my expectations.  Good for relatively
> small numbers of children.
> For example, you would be safe trying to coordinate 1000 machines but not
> safe trying to coordinate the locations of 100,000+ shards over 1000
> machines.
> Bryan
> On 12/4/12 9:42 PM, "Diego Oliveira" <[EMAIL PROTECTED]> wrote:
> >Hello folks,
> >
> >    I got to the Curator wiki page to see something about the distributed
> >queue recipe and got surprised with the big  warning telling to not use
> >zookeeper as a distributed queue controller, the reason page described
> >some
> >cases that I passed here... is this *TRUE*, if so, what option do we have.
> >
> >The message:
> >*IMPORTANT* - We recommend that you do NOT use ZooKeeper for Queues.
> >Please
> >see Tech Note 4 <https://github.com/Netflix/curator/wiki/Tech-Note-4> for
> >details. Links
> >
> >https://github.com/Netflix/curator/wiki/Distributed-Queue
> >https://github.com/Netflix/curator/wiki/Tech-Note-4
> >
> >--
> >Att.
> >Diego de Oliveira
> >System Architect
> >www.diegooliveira.com
> >Never argue with a fool -- people might not be able to tell the difference
Diego de Oliveira
System Architect
Never argue with a fool -- people might not be able to tell the difference