Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> high connection-churn solutions


Copy link to this message
-
Re: high connection-churn solutions
Assuming you can't modify the code to keep sessions around longer and your
queries to zookeeper aren't too varied you could have a REST endpoint web
service type somewhere that creates a single session and keeps it alive
indefinitely.  Your existing suite of clients don't have to change their
behavior and you can avoid the overhead of creating and destroying sessions
at the same time.

Mark

On Mon, Jun 11, 2012 at 11:09 PM, Patrick Hunt <[EMAIL PROTECTED]> wrote:

> On Sun, Jun 10, 2012 at 2:42 PM, Adam Rosien <[EMAIL PROTECTED]> wrote:
> > Various people in my company are using zk as a pure high-throughput
> > read-only cache (only get() operations) and I am trying to convince them
> > this is a bad idea. Specifically:
> >
> > * I believe their clients do not manage (or do not manage well)
> persistent
> > connections, so there is a high amount of connection/session churn.
> > * This churn results in increased quorum writes, because creating a
> session
> > is a quorum operation. (Is this correct? Looking at snapshot log dumps,
> it
> > seems so.)
>
> That's correct.
>
> > * Increased quorum writes from read-only operations is bad.
>
> Why? Not sure I follow the thinking here. (esp if you fix session
> handling to be longer lived)
>
> >
> > To avoid this situation, one could:
> >
> > 1. Not do this.
> > 2. Create a read-only connection/session that doesn't involve the quorum
> or
> > writing, somehow.
>
> I believe Facebook has been working on something like this - a "server
> local session" for things like r/o clients.
>
> > 3. Something else.
> >
> > How could one do #2? Do observers allow #2? Any other suggestions?
> >
>
> Perhaps someone from FB might be able to comment on their progress...
>
> Patrick
>