Various people in my company are using zk as a pure high-throughput
read-only cache (only get() operations) and I am trying to convince them
this is a bad idea. Specifically:

* I believe their clients do not manage (or do not manage well) persistent
connections, so there is a high amount of connection/session churn.
* This churn results in increased quorum writes, because creating a session
is a quorum operation. (Is this correct? Looking at snapshot log dumps, it
seems so.)
* Increased quorum writes from read-only operations is bad.

To avoid this situation, one could:

1. Not do this.
2. Create a read-only connection/session that doesn't involve the quorum or
writing, somehow.
3. Something else.

How could one do #2? Do observers allow #2? Any other suggestions?

thanks,

.. Adam

On Sun, Jun 10, 2012 at 2:42 PM, Adam Rosien <[EMAIL PROTECTED]> wrote:
> Various people in my company are using zk as a pure high-throughput
> read-only cache (only get() operations) and I am trying to convince them
> this is a bad idea. Specifically:
>
> * I believe their clients do not manage (or do not manage well) persistent
> connections, so there is a high amount of connection/session churn.
> * This churn results in increased quorum writes, because creating a session
> is a quorum operation. (Is this correct? Looking at snapshot log dumps, it
> seems so.)

That's correct.

> * Increased quorum writes from read-only operations is bad.

Why? Not sure I follow the thinking here. (esp if you fix session
handling to be longer lived)

>
> To avoid this situation, one could:
>
> 1. Not do this.
> 2. Create a read-only connection/session that doesn't involve the quorum or
> writing, somehow.

I believe Facebook has been working on something like this - a "server
local session" for things like r/o clients.

> 3. Something else.
>
> How could one do #2? Do observers allow #2? Any other suggestions?
>

Perhaps someone from FB might be able to comment on their progress...

Patrick

Assuming you can't modify the code to keep sessions around longer and your
queries to zookeeper aren't too varied you could have a REST endpoint web
service type somewhere that creates a single session and keeps it alive
indefinitely.  Your existing suite of clients don't have to change their
behavior and you can avoid the overhead of creating and destroying sessions
at the same time.

Mark

On Mon, Jun 11, 2012 at 11:09 PM, Patrick Hunt <[EMAIL PROTECTED]> wrote:

> On Sun, Jun 10, 2012 at 2:42 PM, Adam Rosien <[EMAIL PROTECTED]> wrote:
> > Various people in my company are using zk as a pure high-throughput
> > read-only cache (only get() operations) and I am trying to convince them
> > this is a bad idea. Specifically:
> >
> > * I believe their clients do not manage (or do not manage well)
> persistent
> > connections, so there is a high amount of connection/session churn.
> > * This churn results in increased quorum writes, because creating a
> session
> > is a quorum operation. (Is this correct? Looking at snapshot log dumps,
> it
> > seems so.)
>
> That's correct.
>
> > * Increased quorum writes from read-only operations is bad.
>
> Why? Not sure I follow the thinking here. (esp if you fix session
> handling to be longer lived)
>
> >
> > To avoid this situation, one could:
> >
> > 1. Not do this.
> > 2. Create a read-only connection/session that doesn't involve the quorum
> or
> > writing, somehow.
>
> I believe Facebook has been working on something like this - a "server
> local session" for things like r/o clients.
>
> > 3. Something else.
> >
> > How could one do #2? Do observers allow #2? Any other suggestions?
> >
>
> Perhaps someone from FB might be able to comment on their progress...
>
> Patrick
>

On Mon, Jun 11, 2012 at 11:09 PM, Patrick Hunt <[EMAIL PROTECTED]> wrote:

> On Sun, Jun 10, 2012 at 2:42 PM, Adam Rosien <[EMAIL PROTECTED]> wrote:
> > Various people in my company are using zk as a pure high-throughput
> > read-only cache (only get() operations) and I am trying to convince them
> > this is a bad idea. Specifically:
> >
> > * I believe their clients do not manage (or do not manage well)
> persistent
> > connections, so there is a high amount of connection/session churn.
> > * This churn results in increased quorum writes, because creating a
> session
> > is a quorum operation. (Is this correct? Looking at snapshot log dumps,
> it
> > seems so.)
>
> That's correct.
>
> > * Increased quorum writes from read-only operations is bad.
>
> Why? Not sure I follow the thinking here. (esp if you fix session
> handling to be longer lived)
>

Only because the workload is read-only and short-lived, and I can't make
the sessions long-lived, they are from a web server. :/
>
> >
> > To avoid this situation, one could:
> >
> > 1. Not do this.
> > 2. Create a read-only connection/session that doesn't involve the quorum
> or
> > writing, somehow.
>
> I believe Facebook has been working on something like this - a "server
> local session" for things like r/o clients.
>
> > 3. Something else.
> >
> > How could one do #2? Do observers allow #2? Any other suggestions?
> >
>
> Perhaps someone from FB might be able to comment on their progress...
>
> Patrick
>

That's exactly what we've proposed :)

On Tue, Jun 12, 2012 at 12:36 AM, Mark Gius <[EMAIL PROTECTED]> wrote:

> Assuming you can't modify the code to keep sessions around longer and your
> queries to zookeeper aren't too varied you could have a REST endpoint web
> service type somewhere that creates a single session and keeps it alive
> indefinitely.  Your existing suite of clients don't have to change their
> behavior and you can avoid the overhead of creating and destroying sessions
> at the same time.
>
> Mark
>
> On Mon, Jun 11, 2012 at 11:09 PM, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>
> > On Sun, Jun 10, 2012 at 2:42 PM, Adam Rosien <[EMAIL PROTECTED]> wrote:
> > > Various people in my company are using zk as a pure high-throughput
> > > read-only cache (only get() operations) and I am trying to convince
> them
> > > this is a bad idea. Specifically:
> > >
> > > * I believe their clients do not manage (or do not manage well)
> > persistent
> > > connections, so there is a high amount of connection/session churn.
> > > * This churn results in increased quorum writes, because creating a
> > session
> > > is a quorum operation. (Is this correct? Looking at snapshot log dumps,
> > it
> > > seems so.)
> >
> > That's correct.
> >
> > > * Increased quorum writes from read-only operations is bad.
> >
> > Why? Not sure I follow the thinking here. (esp if you fix session
> > handling to be longer lived)
> >
> > >
> > > To avoid this situation, one could:
> > >
> > > 1. Not do this.
> > > 2. Create a read-only connection/session that doesn't involve the
> quorum
> > or
> > > writing, somehow.
> >
> > I believe Facebook has been working on something like this - a "server
> > local session" for things like r/o clients.
> >
> > > 3. Something else.
> > >
> > > How could one do #2? Do observers allow #2? Any other suggestions?
> > >
> >
> > Perhaps someone from FB might be able to comment on their progress...
> >
> > Patrick
> >
>

You've looked at the rest proxy included in ZK's src/contrib/rest?

Patrick

On Tue, Jun 12, 2012 at 7:37 AM, Adam Rosien <[EMAIL PROTECTED]> wrote:
> That's exactly what we've proposed :)
>
> On Tue, Jun 12, 2012 at 12:36 AM, Mark Gius <[EMAIL PROTECTED]> wrote:
>
>> Assuming you can't modify the code to keep sessions around longer and your
>> queries to zookeeper aren't too varied you could have a REST endpoint web
>> service type somewhere that creates a single session and keeps it alive
>> indefinitely.  Your existing suite of clients don't have to change their
>> behavior and you can avoid the overhead of creating and destroying sessions
>> at the same time.
>>
>> Mark
>>
>> On Mon, Jun 11, 2012 at 11:09 PM, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>>
>> > On Sun, Jun 10, 2012 at 2:42 PM, Adam Rosien <[EMAIL PROTECTED]> wrote:
>> > > Various people in my company are using zk as a pure high-throughput
>> > > read-only cache (only get() operations) and I am trying to convince
>> them
>> > > this is a bad idea. Specifically:
>> > >
>> > > * I believe their clients do not manage (or do not manage well)
>> > persistent
>> > > connections, so there is a high amount of connection/session churn.
>> > > * This churn results in increased quorum writes, because creating a
>> > session
>> > > is a quorum operation. (Is this correct? Looking at snapshot log dumps,
>> > it
>> > > seems so.)
>> >
>> > That's correct.
>> >
>> > > * Increased quorum writes from read-only operations is bad.
>> >
>> > Why? Not sure I follow the thinking here. (esp if you fix session
>> > handling to be longer lived)
>> >
>> > >
>> > > To avoid this situation, one could:
>> > >
>> > > 1. Not do this.
>> > > 2. Create a read-only connection/session that doesn't involve the
>> quorum
>> > or
>> > > writing, somehow.
>> >
>> > I believe Facebook has been working on something like this - a "server
>> > local session" for things like r/o clients.
>> >
>> > > 3. Something else.
>> > >
>> > > How could one do #2? Do observers allow #2? Any other suggestions?
>> > >
>> >
>> > Perhaps someone from FB might be able to comment on their progress...
>> >
>> > Patrick
>> >
>>