Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper, mail # user - high connection-churn solutions


Copy link to this message
-
Re: high connection-churn solutions
Patrick Hunt 2012-06-15, 18:07
You've looked at the rest proxy included in ZK's src/contrib/rest?

Patrick

On Tue, Jun 12, 2012 at 7:37 AM, Adam Rosien <[EMAIL PROTECTED]> wrote:
> That's exactly what we've proposed :)
>
> On Tue, Jun 12, 2012 at 12:36 AM, Mark Gius <[EMAIL PROTECTED]> wrote:
>
>> Assuming you can't modify the code to keep sessions around longer and your
>> queries to zookeeper aren't too varied you could have a REST endpoint web
>> service type somewhere that creates a single session and keeps it alive
>> indefinitely.  Your existing suite of clients don't have to change their
>> behavior and you can avoid the overhead of creating and destroying sessions
>> at the same time.
>>
>> Mark
>>
>> On Mon, Jun 11, 2012 at 11:09 PM, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>>
>> > On Sun, Jun 10, 2012 at 2:42 PM, Adam Rosien <[EMAIL PROTECTED]> wrote:
>> > > Various people in my company are using zk as a pure high-throughput
>> > > read-only cache (only get() operations) and I am trying to convince
>> them
>> > > this is a bad idea. Specifically:
>> > >
>> > > * I believe their clients do not manage (or do not manage well)
>> > persistent
>> > > connections, so there is a high amount of connection/session churn.
>> > > * This churn results in increased quorum writes, because creating a
>> > session
>> > > is a quorum operation. (Is this correct? Looking at snapshot log dumps,
>> > it
>> > > seems so.)
>> >
>> > That's correct.
>> >
>> > > * Increased quorum writes from read-only operations is bad.
>> >
>> > Why? Not sure I follow the thinking here. (esp if you fix session
>> > handling to be longer lived)
>> >
>> > >
>> > > To avoid this situation, one could:
>> > >
>> > > 1. Not do this.
>> > > 2. Create a read-only connection/session that doesn't involve the
>> quorum
>> > or
>> > > writing, somehow.
>> >
>> > I believe Facebook has been working on something like this - a "server
>> > local session" for things like r/o clients.
>> >
>> > > 3. Something else.
>> > >
>> > > How could one do #2? Do observers allow #2? Any other suggestions?
>> > >
>> >
>> > Perhaps someone from FB might be able to comment on their progress...
>> >
>> > Patrick
>> >
>>