Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> high connection-churn solutions

Copy link to this message
Re: high connection-churn solutions
On Sun, Jun 10, 2012 at 2:42 PM, Adam Rosien <[EMAIL PROTECTED]> wrote:
> Various people in my company are using zk as a pure high-throughput
> read-only cache (only get() operations) and I am trying to convince them
> this is a bad idea. Specifically:
> * I believe their clients do not manage (or do not manage well) persistent
> connections, so there is a high amount of connection/session churn.
> * This churn results in increased quorum writes, because creating a session
> is a quorum operation. (Is this correct? Looking at snapshot log dumps, it
> seems so.)

That's correct.

> * Increased quorum writes from read-only operations is bad.

Why? Not sure I follow the thinking here. (esp if you fix session
handling to be longer lived)

> To avoid this situation, one could:
> 1. Not do this.
> 2. Create a read-only connection/session that doesn't involve the quorum or
> writing, somehow.

I believe Facebook has been working on something like this - a "server
local session" for things like r/o clients.

> 3. Something else.
> How could one do #2? Do observers allow #2? Any other suggestions?

Perhaps someone from FB might be able to comment on their progress...