Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> high connection-churn solutions


Copy link to this message
-
Re: high connection-churn solutions
On Sun, Jun 10, 2012 at 2:42 PM, Adam Rosien <[EMAIL PROTECTED]> wrote:
> Various people in my company are using zk as a pure high-throughput
> read-only cache (only get() operations) and I am trying to convince them
> this is a bad idea. Specifically:
>
> * I believe their clients do not manage (or do not manage well) persistent
> connections, so there is a high amount of connection/session churn.
> * This churn results in increased quorum writes, because creating a session
> is a quorum operation. (Is this correct? Looking at snapshot log dumps, it
> seems so.)

That's correct.

> * Increased quorum writes from read-only operations is bad.

Why? Not sure I follow the thinking here. (esp if you fix session
handling to be longer lived)

>
> To avoid this situation, one could:
>
> 1. Not do this.
> 2. Create a read-only connection/session that doesn't involve the quorum or
> writing, somehow.

I believe Facebook has been working on something like this - a "server
local session" for things like r/o clients.

> 3. Something else.
>
> How could one do #2? Do observers allow #2? Any other suggestions?
>

Perhaps someone from FB might be able to comment on their progress...

Patrick
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB