Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Zookeeper performance

Copy link to this message
RE: Zookeeper performance
We are using optimistic concurrency similar to one explained in http://en.wikipedia.org/wiki/Optimistic_concurrency_control
I went through the "version" concept in zookeeper. I do not understand how that could be used for "concurrency control". Are you suggesting that all writes go to zookeeper first before making it to the database.
We are trying to do it differently... For example, let us say we are trying to update rowKey 1 of Person table,
1. Create a ephemeral node /locks/Person/rowkey12. Zookeeper may create a node called /locks/Person/rowKey1-0000 (This means that we got lock)3. If we got, /locks/Person/rowKey1-0001 , we decrement and keep checking child nodes of /locks/Person until  /locks/Person/rowKey1-0001 becomes the first one in the list.
Pls help/

> Date: Wed, 31 Jul 2013 08:47:01 -0700
> Subject: Re: Zookeeper performance
> On Wed, Jul 31, 2013 at 4:16 AM, Baskar Duraikannu
> <[EMAIL PROTECTED]> wrote:
> >
> > We are looking to use zookeeper for optimistic concurrency. Basically when the user saves data on a screen, we need to lock,  read to ensure that no one else has changed the row while user is editing data, persist data and unlock znode.
> >
> Why not use the built in optimistic locking that ZK provides? See the
> "version" parameter here:
> http://zookeeper.apache.org/doc/r3.4.5/api/org/apache/zookeeper/ZooKeeper.html#getData(java.lang.String,
> boolean, org.apache.zookeeper.AsyncCallback.DataCallback,
> java.lang.Object)
> http://zookeeper.apache.org/doc/r3.4.5/api/org/apache/zookeeper/ZooKeeper.html#setData(java.lang.String,
> byte[], int, org.apache.zookeeper.AsyncCallback.StatCallback,
> java.lang.Object)
> > If the app/thread does not get a lock, we may set a watch so that polling is avoided.
> >
> > Our application is write intensive certain times of the day. We may get about 100k requests per second.  Can zookeeper handle this volume?
> Unlikely you'll see this level of performance:
> http://zookeeper.apache.org/doc/r3.4.5/zookeeperOver.html#Performance
> Patrick