Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # dev >> An updated ZooKeeper RESTful API - Zest


Copy link to this message
-
Re: An updated ZooKeeper RESTful API - Zest
FYI - A Thrift proxy for Curator will be in Curator’s next release. I mentioned it on this list a few weeks ago:

http://mail-archives.apache.org/mod_mbox/zookeeper-user/201406.mbox/%[EMAIL PROTECTED]l%3E

-Jordan
From: Van Klaveren, Brian N. [EMAIL PROTECTED]
Reply: [EMAIL PROTECTED] [EMAIL PROTECTED]
Date: June 26, 2014 at 3:03:31 PM
To: [EMAIL PROTECTED] [EMAIL PROTECTED]
Subject:  An updated ZooKeeper RESTful API - Zest  

Hi All,  

I’m new to this list.  

I was wondering who I could contact, if this list isn’t appropriate, to possibly discuss design decisions of an update to the RESTful API.  

I recently started working on an update and partial rewrite of the ZooKeeper RESTful API, using Jersey 2 namely, and with support for asynchronous requests/long polling on events. I did this after exploring using etcd possibly, but realizing their API is rather inadequate for locking, and overall the project isn’t very mature. In order to prevent some confusion, I’ve named my project “Zest”, though it largely derives from the current RESTful API.  

One major difference with Zest is that there is an “events” resource. This is a resource a client can poll or long-poll to receive events. My hope was that Zest, through the events resource, could initially support polling and long-polling, with the possibility of plugging in WebSockets (and maybe Server-Side events) in the future. I actually played around with support for Server-Side events, but I was unable to easily program a responsive client in JavaScript or Python.  

I did look into integrating some of this into the existing ZooKeeper server somehow in order to reduce the additional memory and processing overhead, but that didn’t appear to me to be easy or straightforward at the level of abstraction that currently exists.  

On the other hand, I have created a PoC multi-threaded client based on zkclient which appears to work similarly, except that it interacts only with Zest.  

If this all sounds like it could be useful, I’d be interested in collaborating with people to work on the project and discussing an API.  

Brian  
Some info on differences in the Zest implementation:  
A Zest session is effectively an HTTP session currently. This could be changed, but I just used this for ease of implementation. Some people might say this isn’t “RESTful”, but I think that’s okay.  

A Zest session is fully initiated when a client POSTs to the sessions resource. This creates a ZkSessionProxy object, which mainly handles the ZooKeeper client, schedules, cancels and resets ZooKeeper client reapers, and temporarily stores suspended AsyncResponses for long polling, as well as watched events for the events resource. In the future, it could possibly maintain dispatching events and via WebSockets.  

The Zest implementation currently defines it’s own ticks, where the tickTime is half of the ZooKeeper session timeout. A ZooKeeper client reaper is scheduled to be executed at two ticks, or effectively the same time as the session timeout.  

Every incoming HTTP request cancels and reschedules the ZooKeeper client reaper, similar to the current way. A GET request on the sessions resource will act as a generic keep-alive.  

With long-polling, there is a asynchronous request timeout set on a one-tick interval. When that timeout occurs, a “204 NO CONTENT” is sent, and the client should renew it’s request. When that request is renewed, the reaper is canceled and then reset. In this sense, a thread that only watches for events functions as a heartbeat.  

If a ZooKeeper client session is reaped, then the ZooKeeper client is closed. Then, a Zest session reaper is scheduled. This is to allow clients which are unresponsive to discover that they have, in fact, been disconnected when polling the events resource, as well as retrieve any events which may have arrived to the HTTP server before they were disconnected, if they are able to query the web service within a few ticks. Once the Zest session reaper is run, it removes all events and references to the Zest session  

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB