Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper, mail # user - Zookeeper protocol weirdness and pure Python kazoo client

Copy link to this message
Zookeeper protocol weirdness and pure Python kazoo client
Ben Bangert 2012-08-30, 23:00
Based on the pookeeper code (https://github.com/maguro/pookeeper), I've implemented a version of kazoo that uses the wire protocol and is pure Python (https://github.com/python-zk/kazoo/tree/pure-python). Speaking to Zookeeper directly has been quite interesting. All the documentation assumes one uses the C/Java libs, so what actually happens doesn't seem to be cleanly or clearly defined anywhere beyond the implementations. Is there an actual spec somewhere?

So far, I've found that if I send malformed Auth packets that are missing the auth_type int, after a few times I can get the Zookeeper server to segfault. I'll attach some more log scripts and a test script to trigger it next.

One of the oddities of passing a bad auth packet (ie, saying 'asdfjasdf' instead of 'digest'), is that Zookeeper returns an auth packet -4 with an error header, and then drops the connections. In the state diagram, the client is expected to 'end'. That's fine and all, my client happened to reconnect the first time with the prior session id/password.... which was still valid. Is this expected behavior?

Why does Zookeeper drop the connection with a bad auth packet... but doesn't immediately terminate the session id? If the session ID is not supposed to be used, why not expire it? I wonder how much code out there is written with the assumption that once you see a AUTH_FAILED state, *your session is dead and your nodes are deleted*?

This isn't actually the case though, you get an AUTH_FAILED, the socket is dropped, and a reconnect with the same session id/password works just fine. Why bother dropping the connection to begin with?

If anyone is interested in why I decided to go with pure Python vs. the C lib/binding, I'd be happy to espouse on that in a separate thread.

Ben Bangert 2012-08-30, 23:02
Michi Mutsuzaki 2012-08-30, 23:50
Ben Bangert 2012-08-31, 00:06
Henry Robinson 2012-08-31, 00:10
Ben Bangert 2012-08-31, 04:00
Ben Bangert 2012-08-31, 05:04
Henry Robinson 2012-08-31, 06:57
Ben Bangert 2012-08-31, 16:39
Michi Mutsuzaki 2012-08-31, 21:30
Ted Dunning 2012-08-31, 20:02
Ben Bangert 2012-08-30, 23:35
Michi Mutsuzaki 2012-08-30, 23:37