Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> How to deal with fork() properly when using the zkc mt lib


+
Jonathan Simms 2012-05-10, 19:45
+
Michi Mutsuzaki 2012-05-10, 22:41
+
Jonathan Simms 2012-05-10, 23:11
Copy link to this message
-
Re: How to deal with fork() properly when using the zkc mt lib
I've had a similar problem as well, but I've been using the single threaded
async library - I actually find it simpler to use than the mt library.

The way I do it is this:

During session connect -
 1. Grab the file descriptor from the C library via zookeeper_interest()
 2. If this is the first time I saw this file descriptor, and it's valid,
do a setsockopt() on it to set SO_NOSIGPIPE to 1.

When I need to "suspend" the session
 1. close() the file descriptor
 2. call zookeeper_close() on the handle

zookeeper_close() will try to send the close session message at step 2
here. Normally, that would cause a SIGPIPE and your app would crash - but
this time it won't because you've set SO_NOSIGPIPE on the socket. Instead,
the Zookeeper library will see a regular error from its send operation and
it'll free up the handle peacefully without closing the session.

Best Regards,
Martin Kou

On Thu, May 10, 2012 at 4:11 PM, Jonathan Simms <[EMAIL PROTECTED]> wrote:

> Michi, fair point, I actually just looked into it, there doesn't seem
> to be a way through the api to re-establish the session. If you call
> zookeeper_close on the handle:
>
>  "After this call, the client session will no longer be valid. The
> function will flush any outstanding send requests before return. As a
> result it may block."
>
> I tried:
>
> * establish session with handle A
> * copy clientid_t from handle A
> * zookeeper_close handle A
> * construct handle B using clientid_t values from handle A
>
> I get back a SESSION_EXPIRED from the server. (debug from mt lib here:
> https://gist.github.com/3b7e4060746d03cef287)
>
> It would be *really* useful if i could basically "suspend" a session
> while i forked, then reconnect and pick up where i left off. Is this
> not possible?
>
> On Thu, May 10, 2012 at 6:41 PM, Michi Mutsuzaki <[EMAIL PROTECTED]>
> wrote:
> > Hi Jonathan,
> >
> > It would be very difficult to share multi-threaded zk handle with
> > child process. I'm surprised it actually works on mac. I think saving
> > session id/password and re-establishing the session in the child
> > process is more robust and platform independent.
> >
> > Thanks!
> > --Michi
>
>
> >
> > On Thu, May 10, 2012 at 12:45 PM, Jonathan Simms <[EMAIL PROTECTED]>
> wrote:
> >> Hi all,
> >>
> >> I'm the maintainer of the ruby zookeeper library, and I'm having
> >> trouble getting consistent behavior when a user calls fork(). When
> >> developing it on MacOS (using 3.3.5), I was able to fork, then
> >> immediately call zookeeper_close() in the child, and then create a new
> >> handle. Testing on Linux, the behavior is much more unpredictable.
> >> Regularly, it seems there are segfaults when calling zookeeper_close.
> >> https://gist.github.com/22338464cd47e0e50970
> >>
> >>
> >> So I guess my question is, is there any safe way to fork() while the
> >> client is running?
> >>
> >> Another possibility i thought of is to note the session id/passwd,
> >> close the client, fork, then re-open with the same id/passwd to
> >> re-establish the session in the parent.
> >>
> >> Any recommendations?
>
+
Martin Kou 2012-05-10, 23:53
+
Jonathan Simms 2012-05-13, 03:06
+
Jonathan Simms 2012-05-11, 01:46
+
Jonathan Simms 2012-05-11, 04:19
+
Martin Kou 2012-05-11, 06:41
+
Jonathan Simms 2012-05-13, 02:42
+
Michi Mutsuzaki 2012-05-10, 23:49
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB