Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> How to deal with fork() properly when using the zkc mt lib


+
Jonathan Simms 2012-05-10, 19:45
+
Michi Mutsuzaki 2012-05-10, 22:41
+
Jonathan Simms 2012-05-10, 23:11
+
Martin Kou 2012-05-10, 23:51
+
Martin Kou 2012-05-10, 23:53
+
Jonathan Simms 2012-05-13, 03:06
+
Jonathan Simms 2012-05-11, 01:46
+
Jonathan Simms 2012-05-11, 04:19
Copy link to this message
-
Re: How to deal with fork() properly when using the zkc mt lib
Jon,

Hmm... I'm not sure when you're going to fork() though. What I've done
before was to do the fork() before each zookeeper_init(). The simplest
scheme is for each process to hold one event loop. Each event loop can then
be shared by as many single-threaded Zookeeper sessions as you see fit. So
let's say you pre-fork() 4 processes, and each process runs 4 Zookeeper
sessions - you'll be able to run 16 Zookeeper sessions in parallel.

If you do the fork() after zookeeper_init() - then it will get messier. As
I understand each forked process will increment the reference count on any
opened file descriptors. So you'll have to take care to close the "shared"
file descriptors in every "other" process before you call zookeeper_close().

Best Regards,
Martin Kou

On Thu, May 10, 2012 at 9:19 PM, Jonathan Simms <[EMAIL PROTECTED]> wrote:

> Well afaict SO_NOSIGPIPE doesn't exist in linux, which kinda sucks, as
> I need this to be cross platform. I even tried hacking the source to
> allow an option to not send the last message (diff is here:
> http://is.gd/NptC0n and yes, I know this is an incredibly naive
> attempt).
>
> This would be an incredibly useful feature in my case (disconnect the
> client and resume the session within the negotiated timeout).
>
> BTW, is it possible to fork safely when using the st library?
>
> Thanks
>
> On Thu, May 10, 2012 at 9:46 PM, Jonathan Simms <[EMAIL PROTECTED]> wrote:
> > wow
> >
> > That's scary, but, probably also useful. :)
> >
> > I'm considering rewriting this using the st library, considering all
> > the craziness necessary to use the mt lib.
> >
> > I'm gonna go try that out.
> >
> > On Thu, May 10, 2012 at 7:53 PM, Martin Kou <[EMAIL PROTECTED]> wrote:
> >> If you don't mind the hackish-ness, I think you can just grab the file
> >> descriptor from a Zookeeper handle like this for mt -
> >>
> >> int fd = ((int *)zhandle)[0];
> >>
> >> This works because the fd is the first field in the _zhandle struct.
> >>
> >> Best Regards,
> >> Martin Kou
> >>
> >> On Thu, May 10, 2012 at 4:51 PM, Martin Kou <[EMAIL PROTECTED]>
> wrote:
> >>
> >>> I've had a similar problem as well, but I've been using the single
> >>> threaded async library - I actually find it simpler to use than the mt
> >>> library.
> >>>
> >>> The way I do it is this:
> >>>
> >>> During session connect -
> >>>  1. Grab the file descriptor from the C library via
> zookeeper_interest()
> >>>  2. If this is the first time I saw this file descriptor, and it's
> valid,
> >>> do a setsockopt() on it to set SO_NOSIGPIPE to 1.
> >>>
> >>> When I need to "suspend" the session
> >>>  1. close() the file descriptor
> >>>  2. call zookeeper_close() on the handle
> >>>
> >>> zookeeper_close() will try to send the close session message at step 2
> >>> here. Normally, that would cause a SIGPIPE and your app would crash -
> but
> >>> this time it won't because you've set SO_NOSIGPIPE on the socket.
> Instead,
> >>> the Zookeeper library will see a regular error from its send operation
> and
> >>> it'll free up the handle peacefully without closing the session.
> >>>
> >>> Best Regards,
> >>> Martin Kou
> >>>
> >>>
> >>> On Thu, May 10, 2012 at 4:11 PM, Jonathan Simms <[EMAIL PROTECTED]>
> wrote:
> >>>
> >>>> Michi, fair point, I actually just looked into it, there doesn't seem
> >>>> to be a way through the api to re-establish the session. If you call
> >>>> zookeeper_close on the handle:
> >>>>
> >>>>  "After this call, the client session will no longer be valid. The
> >>>> function will flush any outstanding send requests before return. As a
> >>>> result it may block."
> >>>>
> >>>> I tried:
> >>>>
> >>>> * establish session with handle A
> >>>> * copy clientid_t from handle A
> >>>> * zookeeper_close handle A
> >>>> * construct handle B using clientid_t values from handle A
> >>>>
> >>>> I get back a SESSION_EXPIRED from the server. (debug from mt lib here:
> >>>> https://gist.github.com/3b7e4060746d03cef287)
> >>>>
> >>>> It would be *really* useful if i could basically "suspend" a session
+
Jonathan Simms 2012-05-13, 02:42
+
Michi Mutsuzaki 2012-05-10, 23:49