Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> EvalFunc finish() closing connections prematurely


+
Mike Sukmanowsky 2013-03-14, 21:05
+
Jonathan Coveney 2013-03-14, 21:24
+
Mike Sukmanowsky 2013-03-18, 20:53
+
Mike Sukmanowsky 2013-03-22, 15:28
Copy link to this message
-
Re: EvalFunc finish() closing connections prematurely
Mike, have you tried adding logging to any EvalFunc methods that
communicate with Mongo to see which of them is calling it after finish() ?
Are you sure something else doesn't close Mongo connection for you?
On Fri, Mar 22, 2013 at 8:28 AM, Mike Sukmanowsky <[EMAIL PROTECTED]> wrote:

> Bump - any thoughts?
>
>
> On Mon, Mar 18, 2013 at 4:53 PM, Mike Sukmanowsky <[EMAIL PROTECTED]>
> wrote:
>
> > Sure thing - here you go
> > https://gist.github.com/msukmanowsky/198c878e1deaf4e470a9
> >
> > When initialized, the ParselyMetadataService creates a new Mongo and
> Jedis
> > instance which the EvalFunc queries using a public method fetch().
> >  Instance of ParselyMetadataService also have a close() function which
> > simply calls:
> >
> > this.mongoClient.close();
> > this.redisClient.disconnect();
> >
> >
> > On Thu, Mar 14, 2013 at 5:24 PM, Jonathan Coveney <[EMAIL PROTECTED]
> >wrote:
> >
> >> Can you perhaps share more of your implementation? I can imagine a
> couple
> >> of things which would cause errors like this. Are you making sure that
> >> each
> >> instance of EvalFunc is dealing with a different connection?
> >>
> >> That's what I'd take a look at first...if that isn't the issue, I can
> look
> >> into how final() is implemented.
> >>
> >>
> >> 2013/3/14 Mike Sukmanowsky <[EMAIL PROTECTED]>
> >>
> >> > Hi there,
> >> >
> >> > I have an EvalFunc which uses an internal class that opens up
> >> connections
> >> > to a Redis and MongoDB server.  This class has a close() method which
> >> > closes connections to both Redis and MongoDB.
> >> >
> >> > In my EvalFunc implementation, I placed the close() method
> >> >
> >> >     @Override
> >> >   public void finish() {
> >> >   meta.close();
> >> >   }
> >> >
> >> > But this is leading to a
> >> >
> >> > java.lang.IllegalStateException: this Mongo has been closed
> >> >
> >> > indicating that Pig attempted to read from Mongo after a call to
> >> finish().
> >> >  Is this an error specific to running a Pig script locally?
> >> >
> >> > Mike
> >> >
> >> > --
> >> > Mike Sukmanowsky
> >> >
> >> > Product Lead, http://parse.ly
> >> > 989 Avenue of the Americas, 3rd Floor
> >> > New York, NY  10018
> >> > p: +1 (416) 953-4248
> >> > e: [EMAIL PROTECTED]
> >> >
> >>
> >
> >
> >
> > --
> > Mike Sukmanowsky
> >
> > Product Lead, http://parse.ly
> > 989 Avenue of the Americas, 3rd Floor
> > New York, NY  10018
> > p: +1 (416) 953-4248
> > e: [EMAIL PROTECTED]
> >
>
>
>
> --
> Mike Sukmanowsky
>
> Product Lead, http://parse.ly
> 989 Avenue of the Americas, 3rd Floor
> New York, NY  10018
> p: +1 (416) 953-4248
> e: [EMAIL PROTECTED]
>
+
Jonathan Coveney 2013-03-26, 14:48