Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # dev - Resource leak warnings


Copy link to this message
-
Re: Resource leak warnings
Mike Drob 2013-12-26, 18:26
I'm willing to stipulate that this solves the thread leak from web
containers - I haven't verified it, but I am ever hopeful. Does this
solution imply that we should nix the close() methods just added in the
snapshot branches?
On Mon, Dec 23, 2013 at 3:37 PM, Keith Turner <[EMAIL PROTECTED]> wrote:

> We discussed some complex solutions, but I was also thinking of a very
> simple and direct solution to the problem at hand.  Below is a very
> straightforward solution that gives web users a hammer to wack this problem
> with.   Its a very targeted utility.   Would this work?  In JBoss and
> tomcat, can users execute code on undeployment to call this?  It might be
> possible to write this utility for released versions w/ a little
> reflection.
>
>
> package org.apache.accumulo.core.util;
>
> import org.apache.accumulo.core.client.impl.TabletLocatorImpl;
> import org.apache.accumulo.core.client.impl.ThriftTransportPool;
> import org.apache.accumulo.core.zookeeper.ZooSession;
>
> public class ClientThreads {
>   /**
>    * kills all threads created by internal Accumulo singleton resources.
>  After this method is called, no accumulo client will work in the current
> classloader.
>    */
>   public static void shutdownNow(){
>     //kills all threads in transport pool and closes all connections, after
> called no threads or connections can be created
>     ThriftTransportPool.shtudownNow();
>
>     //closes all zookeepers, and does not allow any more to be created
>     ZooSession.shutdownNow();
>   }
> }
>
>
> On Mon, Dec 23, 2013 at 4:36 PM, Mike Drob <[EMAIL PROTECTED]> wrote:
>
> > Keith,
> >
> > I'm not sure I understand the alternative solution that you and
> Christopher
> > discussed. Can you explain it in a bit more detail, please? I have my own
> > interpretation of what I _think_ you were proposing, but I'd rather not
> > risk putting words in your mouth. Specifically, I'm interested in the
> > auto-cleanup aspect of things.
> >
> > I do think that the numbered changes you propose are good ideas. I also
> > agree with Christopher that we need to seriously examine the API that we
> > have, because chaining methods four- or five-deep [e.g. new
> > Instance().getConnector().tableOperations().createTable()] to do
> something
> > is not a great user experience. Determining the scope of the changes is
> > another community conversation.
> >
> > Mike
> >
> >
> > On Mon, Dec 23, 2013 at 9:17 AM, Keith Turner <[EMAIL PROTECTED]> wrote:
> >
> > > On Sun, Dec 22, 2013 at 10:28 PM, Christopher <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > On Sun, Dec 22, 2013 at 2:23 PM, Bill Havanki <
> > [EMAIL PROTECTED]
> > > >
> > > > wrote:
> > > > [snip]
> > > > > Although there was no intention of circumventing consensus, looking
> > at
> > > > the
> > > > > email exchange, consensus was clearly not reached.
> > > >
> > > > It is my understanding that typically, in CtR, consensus is needed to
> > > > resolve issues after they are committed, where there is
> > > > conflict/objections. Perhaps it was my misunderstanding of the
> > > > responses, but it was my understanding that while there was no
> > > > consensus on the final solution, there was no objection that would
> > > > have prevented the interim action taken.
> > > >
> > > > > The short time span did
> > > > > not give others the chance to work on eliminating the warnings, as
> > they
> > > > > offered, or to instead come around to just dropping Closeable.
> > > >
> > > > True... the timespan was short. My goal, as stated in the original
> > > > email, was to commit first (just like I might commit any improvement
> > > > to the current state of the code), and I intended the email to just
> be
> > > > an explanation of the reasoning, as it related to the prior commits,
> > > > and a prompt for discussion of further action. The fact that I
> > > > submitted the email chronologically first was a bit arbitrary. I
> > > > accept blame for the confusion of that, and any inciting wording the