Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo, mail # user - PermGen leak in JBoss


+
Michael Giordano 2013-04-24, 13:48
+
Heath Abelson 2013-04-24, 15:21
+
Keith Turner 2013-04-24, 16:22
+
Michael Giordano 2013-05-03, 15:43
+
Keith Turner 2013-05-03, 15:54
+
Keith Turner 2013-04-24, 15:44
+
Eric Newton 2013-04-24, 15:47
+
Heath Abelson 2013-04-24, 16:08
Copy link to this message
-
Re: PermGen leak in JBoss
Keith Turner 2013-04-24, 16:34
If you execute the following code in the before the app is undeployed and
in the correct classloader context, then it will cause the "Thrift
Connection Pool Checker" thread to die w/ a NPE being printed.

    Field cacheField = ThriftTransportPool.class.getDeclaredField("cache");
    cacheField.setAccessible(true);
    cacheField.set(ThriftTransportPool.getInstance(), null);
On Wed, Apr 24, 2013 at 12:08 PM, Heath Abelson
<[EMAIL PROTECTED]>wrote:

>  As an additional point, we have a second webapp that also uses 1.3.7
> that also leaves the same threads – minus the 2 two watchdog threads****
>
> ** **
>
> *From:* Eric Newton [mailto:[EMAIL PROTECTED]]
> *Sent:* Wednesday, April 24, 2013 11:47 AM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: PermGen leak in JBoss****
>
> ** **
>
> One is for the dynamic reloading of the log4j configuration, IIRC.****
>
> ** **
>
> -Eric****
>
> ** **
>
> ** **
>
> On Wed, Apr 24, 2013 at 11:44 AM, Keith Turner <[EMAIL PROTECTED]> wrote:**
> **
>
> ** **
>
> ** **
>
> On Wed, Apr 24, 2013 at 11:21 AM, Heath Abelson <
> [EMAIL PROTECTED]> wrote:****
>
> Hi, I am a colleague of Michael's, I can provide more info
>
> The following are the 5 threads that are not being cleaned up on an
> undeploy. It's hard to get more stack info than this.
>
> Daemon Thread [FileWatchdog] (Running)
> Daemon Thread [MSC service thread 1-5-SendThread(cbdbtestbox:2181)]
> (Running)
> Daemon Thread [MSC service thread 1-5-EventThread] (Running)
> Daemon Thread [Thrift Connection Pool Checker] (Running)
> Daemon Thread [FileWatchdog] (Running)
>
> Using Plumbr, the report states that the SendThread and EventThread above
> both contain an instance of org.apache.zookeeper.ClientCnxn****
>
> ** **
>
> So two zookeeper threads. An Accumulo thread (Thrift Connection Pool
> Checker).  What are the FileWatchdog threads?  Those could be related to
> accumulo-start or hdfs.****
>
>  ****
>
>
> -Heath****
>
>
> -----Original Message-----
> From: Michael Giordano
> Sent: Wednesday, April 24, 2013 9:48 AM
> To: [EMAIL PROTECTED]
> Subject: PermGen leak in JBoss
>
> Under version 1.3.7 we are using the following code to initialize a
> cloudbase connection during initialization of our web app:
>
>                         ZooKeeperInstance instance = new
> ZooKeeperInstance(instanceName, zooKeepers);
>                         connector = instance.getConnector(userId,
> password.getBytes());
>
> The problem is that under the hood, this call creates several (5) threads
> that are not cleaned up when the app is undeployed in JBoss. This is
> occurring without performing any scans or interacting with cloudbase in any
> other way. After relatively few redeploys of the app, the PermGen Space is
> OOM.
>
> I can't find any reference in the cloudbase API akin to a close() method
> for the Connector object. This is a classloader leak effecting any webapp
> that is accessing cloudbase directly. The result of this leak is not simply
> orphaned threads, but thousands of classes not gc'd because the classloader
> itself can't be gc'd. This is what is filling up PermGen.
>
> Has anyone discovered this particular issue? Has anyone discovered a
> solution?
>
> Thanks,
> Mike G.
>
> NetCentric Technology, Inc.
> 3349 Route 138, Building A
> Wall, NJ  07719
> Phone: 732-544-0888
>
> ****
>
>  ** **
>
> ** **
>
+
Benson Margulies 2013-04-24, 16:58
+
Heath Abelson 2013-04-24, 16:58
+
Josh Elser 2013-04-24, 14:12
+
Keith Turner 2013-04-24, 14:28