Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # user >> PermGen leak in JBoss


+
Michael Giordano 2013-04-24, 13:48
+
Heath Abelson 2013-04-24, 15:21
+
Keith Turner 2013-04-24, 16:22
+
Michael Giordano 2013-05-03, 15:43
+
Keith Turner 2013-05-03, 15:54
+
Keith Turner 2013-04-24, 15:44
+
Eric Newton 2013-04-24, 15:47
+
Heath Abelson 2013-04-24, 16:08
+
Keith Turner 2013-04-24, 16:34
Copy link to this message
-
Re: PermGen leak in JBoss
would it not be better to make the accumulo connection a JNDI resource
outside the webapp?
On Wed, Apr 24, 2013 at 12:34 PM, Keith Turner <[EMAIL PROTECTED]> wrote:

> If you execute the following code in the before the app is undeployed and
> in the correct classloader context, then it will cause the "Thrift
> Connection Pool Checker" thread to die w/ a NPE being printed.
>
>     Field cacheField = ThriftTransportPool.class.getDeclaredField("cache");
>     cacheField.setAccessible(true);
>     cacheField.set(ThriftTransportPool.getInstance(), null);
>
>
> On Wed, Apr 24, 2013 at 12:08 PM, Heath Abelson <
> [EMAIL PROTECTED]> wrote:
>
>>  As an additional point, we have a second webapp that also uses 1.3.7
>> that also leaves the same threads – minus the 2 two watchdog threads****
>>
>> ** **
>>
>> *From:* Eric Newton [mailto:[EMAIL PROTECTED]]
>> *Sent:* Wednesday, April 24, 2013 11:47 AM
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Re: PermGen leak in JBoss****
>>
>> ** **
>>
>> One is for the dynamic reloading of the log4j configuration, IIRC.****
>>
>> ** **
>>
>> -Eric****
>>
>> ** **
>>
>> ** **
>>
>> On Wed, Apr 24, 2013 at 11:44 AM, Keith Turner <[EMAIL PROTECTED]> wrote:*
>> ***
>>
>> ** **
>>
>> ** **
>>
>> On Wed, Apr 24, 2013 at 11:21 AM, Heath Abelson <
>> [EMAIL PROTECTED]> wrote:****
>>
>> Hi, I am a colleague of Michael's, I can provide more info
>>
>> The following are the 5 threads that are not being cleaned up on an
>> undeploy. It's hard to get more stack info than this.
>>
>> Daemon Thread [FileWatchdog] (Running)
>> Daemon Thread [MSC service thread 1-5-SendThread(cbdbtestbox:2181)]
>> (Running)
>> Daemon Thread [MSC service thread 1-5-EventThread] (Running)
>> Daemon Thread [Thrift Connection Pool Checker] (Running)
>> Daemon Thread [FileWatchdog] (Running)
>>
>> Using Plumbr, the report states that the SendThread and EventThread above
>> both contain an instance of org.apache.zookeeper.ClientCnxn****
>>
>> ** **
>>
>> So two zookeeper threads. An Accumulo thread (Thrift Connection Pool
>> Checker).  What are the FileWatchdog threads?  Those could be related to
>> accumulo-start or hdfs.****
>>
>>  ****
>>
>>
>> -Heath****
>>
>>
>> -----Original Message-----
>> From: Michael Giordano
>> Sent: Wednesday, April 24, 2013 9:48 AM
>> To: [EMAIL PROTECTED]
>> Subject: PermGen leak in JBoss
>>
>> Under version 1.3.7 we are using the following code to initialize a
>> cloudbase connection during initialization of our web app:
>>
>>                         ZooKeeperInstance instance = new
>> ZooKeeperInstance(instanceName, zooKeepers);
>>                         connector = instance.getConnector(userId,
>> password.getBytes());
>>
>> The problem is that under the hood, this call creates several (5) threads
>> that are not cleaned up when the app is undeployed in JBoss. This is
>> occurring without performing any scans or interacting with cloudbase in any
>> other way. After relatively few redeploys of the app, the PermGen Space is
>> OOM.
>>
>> I can't find any reference in the cloudbase API akin to a close() method
>> for the Connector object. This is a classloader leak effecting any webapp
>> that is accessing cloudbase directly. The result of this leak is not simply
>> orphaned threads, but thousands of classes not gc'd because the classloader
>> itself can't be gc'd. This is what is filling up PermGen.
>>
>> Has anyone discovered this particular issue? Has anyone discovered a
>> solution?
>>
>> Thanks,
>> Mike G.
>>
>> NetCentric Technology, Inc.
>> 3349 Route 138, Building A
>> Wall, NJ  07719
>> Phone: 732-544-0888
>>
>> ****
>>
>>  ** **
>>
>> ** **
>>
>
>
+
Heath Abelson 2013-04-24, 16:58
+
Josh Elser 2013-04-24, 14:12
+
Keith Turner 2013-04-24, 14:28