Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # user >> PermGen leak in JBoss


+
Michael Giordano 2013-04-24, 13:48
+
Heath Abelson 2013-04-24, 15:21
+
Keith Turner 2013-04-24, 16:22
+
Michael Giordano 2013-05-03, 15:43
+
Keith Turner 2013-05-03, 15:54
+
Keith Turner 2013-04-24, 15:44
+
Eric Newton 2013-04-24, 15:47
+
Heath Abelson 2013-04-24, 16:08
+
Keith Turner 2013-04-24, 16:34
Copy link to this message
-
Re: PermGen leak in JBoss
would it not be better to make the accumulo connection a JNDI resource
outside the webapp?
On Wed, Apr 24, 2013 at 12:34 PM, Keith Turner <[EMAIL PROTECTED]> wrote:

> If you execute the following code in the before the app is undeployed and
> in the correct classloader context, then it will cause the "Thrift
> Connection Pool Checker" thread to die w/ a NPE being printed.
>
>     Field cacheField = ThriftTransportPool.class.getDeclaredField("cache");
>     cacheField.setAccessible(true);
>     cacheField.set(ThriftTransportPool.getInstance(), null);
>
>
> On Wed, Apr 24, 2013 at 12:08 PM, Heath Abelson <
> [EMAIL PROTECTED]> wrote:
>
>>  As an additional point, we have a second webapp that also uses 1.3.7
>> that also leaves the same threads – minus the 2 two watchdog threads****
>>
>> ** **
>>
>> *From:* Eric Newton [mailto:[EMAIL PROTECTED]]
>> *Sent:* Wednesday, April 24, 2013 11:47 AM
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Re: PermGen leak in JBoss****
>>
>> ** **
>>
>> One is for the dynamic reloading of the log4j configuration, IIRC.****
>>
>> ** **
>>
>> -Eric****
>>
>> ** **
>>
>> ** **
>>
>> On Wed, Apr 24, 2013 at 11:44 AM, Keith Turner <[EMAIL PROTECTED]> wrote:*
>> ***
>>
>> ** **
>>
>> ** **
>>
>> On Wed, Apr 24, 2013 at 11:21 AM, Heath Abelson <
>> [EMAIL PROTECTED]> wrote:****
>>
>> Hi, I am a colleague of Michael's, I can provide more info
>>
>> The following are the 5 threads that are not being cleaned up on an
>> undeploy. It's hard to get more stack info than this.
>>
>> Daemon Thread [FileWatchdog] (Running)
>> Daemon Thread [MSC service thread 1-5-SendThread(cbdbtestbox:2181)]
>> (Running)
>> Daemon Thread [MSC service thread 1-5-EventThread] (Running)
>> Daemon Thread [Thrift Connection Pool Checker] (Running)
>> Daemon Thread [FileWatchdog] (Running)
>>
>> Using Plumbr, the report states that the SendThread and EventThread above
>> both contain an instance of org.apache.zookeeper.ClientCnxn****
>>
>> ** **
>>
>> So two zookeeper threads. An Accumulo thread (Thrift Connection Pool
>> Checker).  What are the FileWatchdog threads?  Those could be related to
>> accumulo-start or hdfs.****
>>
>>  ****
>>
>>
>> -Heath****
>>
>>
>> -----Original Message-----
>> From: Michael Giordano
>> Sent: Wednesday, April 24, 2013 9:48 AM
>> To: [EMAIL PROTECTED]
>> Subject: PermGen leak in JBoss
>>
>> Under version 1.3.7 we are using the following code to initialize a
>> cloudbase connection during initialization of our web app:
>>
>>                         ZooKeeperInstance instance = new
>> ZooKeeperInstance(instanceName, zooKeepers);
>>                         connector = instance.getConnector(userId,
>> password.getBytes());
>>
>> The problem is that under the hood, this call creates several (5) threads
>> that are not cleaned up when the app is undeployed in JBoss. This is
>> occurring without performing any scans or interacting with cloudbase in any
>> other way. After relatively few redeploys of the app, the PermGen Space is
>> OOM.
>>
>> I can't find any reference in the cloudbase API akin to a close() method
>> for the Connector object. This is a classloader leak effecting any webapp
>> that is accessing cloudbase directly. The result of this leak is not simply
>> orphaned threads, but thousands of classes not gc'd because the classloader
>> itself can't be gc'd. This is what is filling up PermGen.
>>
>> Has anyone discovered this particular issue? Has anyone discovered a
>> solution?
>>
>> Thanks,
>> Mike G.
>>
>> NetCentric Technology, Inc.
>> 3349 Route 138, Building A
>> Wall, NJ  07719
>> Phone: 732-544-0888
>>
>> ****
>>
>>  ** **
>>
>> ** **
>>
>
>
+
Heath Abelson 2013-04-24, 16:58
+
Josh Elser 2013-04-24, 14:12
+
Keith Turner 2013-04-24, 14:28
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB