Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> identify the cause of a large quantity of zookeeper threads


Copy link to this message
-
identify the cause of a large quantity of zookeeper threads
I am trying to debug a problem in which when I run jstack, I see a very
large number of zookeeper threads, and as the program continues running,
more threads like the following are created. I suspect somewhere in the
program it is not releasing some resource properly, but I am not sure where
it is.

What would be the cause of these threads waiting? Are they created by
launching new instances of zookeeper, or created by read/write operations?

Thanks.
"main-EventThread" daemon prio=5 tid=7fbf2f71d800 nid=0x11c4e4000 waiting
on condition [11c4e3000]
   java.lang.Thread.State: WAITING (parking)
 at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <77e5838b8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
 at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:493)

"main-SendThread(localhost:21819)" daemon prio=5 tid=7fbf2f651800
nid=0x11c3e1000 runnable [11c3e0000]
   java.lang.Thread.State: RUNNABLE
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
 at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69)
 at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <77e582700> (a sun.nio.ch.Util$2)
 - locked <77e5826f0> (a java.util.Collections$UnmodifiableSet)
- locked <77e5824e8> (a sun.nio.ch.KQueueSelectorImpl)
 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:274)
 at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)

"main-EventThread" daemon prio=5 tid=7fbf2f64f800 nid=0x11c0a7000 waiting
on condition [11c0a6000]
   java.lang.Thread.State: WAITING (parking)
 at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <77e39be90> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
 at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:493)

"main-SendThread(localhost:21819)" daemon prio=5 tid=7fbf2f0b6000
nid=0x11b7d2000 runnable [11b7d1000]
   java.lang.Thread.State: RUNNABLE
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
 at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69)
 at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <77e39acd8> (a sun.nio.ch.Util$2)
 - locked <77e39acc8> (a java.util.Collections$UnmodifiableSet)
- locked <77e39aac0> (a sun.nio.ch.KQueueSelectorImpl)
 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:274)
 at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)

"main-EventThread" daemon prio=5 tid=7fbf2f0b5000 nid=0x117c9e000 waiting
on condition [117c9d000]
   java.lang.Thread.State: WAITING (parking)
 at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <77dfd9540> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
 at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:493)

"main-SendThread(localhost:21819)" daemon prio=5 tid=7fbf2f6b4000
nid=0x1174b6000 runnable [1174b5000]
   java.lang.Thread.State: RUNNABLE
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
 at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69)
 at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <77dfd83c8> (a sun.nio.ch.Util$2)
 - locked <77dfd83b8> (a java.util.Collections$UnmodifiableSet)
- locked <77dfd81b0> (a sun.nio.ch.KQueueSelectorImpl)
 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:274)
 at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB