Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase Region Server Spinning on JMX requests


Copy link to this message
-
Re: HBase Region Server Spinning on JMX requests
Hmm... That's not good. Would you mind filing a ticket here: https://issues.apache.org/jira/browse/HBASE ?

-- Lars
________________________________
 From: "Buckley,Ron" <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Tuesday, April 23, 2013 6:57 AM
Subject: HBase Region Server Spinning on JMX requests
 

This is with HBase 0.94.4 & CDH 4.1.1

This morning one our region servers (we have 44) stopped responding to
the '/jmx' request. (It's working for regular activity.)  Additionally,
the region server is now using all the CPU on the host, running all 8
cores at 100%.

I've got several jstacks, they all look like this:
http://pastebin.com/dGTmTEN7

If I do a wget of the /jmx url, it starts responding, but never
completes, always stopping at the same point:
http://pastebin.com/qhNvxrQK

Has anyone ever seen this before? If so, Is there a way out of it?
(other than bouncing the region server).

BTW: There's nothing relevant in the region server log and the garbage
collector log is normal.
----------------------------------------------------------------------
Ron Buckley
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB