This really looks like the issue we got that prompted applying HBASE-5898.
Never heard about the rumor.
If you can isolate the region that exhibits the issue, put it on a
region server by itself and patch that region server to see if it
fixes the issue.
Also, can you confirm that you are running java 1.6 u16?
On Tue, Dec 11, 2012 at 12:14 PM, Shrijeet Paliwal
<[EMAIL PROTECTED]> wrote:
> Done https://gist.github.com/4261746
> On Tue, Dec 11, 2012 at 12:10 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> The attachment didn't go through.
>> Can you use pastebin ?
>> On Tue, Dec 11, 2012 at 12:05 PM, Shrijeet Paliwal
>> <[EMAIL PROTECTED]>wrote:
>> > Hello All,
>> > If our read of https://issues.apache.org/jira/browse/HBASE-5898 is
>> > correct it talks about three issues:
>> > #1 Contention on a lock causing bad performance
>> > #2 HDFS slowness causing IPC handlers blocked for large times.
>> > #3 A mysterious bug which causes what looks like a deadlock
>> > In out environment we are seeing #3 (ish) signals. Attached is the
>> > stack trace. One of our region server got blocked (almost all of IPC
>> > handlers) for 11 hours, so it can not be #1 OR #2.
>> > We have not back ported the patch yet so cant say if it fixes the issue
>> > not. Also it is difficult to reproduce. There is a rumor that a program
>> > reproduce this issue exist, does it?
>> > Eager to hear thoughts.
>> >  HBase version 0.92.2 , Hadoop version CDH3u0
>> > -Shrijeet