Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Region server exception


Copy link to this message
-
Re: Region server exception
Ted,
When there are more than 32 concurrent clients (in a 4 nodes x 8 core cluster). I keep getting responseTooSlow for my coprocessors.
Our app is built mainly using coprocessor and a few multi-get.

(responseTooSlow): {"processingtimems":10682,"call":"execCoprocessor([B@511c627c, getFoo({T_5208=0.004815409309791332,

.....(multiple values of T_<id>=<double value>)
20), rpc version=1, client version=0, methodsFingerPrint=0), rpc version=1, client version=29, methodsFingerPrint
=-1368823753","client":"10.149.5.56:38292","starttimems":1377808493508,"queuetimems":7,"class":"HRegionServer","responsesize":0,"method":"execCoprocessor"}

We do a orderby on the T_<number> and do calculations on the double. This finishes in 400 msec (total T_ values processed is around 600K)  when there is only one client. But takes 8000 or 10000 when the # of concurrent connections are increased to 32 or above.
 
Regards,
- kiru
________________________________
 From: Ted Yu <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; Kiru Pakkirisamy <[EMAIL PROTECTED]>
Sent: Thursday, August 29, 2013 1:17 PM
Subject: Re: Region server exception
 

This exception means some other thread was holding the lock for extended
period of time.

Can you tell us more about your coprocessor ?

Thanks
On Thu, Aug 29, 2013 at 12:55 PM, Kiru Pakkirisamy <
[EMAIL PROTECTED]> wrote:

>
>
> This exception stack happens from within my coprocessor code on concurrent
> reads. Any ideas ?
>
> java.io.InterruptedIOException
> at org.apache.hadoop.hbase.regionserver.HRegion.lock(HRegion.java:5894)
> at org.apache.hadoop.hbase.regionserver.HRegion.lock(HRegion.java:5875)
> at org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRe
> gion.java:5803)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(H
> Region.java:3852)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(H
> Region.java:3896)
>
>
>
>
> Regards,
> - kiru
>
>
> Kiru Pakkirisamy | webcloudtech.wordpress.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB