Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - short scans in one RPC


Copy link to this message
-
short scans in one RPC
Dave Latham 2013-08-22, 16:53
I was recently dealing with a situation of the META table being slammed
with queries for a long time.  While evaluating the problem I saw the IPCs
being in a cycle like the following:

HRegionInterface#getClosestRowBefore queueTime=200 processingTime=1 ... 444
bytes.
HRegionInterface#openScanner queueTime=191 processingTime=0 ... 22 bytes.
HRegionInterface#next queueTime=154 processingTime=1 ...  4551 bytes.
HRegionInterface#close queueTime=216 processingTime=0 ... 16 bytes.
(rinse, repeat)

I'm bumping our prefetching from 10 rows to 50 rows to help cut down on the
number of cycles, but it also occurs to me that it's a bit painful to
require 3 RPCs to do a short scan.

What would people think about adding method for short scans which
encapsulates (openScanner / next / close) in a single RPC?

Dave