Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - short scans in one RPC


+
Dave Latham 2013-08-22, 16:53
+
Ted Yu 2013-08-23, 01:02
Copy link to this message
-
Re: short scans in one RPC
Stack 2013-08-23, 03:24
On Thu, Aug 22, 2013 at 9:53 AM, Dave Latham <[EMAIL PROTECTED]> wrote:

> I was recently dealing with a situation of the META table being slammed
> with queries for a long time.  While evaluating the problem I saw the IPCs
> being in a cycle like the following:
>
> HRegionInterface#getClosestRowBefore queueTime=200 processingTime=1 ... 444
> bytes.
> HRegionInterface#openScanner queueTime=191 processingTime=0 ... 22 bytes.
> HRegionInterface#next queueTime=154 processingTime=1 ...  4551 bytes.
> HRegionInterface#close queueTime=216 processingTime=0 ... 16 bytes.
> (rinse, repeat)
>
> I'm bumping our prefetching from 10 rows to 50 rows to help cut down on the
> number of cycles, but it also occurs to me that it's a bit painful to
> require 3 RPCs to do a short scan.
>
> What would people think about adding method for short scans which
> encapsulates (openScanner / next / close) in a single RPC?
>

Does trunk not do this?

It at least has all the mechanisms to achieve this where open scanner
returns results and you can set close on the scanner so you do not have to
go back again to do the close explicitly.

A Get is a scan; it for sure does this.

St.Ack
+
Dave Latham 2013-08-27, 17:23