Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> short scans in one RPC


Copy link to this message
-
Re: short scans in one RPC
Interesting idea.
On Thu, Aug 22, 2013 at 9:53 AM, Dave Latham <[EMAIL PROTECTED]> wrote:

> I was recently dealing with a situation of the META table being slammed
> with queries for a long time.  While evaluating the problem I saw the IPCs
> being in a cycle like the following:
>
> HRegionInterface#getClosestRowBefore queueTime=200 processingTime=1 ... 444
> bytes.
> HRegionInterface#openScanner queueTime=191 processingTime=0 ... 22 bytes.
> HRegionInterface#next queueTime=154 processingTime=1 ...  4551 bytes.
> HRegionInterface#close queueTime=216 processingTime=0 ... 16 bytes.
> (rinse, repeat)
>
> I'm bumping our prefetching from 10 rows to 50 rows to help cut down on the
> number of cycles, but it also occurs to me that it's a bit painful to
> require 3 RPCs to do a short scan.
>
> What would people think about adding method for short scans which
> encapsulates (openScanner / next / close) in a single RPC?
>
> Dave
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB