Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> short scans in one RPC


Copy link to this message
-
short scans in one RPC
I was recently dealing with a situation of the META table being slammed
with queries for a long time.  While evaluating the problem I saw the IPCs
being in a cycle like the following:

HRegionInterface#getClosestRowBefore queueTime=200 processingTime=1 ... 444
bytes.
HRegionInterface#openScanner queueTime=191 processingTime=0 ... 22 bytes.
HRegionInterface#next queueTime=154 processingTime=1 ...  4551 bytes.
HRegionInterface#close queueTime=216 processingTime=0 ... 16 bytes.
(rinse, repeat)

I'm bumping our prefetching from 10 rows to 50 rows to help cut down on the
number of cycles, but it also occurs to me that it's a bit painful to
require 3 RPCs to do a short scan.

What would people think about adding method for short scans which
encapsulates (openScanner / next / close) in a single RPC?

Dave
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB