Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> what's the typical scan latency?


Copy link to this message
-
RE: what's the typical scan latency?
Thanks Amit

In my envionment, I run a dozens of client to read about 5-20K data per scan concurrently, And the average read latency for cached data is around 5-20ms.
So it seems there must be something wrong with my cluster env or application. Or did you run that with multiple client?
>Depends on so much environment related variables and on data as well.
>But to give you a number after all:
>One of our clusters is on EC2, 6 RS, on m1.xlarge machines (network performance 'high' according to aws), with 90% of the time we do reads; our avg data size is 2K, block cache at 20K, 100 rows per scan avg, bloom filters 'on' at the 'ROW' level, 40% of heap dedicated to block cache (note that it contains several other bits and pieces) and I would say our average latency for cached data (~97% blockCacheHitCachingRatio) is 3-4ms. File system access is much much painful, especially on ec2 m1.xlarge where you really can't tell what's going on, as far as I can tell. To tell you the truth as I see it, this is an abuse (for our use case) of the HBase store and for cache like behavior I would recommend going to something like Redis.
On Mon, Jun 3, 2013 at 12:13 PM, ramkrishna vasudevan < [EMAIL PROTECTED]> wrote:

> What is that you are observing now?
>
> Regards
> Ram
>
>
> On Mon, Jun 3, 2013 at 2:00 PM, Liu, Raymond <[EMAIL PROTECTED]>
> wrote:
>
> > Hi
> >
> >         If all the data is already in RS blockcache.
> >         Then what's the typical scan latency for scan a few rows
> > from a say several GB table ( with dozens of regions ) on a small
> > cluster with
> say
> > 4 RS ?
> >
> >         A few ms? Tens of ms? Or more?
> >
> > Best Regards,
> > Raymond Liu
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB