Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> experiencing high latency for few reads in HBase


Copy link to this message
-
Re: experiencing high latency for few reads in HBase
Another point that could help to stay under the `1s SLA': enable direct
byte buffers for LruBlockCache. Have a look at HBASE-4027.
On Thu, Aug 29, 2013 at 9:27 PM, Kiru Pakkirisamy <[EMAIL PROTECTED]
> wrote:

> Yes, in that case, it matters. I was talking about a case where you are
> mostly serving from cache.
>
> Regards,
> - kiru
>
>
> Kiru Pakkirisamy | webcloudtech.wordpress.com
>
>
> ________________________________
>  From: Saurabh Yahoo <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Sent: Thursday, August 29, 2013 12:09 PM
> Subject: Re: experiencing high latency for few reads in HBase
>
>
> Thanks Kiru.
>
> We have 10TB of data on disk. It would not fit in memory. Also for the
> first time, hbase need to read from the disk. And it has to go through the
> network to read the blocks which are stored at other data node.
>
> So in my opinion, locality matters.
>
> Thanks,
> Saurabh.
>
> On Aug 29, 2013, at 2:33 PM, Kiru Pakkirisamy <[EMAIL PROTECTED]>
> wrote:
>
> > But locality index should not matter right if you are in IN_MEMORY most
> and you are running the test after  a few runs to make sure they are
> already in IN_MEMORY  (ie blockCacheHit is high or blockCacheMiss is low)
> (?)
> >
> > Regards,
> > - kiru
> >
> >
> > Kiru Pakkirisamy | webcloudtech.wordpress.com
> >
> >
> > ________________________________
> > From: Vladimir Rodionov <[EMAIL PROTECTED]>
> > To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> > Sent: Thursday, August 29, 2013 11:11 AM
> > Subject: RE: experiencing high latency for few reads in HBase
> >
> >
> > Usually, either cluster restart or major compaction helps improving
> locality index.
> > There is an issue in region assignment after table disable/enable in
> 0.94.x (x <11) which
> > breaks HDFS locality. Fixed in 0.94.11
> >
> > You can write your own routine to manually "localize" particular table
> using public HBase Client API.
> >
> > But this won't help you to stay withing 1 sec anyway.
> >
> > Best regards,
> > Vladimir Rodionov
> > Principal Platform Engineer
> > Carrier IQ, www.carrieriq.com
> > e-mail: [EMAIL PROTECTED]
> >
> > ________________________________________
> > From: Saurabh Yahoo [[EMAIL PROTECTED]]
> > Sent: Thursday, August 29, 2013 10:52 AM
> > To: [EMAIL PROTECTED]
> > Cc: [EMAIL PROTECTED]
> > Subject: Re: experiencing high latency for few reads in HBase
> >
> > Thanks Vlad.
> >
> > Quick question. I notice hdfsBlocksLocalityIndex is around 50 in all
> region servers.
> >
> > Does that could be a problem? If it is, how to solve that? We already
> ran the major compaction after ingesting the data.
> >
> > Thanks,
> > Saurabh.
> >
> > On Aug 29, 2013, at 12:17 PM, Vladimir Rodionov <[EMAIL PROTECTED]>
> wrote:
> >
> >> Yes. HBase won't guarantee strict sub-second latency.
> >>
> >> Best regards,
> >> Vladimir Rodionov
> >> Principal Platform Engineer
> >> Carrier IQ, www.carrieriq.com
> >> e-mail: [EMAIL PROTECTED]
> >>
> >> ________________________________________
> >> From: Saurabh Yahoo [[EMAIL PROTECTED]]
> >> Sent: Thursday, August 29, 2013 2:49 AM
> >> To: [EMAIL PROTECTED]
> >> Cc: [EMAIL PROTECTED]
> >> Subject: Re: experiencing high latency for few reads in HBase
> >>
> >> Hi Vlad,
> >>
> >> We do have strict latency requirement as it is financial data requiring
> direct access from clients.
> >>
> >> Are you saying that it is not possible to achieve sub second latency
> using hbase (because it is based on java.) ?
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Aug 28, 2013, at 8:10 PM, Vladimir Rodionov <[EMAIL PROTECTED]>
> wrote:
> >>
> >>> Increasing Java heap size will make latency worse, actually.
> >>> You can't guarantee 1 sec max latency if run Java app (unless your
> heap size is much less than 1GB).
> >>> I have never heard about strict maximum latency limit. Usually , its
> 99% , 99.9 or 99.99% query percentiles.

Adrien Mogenet
http://www.borntosegfault.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB