Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - HBase: Paralel Query


Copy link to this message
-
Re: HBase: Paralel Query
Ted Yu 2013-11-27, 15:03
bq. out of maxHeapMB=15983

In previous email you said RAM is 8GB. Above figure is larger than 8GB.

There're 6 coprocessors installed on each table.
I wonder if what you observed was related to HBASE-10047.

Cheers
On Wed, Nov 27, 2013 at 12:22 AM, Job Thomas <[EMAIL PROTECTED]> wrote:

>
> Hi Ted,All
>
> I have set
>
> hfile.block.cache.size to 0.6
> hbase.regionserver.handler.count to 60
> DATA_BLOCK _ENCODING => 'FAST_DIFF'
>  BLOOMFILTER => 'ROW'
>  BLOCKSIZE => '8192'
>  BLOCKCACHE => 'true'
>
> The performance has been increased.
>
> But after creating another table with same size and configurations , the
> performance of previous table has been reduced and I am getting good
> performance for the new table created.
>
> I have seen that whle querying out of maxHeapMB=15983 Hbase using only
>  usedHeapMB=72.
> why hbase not utilizing heap space even though I have set BLOCKSIZE =>
> '8192' ( For to store more number of indexes in memory ).
>
> I have read that once block size of hfile has been reduce, the sequential
> access speed will decrease . but I didn't experienced this  even though my
>   BLOCKSIZE is 192'
>
> Best Regards,
> Job M Thomas
>
> ________________________________
>
> From: Ted Yu [mailto:[EMAIL PROTECTED]]
> Sent: Wed 11/27/2013 11:48 AM
> To: [EMAIL PROTECTED]
> Subject: Re: HBase: Paralel Query
>
>
>
> bq. I didn't enabled blockcache
>
> What if you enable blockcache ?
>
> Cheers
>
>
> On Tue, Nov 26, 2013 at 8:45 PM, Job Thomas <[EMAIL PROTECTED]> wrote:
>
> > Hello lars,
> >
> > Here re the answers ,
> >
> > -> I have only one region server. ( I am testing Hbase via phoenix with
> > Hbase in a single server).
> > -> All queries are fired through Phoenix only.( select Lastname from
> > tablename where Id=? ( Here Id is the primary key))
> > -> hbase.regionserver.handler.count=30(default value).
> > -> Hardware:   Core =8
> >                      Ram =8 Gb
> > -> I didn't enabled blockcache.
> > -> Are the client in multiple threads in the process or multiple
> > processes? - I am not clear
> >
> >
> > Best Regards,
> > Job M Thomas
> >
> > ________________________________
> >
> > From: lars hofhansl [mailto:[EMAIL PROTECTED]]
> > Sent: Tue 11/26/2013 11:16 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: HBase: Paralel Query
> >
> >
> >
> > Hi Job,
> >
> > first off some questions :)
> > How many regions are you accessing?
> > What type of query is this (get or scan)?
> > How many handlers have you configured?
> > What does you hardware look like (how many cores, etc)?
> > Is the data all in the blockcache?
> > If not, what does the disk IO look like?
> > Are the client in multiple threads in the process or multiple processes?
> >
> >
> > Sorry for all the questions, but we need a bit more data.
> >
> >
> > -- Lars
> >
> >
> >
> > ________________________________
> >  From: Job Thomas <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]
> > Sent: Tuesday, November 26, 2013 12:26 AM
> > Subject: HBase: Paralel Query
> >
> >
> >
> >
> > Hi All,
> >
> > How can we configure Hbase  inorder to perform multythreading/parallel
> > query faster .
> >
> > These are some bits from my analysis:
> >
> > Each Thread contain 10 query ( Random)
> >
> > Tread        H2(Msec)  Phoenix(Msec)
> >   1            34             215
> >   2            63             222
> >   4            120           324
> >   6            200          340
> >   8           250           460
> >   10         350          560
> >   12          410         592
> >
> > I have to find some points in the graph ploted with these values where
> > lines are intercepting .
> > So I need hbase to perform well with multythreaded condition .
> >
> >
> > Best Regards,
> > Job M Thomas
> >
> >
>
>
>