Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase: Paralel Query


Copy link to this message
-
Re: HBase: Paralel Query
bq. out of maxHeapMB=15983

In previous email you said RAM is 8GB. Above figure is larger than 8GB.

There're 6 coprocessors installed on each table.
I wonder if what you observed was related to HBASE-10047.

Cheers
On Wed, Nov 27, 2013 at 12:22 AM, Job Thomas <[EMAIL PROTECTED]> wrote:

>
> Hi Ted,All
>
> I have set
>
> hfile.block.cache.size to 0.6
> hbase.regionserver.handler.count to 60
> DATA_BLOCK _ENCODING => 'FAST_DIFF'
>  BLOOMFILTER => 'ROW'
>  BLOCKSIZE => '8192'
>  BLOCKCACHE => 'true'
>
> The performance has been increased.
>
> But after creating another table with same size and configurations , the
> performance of previous table has been reduced and I am getting good
> performance for the new table created.
>
> I have seen that whle querying out of maxHeapMB=15983 Hbase using only
>  usedHeapMB=72.
> why hbase not utilizing heap space even though I have set BLOCKSIZE =>
> '8192' ( For to store more number of indexes in memory ).
>
> I have read that once block size of hfile has been reduce, the sequential
> access speed will decrease . but I didn't experienced this  even though my
>   BLOCKSIZE is 192'
>
> Best Regards,
> Job M Thomas
>
> ________________________________
>
> From: Ted Yu [mailto:[EMAIL PROTECTED]]
> Sent: Wed 11/27/2013 11:48 AM
> To: [EMAIL PROTECTED]
> Subject: Re: HBase: Paralel Query
>
>
>
> bq. I didn't enabled blockcache
>
> What if you enable blockcache ?
>
> Cheers
>
>
> On Tue, Nov 26, 2013 at 8:45 PM, Job Thomas <[EMAIL PROTECTED]> wrote:
>
> > Hello lars,
> >
> > Here re the answers ,
> >
> > -> I have only one region server. ( I am testing Hbase via phoenix with
> > Hbase in a single server).
> > -> All queries are fired through Phoenix only.( select Lastname from
> > tablename where Id=? ( Here Id is the primary key))
> > -> hbase.regionserver.handler.count=30(default value).
> > -> Hardware:   Core =8
> >                      Ram =8 Gb
> > -> I didn't enabled blockcache.
> > -> Are the client in multiple threads in the process or multiple
> > processes? - I am not clear
> >
> >
> > Best Regards,
> > Job M Thomas
> >
> > ________________________________
> >
> > From: lars hofhansl [mailto:[EMAIL PROTECTED]]
> > Sent: Tue 11/26/2013 11:16 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: HBase: Paralel Query
> >
> >
> >
> > Hi Job,
> >
> > first off some questions :)
> > How many regions are you accessing?
> > What type of query is this (get or scan)?
> > How many handlers have you configured?
> > What does you hardware look like (how many cores, etc)?
> > Is the data all in the blockcache?
> > If not, what does the disk IO look like?
> > Are the client in multiple threads in the process or multiple processes?
> >
> >
> > Sorry for all the questions, but we need a bit more data.
> >
> >
> > -- Lars
> >
> >
> >
> > ________________________________
> >  From: Job Thomas <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]
> > Sent: Tuesday, November 26, 2013 12:26 AM
> > Subject: HBase: Paralel Query
> >
> >
> >
> >
> > Hi All,
> >
> > How can we configure Hbase  inorder to perform multythreading/parallel
> > query faster .
> >
> > These are some bits from my analysis:
> >
> > Each Thread contain 10 query ( Random)
> >
> > Tread        H2(Msec)  Phoenix(Msec)
> >   1            34             215
> >   2            63             222
> >   4            120           324
> >   6            200          340
> >   8           250           460
> >   10         350          560
> >   12          410         592
> >
> > I have to find some points in the graph ploted with these values where
> > lines are intercepting .
> > So I need hbase to perform well with multythreaded condition .
> >
> >
> > Best Regards,
> > Job M Thomas
> >
> >
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB