Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - New htable slow


Copy link to this message
-
Re: New htable slow
Mohammad Tariq 2013-01-28, 12:37
It does. These excerpts from Hbase Definitive Guide might help you in a
better fashion :

"HTablePool:
Instead of creating an HTable instance for every request from your client
application, it
makes much more sense to create one initially and subsequently reuse them.
The primary reason for doing so is that creating an HTable instance is a
fairly expensive
operation that takes a few seconds to complete. In a highly contended
environment
with thousands of requests per second, you would not be able to use this
approach at
all—creating the HTable instance would be too slow. You need to create the
instance
at startup and use it for the duration of your client’s life cycle.
There is an additional issue with the HTable being reused by multiple
threads within
the same process.

The HTable class is not thread-safe, that is, the local write buffer is not
guarded against concurrent modifications. Even if you were to use
setAutoFlush(true) this is not advisable. Instead, you should use
one instance of HTable for each thread you are running in your client
application.

Clients can solve this problem using the HTablePool class. It only serves
one purpose,
namely to pool client API instances to the HBase cluster."

HTH

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Mon, Jan 28, 2013 at 5:54 PM, Lsshiu <[EMAIL PROTECTED]> wrote:

> Hi Tariq,
>     Thanks for the update, I'll try it. Btw , will put operation makes any
> performance difference using htable or htablepool ?
>
>
> > Use "HTbalePool" instead and see if it gives you better performance.
> > Creating an HTable instance is a fairly expensive operation that takes a
> > few seconds to complete.
> >
> > Warm Regards,
> > Tariq
> > https://mtariq.jux.com/
> > cloudfront.blogspot.com
> >
> >
> > On Mon, Jan 28, 2013 at 5:35 PM, Lsshiu <[EMAIL PROTECTED]> wrote:
> >
> >>
> >>
> >> Hi Tariq,
> >>    Thanks for the tip, but I only opened a few htable instances. The
> >> total region numbers were quite many (more than 70000 ) though.
> >>
> >>> Hi there,
> >>>
> >>>    Do you have too many Htable instances opened simultaneously?It's not
> >>> advisable to do so.
> >>>
> >>> Warm Regards,
> >>> Tariq
> >>> https://mtariq.jux.com/
> >>> cloudfront.blogspot.com
> >>>
> >>>
> >>> On Mon, Jan 28, 2013 at 5:00 PM, Lsshiu <[EMAIL PROTECTED]> wrote:
> >>>
> >>>> Hi
> >>>> In my 0.90.6 hbase environment , each time when I try "new htable, "
> it
> >>>> was slow. "Put " also slow too. Can I turn on some kind of trace to
> know
> >>>> the exact time spent in each detail function call? Thanks.
> >>
>