Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> max regionserver handler count


Copy link to this message
-
Re: max regionserver handler count
You are making use of batch Gets?  get(List)

-Anoop-

On Tue, Apr 30, 2013 at 11:40 AM, Viral Bajaria <[EMAIL PROTECTED]>wrote:

> Thanks for getting back, Ted. I totally understand other priorities and
> will wait for some feedback. I am adding some more info to this post to
> allow better diagnosing of performance.
>
> I hit my region servers with a lot of GET requests (~20K per second per
> regionserver) using asynchbase in my test environment, the storage pattern
> is very similar to OpenTSDB though with a lot more columns. Each row is
> around 45-50 bytes long. The regionservers have a lot of RAM available to
> them (48 out of 60 GB) and they are not sharing resources with anyone else,
> so memory is not under pressure. The total # of rows in the table is around
> 100M and growing (there is a put process too)
>
> GETs take over 15s for 16K rows, and I don't see any operationTooSlow logs
> in the regionserver logs either. PUTs take around 1s for 16K rows (deferred
> log flush is enabled though).
>
> I looked at the RPC stats and it seems the RPC threads were always doing
> something and I assumed my requests were waiting on handlers and so thought
> of experimenting by increasing number of handlers. But as mentioned in my
> thread, going above 10K kills my regionserver.
>
> Thanks,
> Viral
>
> On Mon, Apr 29, 2013 at 9:43 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > Viral:
> > I am currently dealing with some high priority bugs so I didn't have time
> > to look deeper into your case.
> >
> > My feeling is that raising max regionserver handler count shouldn't be
> the
> > key to boosting performance.
> >
> > Cheers
> >
>