Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> max regionserver handler count

Viral Bajaria 2013-04-28, 20:30
Ted Yu 2013-04-28, 22:19
Viral Bajaria 2013-04-29, 01:39
ramkrishna vasudevan 2013-04-29, 02:37
Viral Bajaria 2013-04-29, 09:17
Ted Yu 2013-04-29, 09:25
Viral Bajaria 2013-04-29, 09:36
Ted Yu 2013-04-29, 09:49
Viral Bajaria 2013-04-29, 20:23
Viral Bajaria 2013-04-30, 03:48
Ted Yu 2013-04-30, 04:43
Copy link to this message
Re: max regionserver handler count
Thanks for getting back, Ted. I totally understand other priorities and
will wait for some feedback. I am adding some more info to this post to
allow better diagnosing of performance.

I hit my region servers with a lot of GET requests (~20K per second per
regionserver) using asynchbase in my test environment, the storage pattern
is very similar to OpenTSDB though with a lot more columns. Each row is
around 45-50 bytes long. The regionservers have a lot of RAM available to
them (48 out of 60 GB) and they are not sharing resources with anyone else,
so memory is not under pressure. The total # of rows in the table is around
100M and growing (there is a put process too)

GETs take over 15s for 16K rows, and I don't see any operationTooSlow logs
in the regionserver logs either. PUTs take around 1s for 16K rows (deferred
log flush is enabled though).

I looked at the RPC stats and it seems the RPC threads were always doing
something and I assumed my requests were waiting on handlers and so thought
of experimenting by increasing number of handlers. But as mentioned in my
thread, going above 10K kills my regionserver.


On Mon, Apr 29, 2013 at 9:43 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Viral:
> I am currently dealing with some high priority bugs so I didn't have time
> to look deeper into your case.
> My feeling is that raising max regionserver handler count shouldn't be the
> key to boosting performance.
> Cheers
Anoop John 2013-04-30, 06:29
Viral Bajaria 2013-04-30, 06:42
Anoop John 2013-04-30, 07:03
Viral Bajaria 2013-04-30, 09:52