Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Regarding QoS for Handlers


Copy link to this message
-
Re: Regarding QoS for Handlers
Varun Sharma 2013-10-13, 20:54
Or a slightly alternative approach would be to have a CallQueue
implementation which is more of a priorityQueue.

The Queue wrapper has a number of different queues [TableName:RpcName]
sorted in descending order by priority level and finally the default quue.
Dequeue on this queue shall go in descending order of priority.

Varun
On Sun, Oct 13, 2013 at 1:36 PM, Varun Sharma <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am wondering if QoS on handler threads is being looked at by someone ?
> We have found huge gains by doing this for asymmetric workloads which are
> extremely write heavy but latency on reads matters the most. Basically we
> forked out a separate thread pool of X threads for handling reads to
> prevent starvation of read requests which were far outnumbered by the # of
> writes.
>
> As of now, we have been making adhoc changes (like introduce higher
> priority levels for certain types of requests like gets and scans) but
> having a framework to control this would be a really nice thing.
>
> One simple way to achieve this would be to have a group of "high priority
> handler threads" and be able to mark each table to prioritize
> "reads/writes". On receiving an RPC, the request is directly dispatched to
> this other pool of threads and the remaining RPCs go to the regular pool.
> This would be a naive implementation.
>
> The issue with this could be that what happens when the user ends up
> directing most of their load to the high priority pool which has fewer
> threads. We could do something simple like having a tight upper bound on
> the call queue length and if a new high priority call is rejected from this
> pool, just enqueue it to the regular pool of requests.
>
> Thoughts ?
>
> Thanks !
> Varun
>