Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Regarding QoS for Handlers

Varun Sharma 2013-10-13, 20:36
Varun Sharma 2013-10-13, 20:54
Devaraj Das 2013-10-14, 05:17
Copy link to this message
Re: Regarding QoS for Handlers
On Sun, Oct 13, 2013 at 1:36 PM, Varun Sharma <[EMAIL PROTECTED]> wrote:

> Hi,
> I am wondering if QoS on handler threads is being looked at by someone ?

What Devaraj said plus, trunk has HBASE-8884 "Pluggable RpcScheduler"
committed w/ two implementations currently -- our current coarse QoS that
works off method annotations and target region name and a new FIFO schedule
for the master RPCs.  The authors, like yourselves, were motivated by
desire for a treatment that was other than our default prioritization
wanting to demark on whether the incoming RPC was a read or a write
(Beware, backport would be messy -- there was a few follow-on patches after
the initial commit).


> The issue with this could be that what happens when the user ends up
> directing most of their load to the high priority pool which has fewer
> threads. We could do something simple like having a tight upper bound on
> the call queue length and if a new high priority call is rejected from this
> pool, just enqueue it to the regular pool of requests.
> Thoughts ?
All sounds good to me Varun.

+ At a minimum we need a means of second-classing mapreduce workload.
+ Naive is fine.  Beyond this, the problem gets complicated fast (LarsG and
AndrewWang talk at HBaseCon)
+ We just added 'priority' to the RPC header so client can say what
priority they want (Server can do what it likes with it).