Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Regarding QoS for Handlers


+
Varun Sharma 2013-10-13, 20:36
+
Varun Sharma 2013-10-13, 20:54
+
Devaraj Das 2013-10-14, 05:17
Copy link to this message
-
Re: Regarding QoS for Handlers
On Sun, Oct 13, 2013 at 1:36 PM, Varun Sharma <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am wondering if QoS on handler threads is being looked at by someone ?

What Devaraj said plus, trunk has HBASE-8884 "Pluggable RpcScheduler"
committed w/ two implementations currently -- our current coarse QoS that
works off method annotations and target region name and a new FIFO schedule
for the master RPCs.  The authors, like yourselves, were motivated by
desire for a treatment that was other than our default prioritization
wanting to demark on whether the incoming RPC was a read or a write
(Beware, backport would be messy -- there was a few follow-on patches after
the initial commit).

...

> The issue with this could be that what happens when the user ends up
> directing most of their load to the high priority pool which has fewer
> threads. We could do something simple like having a tight upper bound on
> the call queue length and if a new high priority call is rejected from this
> pool, just enqueue it to the regular pool of requests.
>
> Thoughts ?
>
All sounds good to me Varun.

+ At a minimum we need a means of second-classing mapreduce workload.
+ Naive is fine.  Beyond this, the problem gets complicated fast (LarsG and
AndrewWang talk at HBaseCon)
+ We just added 'priority' to the RPC header so client can say what
priority they want (Server can do what it likes with it).

St.Ack
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB