Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> high-percentile latency metrics, aka HBASE-6261


Copy link to this message
-
high-percentile latency metrics, aka HBASE-6261
Hi all,

I'm looking into some better ways of estimating high-percentile latency,
since I don't think the existing sampling-based method does a good job. I
wrote up a document and put it on HBASE-6261 that outlines what I think are
the available options; I'd encourage any resident stats experts / people
interested in metrics to take a look.

I wanted to ask off JIRA though about what would be useful in practice. I
think it'd be nice to see, for example, accurate 90th and 99th percentile
latency over recent 10s, 1m, 5m, and 15m time windows. I found some nice
algos to do this, I think at the cost of MBs of memory.

If this sounds like overkill though, there are even cheaper algos that
provide a more qualitative feeling of how the latency distribution is
changing over time. You give up one or more of bounded error, exact
percentiles, or time-based windows, but you do get the general feeling of
up vs. down.

So, is the "full" solution compelling enough to proceed? Anything
missing/extraneous?

Thanks,
Andrew
+
Stack 2012-06-28, 22:31
+
Andrew Wang 2012-06-28, 22:53
+
Stack 2012-06-29, 17:07
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB