Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Issue HADOOP-8905


+
syates@... 2013-05-07, 22:49
+
Harsh J 2013-05-09, 03:40
+
steve@... 2013-05-09, 04:20
+
Steve Loughran 2013-05-09, 16:58
+
syates@... 2013-05-10, 03:39
+
Steve Loughran 2013-05-10, 16:46
Copy link to this message
-
Re: Issue HADOOP-8905
HBase has some nice metrics stuff that's not part of hadoop-common,
including a nifty reservoir sampling histogram / stats class. Would
definitely be nice to pull in.

There's also the MutableQuantiles class in hadoop-common if you want to
calculate a more accurate histogram.

If you're looking for yet more metrics, Coda Hale's library comes
recommended:

http://metrics.codahale.com/

On Fri, May 10, 2013 at 9:46 AM, Steve Loughran <[EMAIL PROTECTED]>wrote:

> On 9 May 2013 20:39, <[EMAIL PROTECTED]> wrote:
>
> >
> >> Unless there are existing bits of this stuff lurking somewhere in the
> >> Hadoop codebase that I haven't noticed, these could be copied into
> hadoop
> >> core. Reviewing the code as it is would be welcome
> >>
> >> https://github.com/**steveloughran/hadoop-trunk/**
> >> tree/stevel/HADOOP-8545-swift/**hadoop-tools/hadoop-openstack/**
> >> src/main/java/org/apache/**hadoop/fs/swift/util<
> https://github.com/steveloughran/hadoop-trunk/tree/stevel/HADOOP-8545-swift/hadoop-tools/hadoop-openstack/src/main/java/org/apache/hadoop/fs/swift/util
> >
> >>
> >>
> > Thank you Steve, if you don't mind me taking a look at and possibly using
> > your stats classes with this issue that would be great.
> >
> > I will review and get back in touch
> >
>
> great!
>
> This is what a post-test-run dump of the stats looks like
> 2013-05-09 16:35:19,878 [main] INFO   HEAD getObjectMetadata-newest count=3
> total=0.732s mean=0.244s stddev=0.000s min=0.171s max=0.287s
> 2013-05-09 16:35:19,879 [main] INFO   DELETE  count=10 total=4.594s
> mean=0.459s stddev=0.001s min=0.188s max=2.526s
> 2013-05-09 16:35:19,879 [main] INFO   PUT  count=1 total=6.253s mean=6.253s
> stddev=0.000s min=6.253s max=6.253s
> 2013-05-09 16:35:19,880 [main] INFO   GET findObjects count=2 total=0.443s
> mean=0.222s stddev=0.000s min=0.220s max=0.223s
> 2013-05-09 16:35:19,880 [main] INFO   HEAD createContainer count=1
> total=0.324s mean=0.324s stddev=0.000s min=0.324s max=0.324s
> 2013-05-09 16:35:19,881 [main] INFO   HEAD getObjectMetadata-newest-FAIL
> count=2 total=10.755s mean=5.378s stddev=0.007s min=0.178s max=10.577s
> 2013-05-09 16:35:19,881 [main] INFO   PUT upload count=9 total=2.546s
> mean=0.283s stddev=0.000s min=0.216s max=0.377s
> 2013-05-09 16:35:19,881 [main] INFO   POST authentication count=1
> total=1.096s mean=1.096s stddev=0.000s min=1.096s max=1.096s
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB