Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # dev - Issue HADOOP-8905


+
syates@... 2013-05-07, 22:49
+
Harsh J 2013-05-09, 03:40
+
steve@... 2013-05-09, 04:20
+
Steve Loughran 2013-05-09, 16:58
+
syates@... 2013-05-10, 03:39
+
Steve Loughran 2013-05-10, 16:46
Copy link to this message
-
Re: Issue HADOOP-8905
Andrew Wang 2013-05-10, 18:49
HBase has some nice metrics stuff that's not part of hadoop-common,
including a nifty reservoir sampling histogram / stats class. Would
definitely be nice to pull in.

There's also the MutableQuantiles class in hadoop-common if you want to
calculate a more accurate histogram.

If you're looking for yet more metrics, Coda Hale's library comes
recommended:

http://metrics.codahale.com/

On Fri, May 10, 2013 at 9:46 AM, Steve Loughran <[EMAIL PROTECTED]>wrote:

> On 9 May 2013 20:39, <[EMAIL PROTECTED]> wrote:
>
> >
> >> Unless there are existing bits of this stuff lurking somewhere in the
> >> Hadoop codebase that I haven't noticed, these could be copied into
> hadoop
> >> core. Reviewing the code as it is would be welcome
> >>
> >> https://github.com/**steveloughran/hadoop-trunk/**
> >> tree/stevel/HADOOP-8545-swift/**hadoop-tools/hadoop-openstack/**
> >> src/main/java/org/apache/**hadoop/fs/swift/util<
> https://github.com/steveloughran/hadoop-trunk/tree/stevel/HADOOP-8545-swift/hadoop-tools/hadoop-openstack/src/main/java/org/apache/hadoop/fs/swift/util
> >
> >>
> >>
> > Thank you Steve, if you don't mind me taking a look at and possibly using
> > your stats classes with this issue that would be great.
> >
> > I will review and get back in touch
> >
>
> great!
>
> This is what a post-test-run dump of the stats looks like
> 2013-05-09 16:35:19,878 [main] INFO   HEAD getObjectMetadata-newest count=3
> total=0.732s mean=0.244s stddev=0.000s min=0.171s max=0.287s
> 2013-05-09 16:35:19,879 [main] INFO   DELETE  count=10 total=4.594s
> mean=0.459s stddev=0.001s min=0.188s max=2.526s
> 2013-05-09 16:35:19,879 [main] INFO   PUT  count=1 total=6.253s mean=6.253s
> stddev=0.000s min=6.253s max=6.253s
> 2013-05-09 16:35:19,880 [main] INFO   GET findObjects count=2 total=0.443s
> mean=0.222s stddev=0.000s min=0.220s max=0.223s
> 2013-05-09 16:35:19,880 [main] INFO   HEAD createContainer count=1
> total=0.324s mean=0.324s stddev=0.000s min=0.324s max=0.324s
> 2013-05-09 16:35:19,881 [main] INFO   HEAD getObjectMetadata-newest-FAIL
> count=2 total=10.755s mean=5.378s stddev=0.007s min=0.178s max=10.577s
> 2013-05-09 16:35:19,881 [main] INFO   PUT upload count=9 total=2.546s
> mean=0.283s stddev=0.000s min=0.216s max=0.377s
> 2013-05-09 16:35:19,881 [main] INFO   POST authentication count=1
> total=1.096s mean=1.096s stddev=0.000s min=1.096s max=1.096s
>