Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - Issue HADOOP-8905


Copy link to this message
-
Re: Issue HADOOP-8905
Steve Loughran 2013-05-09, 16:58
On 8 May 2013 21:20, <[EMAIL PROTECTED]> wrote:

> Hi Harsh,
>
> Thanks for responding,
>
> I would be interested in what the dev group had in mind for this and I
> also have a couple of additional queries ;
>
> I can see that a quick win for this would be to expose the existing Jetty
> statistics metrics within the jetty-management api/
>
> Existing metrics which can be exposed include without much effort include;
>
> statsOnMs: Time in milliseconds stats have been collected for.
> statsReset(): Reset statistics.
> requests: Number of requests since statsReset() called.
> requestsActive: Number of requests currently active.
> requestsActiveMin: Minimum number of active requests since statsReset()
> called.
> requestsActiveMax: Maximum number of active requests since statsReset()
> called.
> requestsDurationAve: Average duration of request handling in milliseconds
> since statsReset() called.
> requestsDurationMin: Get minimum duration in milliseconds of request
> handling since statsReset() called.
> requestsDurationMax: Get maximum duration in milliseconds of request
> handling since statsReset() called.
> requestsDurationTotal: Get total duration in milliseconds of all request
> handling since statsReset() called.
>
>
I have some code on a not-yet-merged fork to create duruations, build
rolling average and variance, that variance being a key one if you are
trying to serve end users; ave can hide problems. There's Duration,
DurationStats to aggregate stats, and a DurationTable to map to duration
stats by name, e.g. "POST", and whether the op passed or failed

Unless there are existing bits of this stuff lurking somewhere in the
Hadoop codebase that I haven't noticed, these could be copied into hadoop
core. Reviewing the code as it is would be welcome

https://github.com/steveloughran/hadoop-trunk/tree/stevel/HADOOP-8545-swift/hadoop-tools/hadoop-openstack/src/main/java/org/apache/hadoop/fs/swift/util