Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Sorting user defined MR counters.


Copy link to this message
-
Re: Sorting user defined MR counters.
On 29 December 2012 23:32, Niels Basjes <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I've had this 'itch' with Hadoop that it is hard to sort the counters in a
> "nice" way.
> Now the current trunk sorts the framework counters in such a way that they
> follow the flow quite nicely. For the generic counters (i.e. user code
> counters) this is not possible.
>
> I've been playing around these last few days to see if I can extend the API
> so I can create custom counters from my MR code and have the framework
> report them in the sorting order I defined that is useful  for my specific
> application.
>
> I currently have a working version of this idea here so I'm wondering ...
>
> Is this something you would like to have in the main source tree? From my
> point of view this is very generic and reusable by many projects.
>
> If you say 'yes' then I'll simply create a Jira for this and submit the
> patch.
>
> In addition I have a question about the content of such a patch.
> In some source files I'll be changing there are numerous very basic
> warnings from both the Java compiler, findbugs and checkstyle.
> With simple things I mean silly things "unused imports", "an interface that
> has directives like private and public", "first line of javadoc must end
> with '.'", "unused SuppressWarnings directives" .
>

yes, the Java codebase is a bit messy. It's the price of
review-then-commit; it adds overhead to do cleanups as you go along,
although doing a code cleanup is something that many people would
appreciate.
>
> I my normal job I commit a source file at least  "just as clean" as what I
> got to start with but preferably I make it "cleaner". That way a warning is
> something you look at instead of "part of the landslide" which people tend
> to ignore.
>
> Now for submitting changes for Hadoop: Is it desirable that I fix these in
> my change set or should I leave these as-is to avoid "obfuscating" the
> changes that are relevant to the Jira at hand?
>

I recommend a cleanup first -that's likely to go in without any argument.
Your patch with the new features would be a diff against the clean, so have
less changes to be reviewed.