Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Sorting user defined MR counters.


Copy link to this message
-
Re: Sorting user defined MR counters.
On 29 December 2012 23:32, Niels Basjes <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I've had this 'itch' with Hadoop that it is hard to sort the counters in a
> "nice" way.
> Now the current trunk sorts the framework counters in such a way that they
> follow the flow quite nicely. For the generic counters (i.e. user code
> counters) this is not possible.
>
> I've been playing around these last few days to see if I can extend the API
> so I can create custom counters from my MR code and have the framework
> report them in the sorting order I defined that is useful  for my specific
> application.
>
> I currently have a working version of this idea here so I'm wondering ...
>
> Is this something you would like to have in the main source tree? From my
> point of view this is very generic and reusable by many projects.
>
> If you say 'yes' then I'll simply create a Jira for this and submit the
> patch.
>
> In addition I have a question about the content of such a patch.
> In some source files I'll be changing there are numerous very basic
> warnings from both the Java compiler, findbugs and checkstyle.
> With simple things I mean silly things "unused imports", "an interface that
> has directives like private and public", "first line of javadoc must end
> with '.'", "unused SuppressWarnings directives" .
>

yes, the Java codebase is a bit messy. It's the price of
review-then-commit; it adds overhead to do cleanups as you go along,
although doing a code cleanup is something that many people would
appreciate.
>
> I my normal job I commit a source file at least  "just as clean" as what I
> got to start with but preferably I make it "cleaner". That way a warning is
> something you look at instead of "part of the landslide" which people tend
> to ignore.
>
> Now for submitting changes for Hadoop: Is it desirable that I fix these in
> my change set or should I leave these as-is to avoid "obfuscating" the
> changes that are relevant to the Jira at hand?
>

I recommend a cleanup first -that's likely to go in without any argument.
Your patch with the new features would be a diff against the clean, so have
less changes to be reviewed.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB