Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - Hadoop testing project [Was: [VOTE] Abandon mrunit MapReduce contrib]


Copy link to this message
-
Re: Hadoop testing project [Was: [VOTE] Abandon mrunit MapReduce contrib]
Aaron Kimball 2011-02-17, 19:27
Working to develop code as a client of Hadoop is a path full of landmines.
The more tools we can provide to users to improve the quality of their code,
the better. I think it is important, though, to draw a clear distinction
between tools intended for different audiences. Talking about system testing
tools for Hadoop release/QA processes is good, but one of the benefits I see
of calling MRUnit (designed for client app developers) out of the Hadoop
project at large is to increase its usability. Conflating it with a system
testing tool (for release engineers) would not fulfill that need.

As long as the new project can release several distinct artifacts in a way
that makes their intent clear to the user community, I'm in favor of
gathering as many perspectives on Hadoop testing under one "roof" as
possible.

- Aaron

On Wed, Feb 16, 2011 at 10:19 AM, Rottinghuis, Joep
<[EMAIL PROTECTED]>wrote:

> +1
> Having a coherent approach for system level testing increases confidence in
> the various Hadoop releases and and will reduce the effort to take any (set
> of ) changes from development into production.
> The more automated and formalized system testing, the better!
>
> Thanks,
>
> Joep
> ________________________________________
> From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] On Behalf Of Konstantin Boudnik [
> [EMAIL PROTECTED]]
> Sent: Tuesday, February 15, 2011 1:58 PM
> To: [EMAIL PROTECTED]
> Subject: Hadoop testing project [Was: [VOTE] Abandon mrunit MapReduce
> contrib]
>
> While MrUnit discussion draws to its natural conclusion I would like
> to bring up another point which might be well aligned with that
> discussion. Patrick Hunt has brought up this idea earlier today and I
> believe it has to be elaborated further.
>
> A number of testing projects both for Hadoop and Hadoop-related
> component were brought to life over last year or two. Among those are
> MRUnit, PigUnit, YCSB, Herriot, and perhaps a few more. They all
> focusing on more or less the same problem e.g. validation of Hadoop or
> on-top-of-Hadoop components, or application level testing for Hadoop.
> However, the fact that they all are spread across a wide variety of
> projects seems to confuse/mislead Hadoop users.
>
> How about incubating a bigger Hadoop (Pig, Oozie, HBase) testing
> project which will take care about development and support of common
> (where's possible) tools, frameworks and the like? Please feel free to
> share your thoughts :)
> --
>  Take care,
> Konstantin (Cos) Boudnik
>
>
> On Tue, Feb 15, 2011 at 10:44, Eric Sammer <[EMAIL PROTECTED]> wrote:
> > I've started the wiki page proposal for Incubator for mrunit. I'll ping
> > people off list for mentoring. Much appreciated for all the help!
> >
> > On Tue, Feb 15, 2011 at 1:36 PM, Nigel Daley <[EMAIL PROTECTED]> wrote:
> >
> >> I'm happy to help mentor as well.
> >>
> >> Cheers,
> >> Nige
> >>
> >> On Feb 11, 2011, at 11:52 AM, Patrick Hunt wrote:
> >>
> >> > On Fri, Feb 11, 2011 at 9:44 AM, Mattmann, Chris A (388J)
> >> > <[EMAIL PROTECTED]> wrote:
> >> >> Guys, BTW, if you need help or a mentor in Apache Incubator-ville for
> >> MRUnit, I would be happy to help.
> >> >
> >> > I was going to suggest the same thing (mrunit to incubator). I would
> >> > also be happy to be a mentor.
> >> >
> >> > Patrick
>