-Re: Hadoop testing project [Was: [VOTE] Abandon mrunit MapReduce contrib]
Konstantin Boudnik 2011-02-17, 20:03
On Thu, Feb 17, 2011 at 11:27, Aaron Kimball <[EMAIL PROTECTED]> wrote:
> Working to develop code as a client of Hadoop is a path full of landmines.
> The more tools we can provide to users to improve the quality of their code,
> the better. I think it is important, though, to draw a clear distinction
> between tools intended for different audiences. Talking about system testing
> tools for Hadoop release/QA processes is good, but one of the benefits I see
> of calling MRUnit (designed for client app developers) out of the Hadoop
> project at large is to increase its usability. Conflating it with a system
> testing tool (for release engineers) would not fulfill that need.
Yup, they are different all right.
> As long as the new project can release several distinct artifacts in a way
> that makes their intent clear to the user community, I'm in favor of
> gathering as many perspectives on Hadoop testing under one "roof" as
That's the goal.
> - Aaron
> On Wed, Feb 16, 2011 at 10:19 AM, Rottinghuis, Joep
> <[EMAIL PROTECTED]>wrote:
>> Having a coherent approach for system level testing increases confidence in
>> the various Hadoop releases and and will reduce the effort to take any (set
>> of ) changes from development into production.
>> The more automated and formalized system testing, the better!
>> From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] On Behalf Of Konstantin Boudnik [
>> [EMAIL PROTECTED]]
>> Sent: Tuesday, February 15, 2011 1:58 PM
>> To: [EMAIL PROTECTED]
>> Subject: Hadoop testing project [Was: [VOTE] Abandon mrunit MapReduce
>> While MrUnit discussion draws to its natural conclusion I would like
>> to bring up another point which might be well aligned with that
>> discussion. Patrick Hunt has brought up this idea earlier today and I
>> believe it has to be elaborated further.
>> A number of testing projects both for Hadoop and Hadoop-related
>> component were brought to life over last year or two. Among those are
>> MRUnit, PigUnit, YCSB, Herriot, and perhaps a few more. They all
>> focusing on more or less the same problem e.g. validation of Hadoop or
>> on-top-of-Hadoop components, or application level testing for Hadoop.
>> However, the fact that they all are spread across a wide variety of
>> projects seems to confuse/mislead Hadoop users.
>> How about incubating a bigger Hadoop (Pig, Oozie, HBase) testing
>> project which will take care about development and support of common
>> (where's possible) tools, frameworks and the like? Please feel free to
>> share your thoughts :)
>> Take care,
>> Konstantin (Cos) Boudnik
>> On Tue, Feb 15, 2011 at 10:44, Eric Sammer <[EMAIL PROTECTED]> wrote:
>> > I've started the wiki page proposal for Incubator for mrunit. I'll ping
>> > people off list for mentoring. Much appreciated for all the help!
>> > On Tue, Feb 15, 2011 at 1:36 PM, Nigel Daley <[EMAIL PROTECTED]> wrote:
>> >> I'm happy to help mentor as well.
>> >> Cheers,
>> >> Nige
>> >> On Feb 11, 2011, at 11:52 AM, Patrick Hunt wrote:
>> >> > On Fri, Feb 11, 2011 at 9:44 AM, Mattmann, Chris A (388J)
>> >> > <[EMAIL PROTECTED]> wrote:
>> >> >> Guys, BTW, if you need help or a mentor in Apache Incubator-ville for
>> >> MRUnit, I would be happy to help.
>> >> >
>> >> > I was going to suggest the same thing (mrunit to incubator). I would
>> >> > also be happy to be a mentor.
>> >> >
>> >> > Patrick