Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - [VOTE] Abandon mrunit MapReduce contrib


Copy link to this message
-
Re: [VOTE] Abandon mrunit MapReduce contrib
Aaron Kimball 2011-02-22, 01:54
I have made some revisions and improvements here too..
- Aaron

On Sat, Feb 19, 2011 at 6:52 PM, Mattmann, Chris A (388J) <
[EMAIL PROTECTED]> wrote:

> Hi Guys,
>
> I took a pass at the MRUnit proposal [1]. It's far from complete, but just
> letting you know I took a swipe :)
>
> Cheers,
> Chris
>
> [1] http://wiki.apache.org/incubator/MRUnitProposal
>
> On Feb 17, 2011, at 1:52 PM, Mattmann, Chris A (388J) wrote:
>
> > Thanks Patrick.
> >
> > Sounds good. Besides mentors, we need a champion, who must be an ASF
> member. So, either me, you or Nigel can do it. I'm already championing Gora,
> and would be happy to champion MRUnit, but am open to either one of you guys
> doing it too (or even another ASF member). Just let me know. I'll head over
> to the wiki and add my info. Thanks for getting this started Eric+Patrick!
> >
> > Cheers,
> > Chris
> >
> > On Feb 17, 2011, at 1:00 PM, Patrick Hunt wrote:
> >
> >> Chris a page is up (still being created by Eric afaict):
> >> http://wiki.apache.org/incubator/MRUnitProposal
> >>
> >> I took the liberty of listing us as mentors.
> >>
> >> Patrick
> >>
> >> On Thu, Feb 17, 2011 at 11:31 AM, Mattmann, Chris A (388J)
> >> <[EMAIL PROTECTED]> wrote:
> >>> Hey Guys,
> >>>
> >>> FYI on this: Eric has mentioned he is going to start the Incubator
> proposal for MRUnit. Let's start small and then grow big (as needed). It
> seems like we've achieved enough consensus for the required mentors and
> critical mass to make an MRUnit Incubator proposal and then to have the
> Incubator community weigh in. If that expands to include other testing
> projects/etc., we can address that over the Incubation process, and as
> needed.
> >>>
> >>> Eric: as soon as that wiki page is up, I'd be happy to add my name to
> it as a mentor and /kick the can on this.
> >>>
> >>> Cheers,
> >>> Chris
> >>>
> >>> On Feb 17, 2011, at 11:11 AM, Aaron Kimball wrote:
> >>>
> >>>> The MRUnit community is a specific subset of the Hadoop community:
> Engineers
> >>>> writing Java code running on Hadoop. The Hadoop community also
> includes
> >>>> IT/ops staff who maintain Hadoop clusters, data scientists who use
> tools
> >>>> such as Pig & Hive, as well as those written by the aforementioned
> >>>> engineers, etc.
> >>>>
> >>>> The Hadoop project has long recognized that tools aimed at a specific
> subset
> >>>> of the Hadoop community, with separate release cycles, can more
> successfully
> >>>> reach their aims by splitting into incubator projects. Hive, Pig, and
> HBase,
> >>>> for example, have all gone this path.
> >>>>
> >>>> A "current" version of MRUnit would need to compile against multiple
> >>>> versions of Hadoop itself. This is not possible if it is in the same
> source
> >>>> tree as Hadoop.
> >>>>
> >>>> - Aaron
> >>>>
> >>>> On Thu, Feb 17, 2011 at 5:31 AM, Bernd Fondermann <
> >>>> [EMAIL PROTECTED]> wrote:
> >>>>
> >>>>> On Fri, Feb 11, 2011 at 23:10, Aaron Kimball <[EMAIL PROTECTED]>
> wrote:
> >>>>>> The main reason I am interested in removing MRUnit from Hadoop is
> that I
> >>>>>> believe that MRUnit deserves its own release cycle. I think this is
> in
> >>>>> the
> >>>>>> best interest of its users.
> >>>>>
> >>>>> Not in mine, at least. (I'm writing MR unit tests.)
> >>>>> Many projects release more than one product. I'd rather get MRUnit
> >>>>> from the same source where I get my MR from.
> >>>>> Separate release cylcles would be ok for me, though.
> >>>>>
> >>>>>> Perhaps more importantly, access to new features in MRUnit should
> not
> >>>>>> require upgrading one's entire Hadoop deployment; this is a client
> >>>>> library
> >>>>>> that depends only on Hadoop's public APIs.
> >>>>>
> >>>>> +1.
> >>>>>
> >>>>>> My primary concern is to move MRUnit to a place where the community
> can
> >>>>>> derive the most benefit from it. The Apache Incubator could fulfill
> this
> >>>>>> role; given the presence of individuals willing to mentor this
> project, I
> >>>>>> believe this would be a successful way to release MRUnit more