I think the PMC should abandon the mrunit MapReduce contrib component.  The originator of mrunit and primary maintainer (Aaron Kimball) is moving the active development elsewhere.

There are 2 unresolved contrib/mrunit issues in Jira, none of them Patch Available.

Here is my +1.

Nige

+1. Eric Sammer and I will be working on this via github. (Come join us!)

- Aaron

On Thu, Feb 10, 2011 at 11:08 PM, Nigel Daley <[EMAIL PROTECTED]> wrote:

> I think the PMC should abandon the mrunit MapReduce contrib component.  The
> originator of mrunit and primary maintainer (Aaron Kimball) is moving the
> active development elsewhere.
>
> There are 2 unresolved contrib/mrunit issues in Jira, none of them Patch
> Available.
>
> Here is my +1.
>
> Nige
>

On Thu, Feb 10, 2011 at 11:36 PM, Aaron Kimball <[EMAIL PROTECTED]>wrote:

> +1. Eric Sammer and I will be working on this via github. (Come join us!)
Votes to remove code should be because the PMC doesn't think the code is
worth maintaining any more. I don't think that applies in this case. Aaron
is a committer in Hadoop now and I'd strongly encourage you both to give
your changes back to Apache instead of forking it into GitHub.

-- Owen

Hi All,

> Votes to remove code should be because the PMC doesn't think the code is
> worth maintaining any more. I don't think that applies in this case. Aaron
> is a committer in Hadoop now and I'd strongly encourage you both to give
> your changes back to Apache instead of forking it into GitHub.

+1.

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Owen:

I think you make a fair point. The reason I think it still makes sense to
bring mrunit out of Hadoop contrib is to:

- start to simplify the build by breaking projects that are only clients of
Hadoop libs out of contrib.
- allow mrunit to have its own release cycle. This is, I think, the most
important.

I would actually prefer to move it to Extras or Incubator and leave this
within the ASF. Right now, I picked github because of the ability to easily
collaborate with others (and to use git).

Thanks!

On Fri, Feb 11, 2011 at 2:52 AM, Owen O'Malley <[EMAIL PROTECTED]> wrote:

> On Thu, Feb 10, 2011 at 11:36 PM, Aaron Kimball <[EMAIL PROTECTED]
> >wrote:
>
> > +1. Eric Sammer and I will be working on this via github. (Come join us!)
>
>
> Votes to remove code should be because the PMC doesn't think the code is
> worth maintaining any more. I don't think that applies in this case. Aaron
> is a committer in Hadoop now and I'd strongly encourage you both to give
> your changes back to Apache instead of forking it into GitHub.
>
> -- Owen
>

--
Eric Sammer
twitter: esammer
data: www.cloudera.com

On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:

> - allow mrunit to have its own release cycle. This is, I think, the  
> most

> important.

If you submit your work to Apache we can evaluate it for inclusion in  
the 0.20.100 branch to get your changes released in a timely manner.

> I would actually prefer to move it to Extras or Incubator and leave  
> this
> within the ASF.

Extras is **NOT** inside of the ASF. Extras is a source hosting system  
for non-Apache projects that are related to Apache projects.

> Right now, I picked github because of the ability to easily
> collaborate with others (and to use git).

I agree that it is unfortunate that Apache doesn't yet support read-
write git access. However, you'll find that building a community is  
easier at Apache than at github.

-- Owen

On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley <[EMAIL PROTECTED]> wrote:

> On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:
>
>  - allow mrunit to have its own release cycle. This is, I think, the most
>>
>
>  important.
>>
>
> If you submit your work to Apache we can evaluate it for inclusion in the
> 0.20.100 branch to get your changes released in a timely manner.
I'm thinking in general (beyond the next immediate release). Independent of
where mrunit goes, I think it should leave the contrib tree to facilitate
light weight releases (the dependency on Hadoop proper is a public facing
API - a pure client). I think most projects could benefit from this with the
exception of things that are tightly coupled to Hadoop releases or touch
non-public APIs.
>  I would actually prefer to move it to Extras or Incubator and leave this
>> within the ASF.
>>
>
> Extras is **NOT** inside of the ASF. Extras is a source hosting system for
> non-Apache projects that are related to Apache projects.
Got it. Thanks for correcting me. I only mentioned it because someone
suggested it to me initially.
> Right now, I picked github because of the ability to easily
> collaborate with others (and to use git).
>

I agree that it is unfortunate that Apache doesn't yet support read-write
> git access. However, you'll find that building a community is easier at
> Apache than at github.
>

> -- Owen
>

--
Eric Sammer
twitter: esammer
data: www.cloudera.com

Guys, BTW, if you need help or a mentor in Apache Incubator-ville for MRUnit, I would be happy to help.

Cheers,
Chris

On Feb 11, 2011, at 9:04 AM, Eric Sammer wrote:

> On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>
>> On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:
>>
>> - allow mrunit to have its own release cycle. This is, I think, the most
>>>
>>
>> important.
>>>
>>
>> If you submit your work to Apache we can evaluate it for inclusion in the
>> 0.20.100 branch to get your changes released in a timely manner.
>
>
> I'm thinking in general (beyond the next immediate release). Independent of
> where mrunit goes, I think it should leave the contrib tree to facilitate
> light weight releases (the dependency on Hadoop proper is a public facing
> API - a pure client). I think most projects could benefit from this with the
> exception of things that are tightly coupled to Hadoop releases or touch
> non-public APIs.
>
>
>> I would actually prefer to move it to Extras or Incubator and leave this
>>> within the ASF.
>>>
>>
>> Extras is **NOT** inside of the ASF. Extras is a source hosting system for
>> non-Apache projects that are related to Apache projects.
>
>
> Got it. Thanks for correcting me. I only mentioned it because someone
> suggested it to me initially.
>
>
>> Right now, I picked github because of the ability to easily
>> collaborate with others (and to use git).
>>
>
> I agree that it is unfortunate that Apache doesn't yet support read-write
>> git access. However, you'll find that building a community is easier at
>> Apache than at github.
>>
>
>> -- Owen
>>
>
>
>
> --
> Eric Sammer
> twitter: esammer
> data: www.cloudera.com
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

On Fri, Feb 11, 2011 at 9:04 AM, Eric Sammer <[EMAIL PROTECTED]> wrote:

> On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley <[EMAIL PROTECTED]>
> wrote:
>
> > On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:
> >
> >  - allow mrunit to have its own release cycle. This is, I think, the most
> >>
> >
> >  important.
> >>
> >
> > If you submit your work to Apache we can evaluate it for inclusion in the
> > 0.20.100 branch to get your changes released in a timely manner.
>
>
> I'm thinking in general (beyond the next immediate release). Independent of
> where mrunit goes, I think it should leave the contrib tree to facilitate
> light weight releases (the dependency on Hadoop proper is a public facing
> API - a pure client). I think most projects could benefit from this with
> the
> exception of things that are tightly coupled to Hadoop releases or touch
> non-public APIs.
>
>
+1 for a faster release cycle and using git.  I have a couple of patches for
mrunit, and it would be nice to get those in separately from hadoop
releases.
>
> >  I would actually prefer to move it to Extras or Incubator and leave this
> >> within the ASF.
> >>
> >
> > Extras is **NOT** inside of the ASF. Extras is a source hosting system
> for
> > non-Apache projects that are related to Apache projects.
>
>
> Got it. Thanks for correcting me. I only mentioned it because someone
> suggested it to me initially.
>
>
> > Right now, I picked github because of the ability to easily
> > collaborate with others (and to use git).
> >
>
> I agree that it is unfortunate that Apache doesn't yet support read-write
> > git access. However, you'll find that building a community is easier at
> > Apache than at github.
> >
>
> > -- Owen
> >
>
>
>
> --
> Eric Sammer
> twitter: esammer
> data: www.cloudera.com
>

On Fri, Feb 11, 2011 at 9:44 AM, Mattmann, Chris A (388J)
<[EMAIL PROTECTED]> wrote:
> Guys, BTW, if you need help or a mentor in Apache Incubator-ville for MRUnit, I would be happy to help.

I was going to suggest the same thing (mrunit to incubator). I would
also be happy to be a mentor.

Patrick

>
> On Feb 11, 2011, at 9:04 AM, Eric Sammer wrote:
>
>> On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>>
>>> On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:
>>>
>>> - allow mrunit to have its own release cycle. This is, I think, the most
>>>>
>>>
>>> important.
>>>>
>>>
>>> If you submit your work to Apache we can evaluate it for inclusion in the
>>> 0.20.100 branch to get your changes released in a timely manner.
>>
>>
>> I'm thinking in general (beyond the next immediate release). Independent of
>> where mrunit goes, I think it should leave the contrib tree to facilitate
>> light weight releases (the dependency on Hadoop proper is a public facing
>> API - a pure client). I think most projects could benefit from this with the
>> exception of things that are tightly coupled to Hadoop releases or touch
>> non-public APIs.
>>
>>
>>> I would actually prefer to move it to Extras or Incubator and leave this
>>>> within the ASF.
>>>>
>>>
>>> Extras is **NOT** inside of the ASF. Extras is a source hosting system for
>>> non-Apache projects that are related to Apache projects.
>>
>>
>> Got it. Thanks for correcting me. I only mentioned it because someone
>> suggested it to me initially.
>>
>>
>>> Right now, I picked github because of the ability to easily
>>> collaborate with others (and to use git).
>>>
>>
>> I agree that it is unfortunate that Apache doesn't yet support read-write
>>> git access. However, you'll find that building a community is easier at
>>> Apache than at github.
>>>
>>
>>> -- Owen
>>>
>>
>>
>>
>> --
>> Eric Sammer
>> twitter: esammer
>> data: www.cloudera.com
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: [EMAIL PROTECTED]
> WWW:   http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>

Awesome Patrick, we'd probably need one more active mentor. Any takers?

After we get that, then we cook up a proposal on the Incubator wiki here [1], and follow the process here [2] to get started...

Cheers,
Chris

[1] http://wiki.apache.org/incubator/MRUnitProposal
[2] http://incubator.apache.org/guides/proposal.html

On Feb 11, 2011, at 11:52 AM, Patrick Hunt wrote:

> On Fri, Feb 11, 2011 at 9:44 AM, Mattmann, Chris A (388J)
> <[EMAIL PROTECTED]> wrote:
>> Guys, BTW, if you need help or a mentor in Apache Incubator-ville for MRUnit, I would be happy to help.
>
> I was going to suggest the same thing (mrunit to incubator). I would
> also be happy to be a mentor.
>
> Patrick
>
>>
>> On Feb 11, 2011, at 9:04 AM, Eric Sammer wrote:
>>
>>> On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>>>
>>>> On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:
>>>>
>>>> - allow mrunit to have its own release cycle. This is, I think, the most
>>>>>
>>>>
>>>> important.
>>>>>
>>>>
>>>> If you submit your work to Apache we can evaluate it for inclusion in the
>>>> 0.20.100 branch to get your changes released in a timely manner.
>>>
>>>
>>> I'm thinking in general (beyond the next immediate release). Independent of
>>> where mrunit goes, I think it should leave the contrib tree to facilitate
>>> light weight releases (the dependency on Hadoop proper is a public facing
>>> API - a pure client). I think most projects could benefit from this with the
>>> exception of things that are tightly coupled to Hadoop releases or touch
>>> non-public APIs.
>>>
>>>
>>>> I would actually prefer to move it to Extras or Incubator and leave this
>>>>> within the ASF.
>>>>>
>>>>
>>>> Extras is **NOT** inside of the ASF. Extras is a source hosting system for
>>>> non-Apache projects that are related to Apache projects.
>>>
>>>
>>> Got it. Thanks for correcting me. I only mentioned it because someone
>>> suggested it to me initially.
>>>
>>>
>>>> Right now, I picked github because of the ability to easily
>>>> collaborate with others (and to use git).
>>>>
>>>
>>> I agree that it is unfortunate that Apache doesn't yet support read-write
>>>> git access. However, you'll find that building a community is easier at
>>>> Apache than at github.
>>>>
>>>
>>>> -- Owen
>>>>
>>>
>>>
>>>
>>> --
>>> Eric Sammer
>>> twitter: esammer
>>> data: www.cloudera.com
>>
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: [EMAIL PROTECTED]
>> WWW:   http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

The main reason I am interested in removing MRUnit from Hadoop is that I
believe that MRUnit deserves its own release cycle. I think this is in the
best interest of its users.

MRUnit is valuable to users of several different versions of Hadoop. But
MRUnit has only ever been committed to version 0.21 and above -- even though
in practice, the majority (dare I say--all) of its users are running on
0.20. The only place today to get a version of MRUnit compatible with 0.20
has been through a Cloudera release, which backported the entire MRUnit
patchset.

My thoughts on MRUnit in 0.20.100 resonate with Eric's. There will be
further fixes to MRUnit and its lightweight codebase can be released far
more rapidly than whenever the next 0.20.1xx release of Hadoop would occur.
Given that MRUnit has already been in the repository since April 2009 (see
https://issues.apache.org/jira/browse/HADOOP-5518) and has yet to see an
Apache 0.20-based release, I do not think it is in the best interest of the
library's userbase to couple MRUnit's release cycle to that of Hadoop
itself.

Perhaps more importantly, access to new features in MRUnit should not
require upgrading one's entire Hadoop deployment; this is a client library
that depends only on Hadoop's public APIs.

My primary concern is to move MRUnit to a place where the community can
derive the most benefit from it. The Apache Incubator could fulfill this
role; given the presence of individuals willing to mentor this project, I
believe this would be a successful way to release MRUnit more quickly and
continue to work to grow the MRUnit community.

Regards,
- Aaron
On Fri, Feb 11, 2011 at 11:57 AM, Mattmann, Chris A (388J) <
[EMAIL PROTECTED]> wrote:

> Awesome Patrick, we'd probably need one more active mentor. Any takers?
>
> After we get that, then we cook up a proposal on the Incubator wiki here
> [1], and follow the process here [2] to get started...
>
> Cheers,
> Chris
>
> [1] http://wiki.apache.org/incubator/MRUnitProposal
> [2] http://incubator.apache.org/guides/proposal.html
>
> On Feb 11, 2011, at 11:52 AM, Patrick Hunt wrote:
>
> > On Fri, Feb 11, 2011 at 9:44 AM, Mattmann, Chris A (388J)
> > <[EMAIL PROTECTED]> wrote:
> >> Guys, BTW, if you need help or a mentor in Apache Incubator-ville for
> MRUnit, I would be happy to help.
> >
> > I was going to suggest the same thing (mrunit to incubator). I would
> > also be happy to be a mentor.
> >
> > Patrick
> >
> >>
> >> On Feb 11, 2011, at 9:04 AM, Eric Sammer wrote:
> >>
> >>> On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley <[EMAIL PROTECTED]>
> wrote:
> >>>
> >>>> On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:
> >>>>
> >>>> - allow mrunit to have its own release cycle. This is, I think, the
> most
> >>>>>
> >>>>
> >>>> important.
> >>>>>
> >>>>
> >>>> If you submit your work to Apache we can evaluate it for inclusion in
> the
> >>>> 0.20.100 branch to get your changes released in a timely manner.
> >>>
> >>>
> >>> I'm thinking in general (beyond the next immediate release).
> Independent of
> >>> where mrunit goes, I think it should leave the contrib tree to
> facilitate
> >>> light weight releases (the dependency on Hadoop proper is a public
> facing
> >>> API - a pure client). I think most projects could benefit from this
> with the
> >>> exception of things that are tightly coupled to Hadoop releases or
> touch
> >>> non-public APIs.
> >>>
> >>>
> >>>> I would actually prefer to move it to Extras or Incubator and leave
> this
> >>>>> within the ASF.
> >>>>>
> >>>>
> >>>> Extras is **NOT** inside of the ASF. Extras is a source hosting system
> for
> >>>> non-Apache projects that are related to Apache projects.
> >>>
> >>>
> >>> Got it. Thanks for correcting me. I only mentioned it because someone
> >>> suggested it to me initially.
> >>>
> >>>
> >>>> Right now, I picked github because of the ability to easily
> >>>> collaborate with others (and to use git).
> >>>>
> >>>
> >>> I agree that it is unfortunate that Apache doesn't yet support

Just to add to the option of going to incubator, I'm fine with that as well.
Github was an easy thing to get started and I was under the impression we
needed some greater degree of committer diversity and, frankly, a bigger
project. If mrunit is a candidate, keeping this under the ASF umbrella is
more than fine with me.

On Fri, Feb 11, 2011 at 5:10 PM, Aaron Kimball <[EMAIL PROTECTED]> wrote:

> The main reason I am interested in removing MRUnit from Hadoop is that I
> believe that MRUnit deserves its own release cycle. I think this is in the
> best interest of its users.
>
> MRUnit is valuable to users of several different versions of Hadoop. But
> MRUnit has only ever been committed to version 0.21 and above -- even
> though
> in practice, the majority (dare I say--all) of its users are running on
> 0.20. The only place today to get a version of MRUnit compatible with 0.20
> has been through a Cloudera release, which backported the entire MRUnit
> patchset.
>
> My thoughts on MRUnit in 0.20.100 resonate with Eric's. There will be
> further fixes to MRUnit and its lightweight codebase can be released far
> more rapidly than whenever the next 0.20.1xx release of Hadoop would occur.
> Given that MRUnit has already been in the repository since April 2009 (see
> https://issues.apache.org/jira/browse/HADOOP-5518) and has yet to see an
> Apache 0.20-based release, I do not think it is in the best interest of the
> library's userbase to couple MRUnit's release cycle to that of Hadoop
> itself.
>
> Perhaps more importantly, access to new features in MRUnit should not
> require upgrading one's entire Hadoop deployment; this is a client library
> that depends only on Hadoop's public APIs.
>
> My primary concern is to move MRUnit to a place where the community can
> derive the most benefit from it. The Apache Incubator could fulfill this
> role; given the presence of individuals willing to mentor this project, I
> believe this would be a successful way to release MRUnit more quickly and
> continue to work to grow the MRUnit community.
>
> Regards,
> - Aaron
>
>
> On Fri, Feb 11, 2011 at 11:57 AM, Mattmann, Chris A (388J) <
> [EMAIL PROTECTED]> wrote:
>
> > Awesome Patrick, we'd probably need one more active mentor. Any takers?
> >
> > After we get that, then we cook up a proposal on the Incubator wiki here
> > [1], and follow the process here [2] to get started...
> >
> > Cheers,
> > Chris
> >
> > [1] http://wiki.apache.org/incubator/MRUnitProposal
> > [2] http://incubator.apache.org/guides/proposal.html
> >
> > On Feb 11, 2011, at 11:52 AM, Patrick Hunt wrote:
> >
> > > On Fri, Feb 11, 2011 at 9:44 AM, Mattmann, Chris A (388J)
> > > <[EMAIL PROTECTED]> wrote:
> > >> Guys, BTW, if you need help or a mentor in Apache Incubator-ville for
> > MRUnit, I would be happy to help.
> > >
> > > I was going to suggest the same thing (mrunit to incubator). I would
> > > also be happy to be a mentor.
> > >
> > > Patrick
> > >
> > >>
> > >> On Feb 11, 2011, at 9:04 AM, Eric Sammer wrote:
> > >>
> > >>> On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley <[EMAIL PROTECTED]>
> > wrote:
> > >>>
> > >>>> On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:
> > >>>>
> > >>>> - allow mrunit to have its own release cycle. This is, I think, the
> > most
> > >>>>>
> > >>>>
> > >>>> important.
> > >>>>>
> > >>>>
> > >>>> If you submit your work to Apache we can evaluate it for inclusion
> in
> > the
> > >>>> 0.20.100 branch to get your changes released in a timely manner.
> > >>>
> > >>>
> > >>> I'm thinking in general (beyond the next immediate release).
> > Independent of
> > >>> where mrunit goes, I think it should leave the contrib tree to
> > facilitate
> > >>> light weight releases (the dependency on Hadoop proper is a public
> > facing
> > >>> API - a pure client). I think most projects could benefit from this
> > with the
> > >>> exception of things that are tightly coupled to Hadoop releases or
> > touch
> > >>> non-public APIs.
>
Eric Sammer
twitter: esammer
data: www.cloudera.com

This is great!  So we'll leave mrunit in contrib until it can be moved to incubator.

Nige

On Feb 11, 2011, at 2:26 PM, Eric Sammer wrote:

> Just to add to the option of going to incubator, I'm fine with that as well.
> Github was an easy thing to get started and I was under the impression we
> needed some greater degree of committer diversity and, frankly, a bigger
> project. If mrunit is a candidate, keeping this under the ASF umbrella is
> more than fine with me.
>
> On Fri, Feb 11, 2011 at 5:10 PM, Aaron Kimball <[EMAIL PROTECTED]> wrote:
>
>> The main reason I am interested in removing MRUnit from Hadoop is that I
>> believe that MRUnit deserves its own release cycle. I think this is in the
>> best interest of its users.
>>
>> MRUnit is valuable to users of several different versions of Hadoop. But
>> MRUnit has only ever been committed to version 0.21 and above -- even
>> though
>> in practice, the majority (dare I say--all) of its users are running on
>> 0.20. The only place today to get a version of MRUnit compatible with 0.20
>> has been through a Cloudera release, which backported the entire MRUnit
>> patchset.
>>
>> My thoughts on MRUnit in 0.20.100 resonate with Eric's. There will be
>> further fixes to MRUnit and its lightweight codebase can be released far
>> more rapidly than whenever the next 0.20.1xx release of Hadoop would occur.
>> Given that MRUnit has already been in the repository since April 2009 (see
>> https://issues.apache.org/jira/browse/HADOOP-5518) and has yet to see an
>> Apache 0.20-based release, I do not think it is in the best interest of the
>> library's userbase to couple MRUnit's release cycle to that of Hadoop
>> itself.
>>
>> Perhaps more importantly, access to new features in MRUnit should not
>> require upgrading one's entire Hadoop deployment; this is a client library
>> that depends only on Hadoop's public APIs.
>>
>> My primary concern is to move MRUnit to a place where the community can
>> derive the most benefit from it. The Apache Incubator could fulfill this
>> role; given the presence of individuals willing to mentor this project, I
>> believe this would be a successful way to release MRUnit more quickly and
>> continue to work to grow the MRUnit community.
>>
>> Regards,
>> - Aaron
>>
>>
>> On Fri, Feb 11, 2011 at 11:57 AM, Mattmann, Chris A (388J) <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Awesome Patrick, we'd probably need one more active mentor. Any takers?
>>>
>>> After we get that, then we cook up a proposal on the Incubator wiki here
>>> [1], and follow the process here [2] to get started...
>>>
>>> Cheers,
>>> Chris
>>>
>>> [1] http://wiki.apache.org/incubator/MRUnitProposal
>>> [2] http://incubator.apache.org/guides/proposal.html
>>>
>>> On Feb 11, 2011, at 11:52 AM, Patrick Hunt wrote:
>>>
>>>> On Fri, Feb 11, 2011 at 9:44 AM, Mattmann, Chris A (388J)
>>>> <[EMAIL PROTECTED]> wrote:
>>>>> Guys, BTW, if you need help or a mentor in Apache Incubator-ville for
>>> MRUnit, I would be happy to help.
>>>>
>>>> I was going to suggest the same thing (mrunit to incubator). I would
>>>> also be happy to be a mentor.
>>>>
>>>> Patrick
>>>>
>>>>>
>>>>> On Feb 11, 2011, at 9:04 AM, Eric Sammer wrote:
>>>>>
>>>>>> On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley <[EMAIL PROTECTED]>
>>> wrote:
>>>>>>
>>>>>>> On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:
>>>>>>>
>>>>>>> - allow mrunit to have its own release cycle. This is, I think, the
>>> most
>>>>>>>>
>>>>>>>
>>>>>>> important.
>>>>>>>>
>>>>>>>
>>>>>>> If you submit your work to Apache we can evaluate it for inclusion
>> in
>>> the
>>>>>>> 0.20.100 branch to get your changes released in a timely manner.
>>>>>>
>>>>>>
>>>>>> I'm thinking in general (beyond the next immediate release).
>>> Independent of
>>>>>> where mrunit goes, I think it should leave the contrib tree to
>>> facilitate
>>>>>> light weight releases (the dependency on Hadoop proper is a public
>>> facing
>>>>>> API - a pure client). I think most projects could benefit from this

On 11/02/11 22:26, Eric Sammer wrote:
> Just to add to the option of going to incubator, I'm fine with that as well.
> Github was an easy thing to get started and I was under the impression we
> needed some greater degree of committer diversity and, frankly, a bigger
> project. If mrunit is a candidate, keeping this under the ASF umbrella is
> more than fine with me.
>

There is a git repository coming up at Apache, with global read and
LDAP-authenticated write:
https://issues.apache.org/jira/browse/INFRA-3165

I know a lot of the in house Hadoop teams use Git for SCM, so I think
having the big Hadoop projects work on Git too makes sense -it would
certainly help Owen and team get their changes over if their branches
could be merged.

Starting off with a small but active Hadoop family project would be the
way to start this, and MRUnit does appeal as a first step
  -active development
  -fast and decoupled release cycle
  -off the critical path

I'll volunteer to go down as a committer with the usual caveat that I am
over-committed and unless I have a direct personal need to touch the
code I'll be effectively offline. Tom White can vouch for that, given my
engagement in Whirr.

Here then are my votes

  +1 to move Mrunit into incubator
  +1 to experiment with mrunit being live in Apache's Git repository.

-steve

I'm happy to help mentor as well.

Cheers,
Nige

On Feb 11, 2011, at 11:52 AM, Patrick Hunt wrote:

> On Fri, Feb 11, 2011 at 9:44 AM, Mattmann, Chris A (388J)
> <[EMAIL PROTECTED]> wrote:
>> Guys, BTW, if you need help or a mentor in Apache Incubator-ville for MRUnit, I would be happy to help.
>
> I was going to suggest the same thing (mrunit to incubator). I would
> also be happy to be a mentor.
>
> Patrick
>
>>
>> On Feb 11, 2011, at 9:04 AM, Eric Sammer wrote:
>>
>>> On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>>>
>>>> On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:
>>>>
>>>> - allow mrunit to have its own release cycle. This is, I think, the most
>>>>>
>>>>
>>>> important.
>>>>>
>>>>
>>>> If you submit your work to Apache we can evaluate it for inclusion in the
>>>> 0.20.100 branch to get your changes released in a timely manner.
>>>
>>>
>>> I'm thinking in general (beyond the next immediate release). Independent of
>>> where mrunit goes, I think it should leave the contrib tree to facilitate
>>> light weight releases (the dependency on Hadoop proper is a public facing
>>> API - a pure client). I think most projects could benefit from this with the
>>> exception of things that are tightly coupled to Hadoop releases or touch
>>> non-public APIs.
>>>
>>>
>>>> I would actually prefer to move it to Extras or Incubator and leave this
>>>>> within the ASF.
>>>>>
>>>>
>>>> Extras is **NOT** inside of the ASF. Extras is a source hosting system for
>>>> non-Apache projects that are related to Apache projects.
>>>
>>>
>>> Got it. Thanks for correcting me. I only mentioned it because someone
>>> suggested it to me initially.
>>>
>>>
>>>> Right now, I picked github because of the ability to easily
>>>> collaborate with others (and to use git).
>>>>
>>>
>>> I agree that it is unfortunate that Apache doesn't yet support read-write
>>>> git access. However, you'll find that building a community is easier at
>>>> Apache than at github.
>>>>
>>>
>>>> -- Owen
>>>>
>>>
>>>
>>>
>>> --
>>> Eric Sammer
>>> twitter: esammer
>>> data: www.cloudera.com
>>
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: [EMAIL PROTECTED]
>> WWW:   http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>

I've started the wiki page proposal for Incubator for mrunit. I'll ping
people off list for mentoring. Much appreciated for all the help!

On Tue, Feb 15, 2011 at 1:36 PM, Nigel Daley <[EMAIL PROTECTED]> wrote:

> I'm happy to help mentor as well.
>
> Cheers,
> Nige
>
> On Feb 11, 2011, at 11:52 AM, Patrick Hunt wrote:
>
> > On Fri, Feb 11, 2011 at 9:44 AM, Mattmann, Chris A (388J)
> > <[EMAIL PROTECTED]> wrote:
> >> Guys, BTW, if you need help or a mentor in Apache Incubator-ville for
> MRUnit, I would be happy to help.
> >
> > I was going to suggest the same thing (mrunit to incubator). I would
> > also be happy to be a mentor.
> >
> > Patrick
> >
> >>
> >> On Feb 11, 2011, at 9:04 AM, Eric Sammer wrote:
> >>
> >>> On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley <[EMAIL PROTECTED]>
> wrote:
> >>>
> >>>> On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:
> >>>>
> >>>> - allow mrunit to have its own release cycle. This is, I think, the
> most
> >>>>>
> >>>>
> >>>> important.
> >>>>>
> >>>>
> >>>> If you submit your work to Apache we can evaluate it for inclusion in
> the
> >>>> 0.20.100 branch to get your changes released in a timely manner.
> >>>
> >>>
> >>> I'm thinking in general (beyond the next immediate release).
> Independent of
> >>> where mrunit goes, I think it should leave the contrib tree to
> facilitate
> >>> light weight releases (the dependency on Hadoop proper is a public
> facing
> >>> API - a pure client). I think most projects could benefit from this
> with the
> >>> exception of things that are tightly coupled to Hadoop releases or
> touch
> >>> non-public APIs.
> >>>
> >>>
> >>>> I would actually prefer to move it to Extras or Incubator and leave
> this
> >>>>> within the ASF.
> >>>>>
> >>>>
> >>>> Extras is **NOT** inside of the ASF. Extras is a source hosting system
> for
> >>>> non-Apache projects that are related to Apache projects.
> >>>
> >>>
> >>> Got it. Thanks for correcting me. I only mentioned it because someone
> >>> suggested it to me initially.
> >>>
> >>>
> >>>> Right now, I picked github because of the ability to easily
> >>>> collaborate with others (and to use git).
> >>>>
> >>>
> >>> I agree that it is unfortunate that Apache doesn't yet support
> read-write
> >>>> git access. However, you'll find that building a community is easier
> at
> >>>> Apache than at github.
> >>>>
> >>>
> >>>> -- Owen
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Eric Sammer
> >>> twitter: esammer
> >>> data: www.cloudera.com
> >>
> >>
> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> Chris Mattmann, Ph.D.
> >> Senior Computer Scientist
> >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> >> Office: 171-266B, Mailstop: 171-246
> >> Email: [EMAIL PROTECTED]
> >> WWW:   http://sunset.usc.edu/~mattmann/
> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> Adjunct Assistant Professor, Computer Science Department
> >> University of Southern California, Los Angeles, CA 90089 USA
> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>
> >>
>
>
--
Eric Sammer
twitter: esammer
data: www.cloudera.com

I am up to help as a committer on this project.
--
  Take care,
Konstantin (Cos) Boudnik
On Tue, Feb 15, 2011 at 10:36, Nigel Daley <[EMAIL PROTECTED]> wrote:
> I'm happy to help mentor as well.
>
> Cheers,
> Nige
>
> On Feb 11, 2011, at 11:52 AM, Patrick Hunt wrote:
>
>> On Fri, Feb 11, 2011 at 9:44 AM, Mattmann, Chris A (388J)
>> <[EMAIL PROTECTED]> wrote:
>>> Guys, BTW, if you need help or a mentor in Apache Incubator-ville for MRUnit, I would be happy to help.
>>
>> I was going to suggest the same thing (mrunit to incubator). I would
>> also be happy to be a mentor.
>>
>> Patrick
>>
>>>
>>> On Feb 11, 2011, at 9:04 AM, Eric Sammer wrote:
>>>
>>>> On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:
>>>>>
>>>>> - allow mrunit to have its own release cycle. This is, I think, the most
>>>>>>
>>>>>
>>>>> important.
>>>>>>
>>>>>
>>>>> If you submit your work to Apache we can evaluate it for inclusion in the
>>>>> 0.20.100 branch to get your changes released in a timely manner.
>>>>
>>>>
>>>> I'm thinking in general (beyond the next immediate release). Independent of
>>>> where mrunit goes, I think it should leave the contrib tree to facilitate
>>>> light weight releases (the dependency on Hadoop proper is a public facing
>>>> API - a pure client). I think most projects could benefit from this with the
>>>> exception of things that are tightly coupled to Hadoop releases or touch
>>>> non-public APIs.
>>>>
>>>>
>>>>> I would actually prefer to move it to Extras or Incubator and leave this
>>>>>> within the ASF.
>>>>>>
>>>>>
>>>>> Extras is **NOT** inside of the ASF. Extras is a source hosting system for
>>>>> non-Apache projects that are related to Apache projects.
>>>>
>>>>
>>>> Got it. Thanks for correcting me. I only mentioned it because someone
>>>> suggested it to me initially.
>>>>
>>>>
>>>>> Right now, I picked github because of the ability to easily
>>>>> collaborate with others (and to use git).
>>>>>
>>>>
>>>> I agree that it is unfortunate that Apache doesn't yet support read-write
>>>>> git access. However, you'll find that building a community is easier at
>>>>> Apache than at github.
>>>>>
>>>>
>>>>> -- Owen
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Eric Sammer
>>>> twitter: esammer
>>>> data: www.cloudera.com
>>>
>>>
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Chris Mattmann, Ph.D.
>>> Senior Computer Scientist
>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>> Office: 171-266B, Mailstop: 171-246
>>> Email: [EMAIL PROTECTED]
>>> WWW:   http://sunset.usc.edu/~mattmann/
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Adjunct Assistant Professor, Computer Science Department
>>> University of Southern California, Los Angeles, CA 90089 USA
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>>>
>
>

On Feb 11, 2011, at 9:57 PM, Nigel Daley wrote:

> This is great!  So we'll leave mrunit in contrib until it can be  
> moved to incubator.
>

+1

On Fri, Feb 11, 2011 at 23:10, Aaron Kimball <[EMAIL PROTECTED]> wrote:
> The main reason I am interested in removing MRUnit from Hadoop is that I
> believe that MRUnit deserves its own release cycle. I think this is in the
> best interest of its users.

Not in mine, at least. (I'm writing MR unit tests.)
Many projects release more than one product. I'd rather get MRUnit
from the same source where I get my MR from.
Separate release cylcles would be ok for me, though.

> Perhaps more importantly, access to new features in MRUnit should not
> require upgrading one's entire Hadoop deployment; this is a client library
> that depends only on Hadoop's public APIs.

+1.

> My primary concern is to move MRUnit to a place where the community can
> derive the most benefit from it. The Apache Incubator could fulfill this
> role; given the presence of individuals willing to mentor this project, I
> believe this would be a successful way to release MRUnit more quickly and
> continue to work to grow the MRUnit community.

What are your expectations what MRUnit would become, software-wise?
Wouldn't the MRUnit community be largely the same as the Hadoop-MR
community?

  Bernd

The MRUnit community is a specific subset of the Hadoop community: Engineers
writing Java code running on Hadoop. The Hadoop community also includes
IT/ops staff who maintain Hadoop clusters, data scientists who use tools
such as Pig & Hive, as well as those written by the aforementioned
engineers, etc.

The Hadoop project has long recognized that tools aimed at a specific subset
of the Hadoop community, with separate release cycles, can more successfully
reach their aims by splitting into incubator projects. Hive, Pig, and HBase,
for example, have all gone this path.

A "current" version of MRUnit would need to compile against multiple
versions of Hadoop itself. This is not possible if it is in the same source
tree as Hadoop.

- Aaron

On Thu, Feb 17, 2011 at 5:31 AM, Bernd Fondermann <
[EMAIL PROTECTED]> wrote:

> On Fri, Feb 11, 2011 at 23:10, Aaron Kimball <[EMAIL PROTECTED]> wrote:
> > The main reason I am interested in removing MRUnit from Hadoop is that I
> > believe that MRUnit deserves its own release cycle. I think this is in
> the
> > best interest of its users.
>
> Not in mine, at least. (I'm writing MR unit tests.)
> Many projects release more than one product. I'd rather get MRUnit
> from the same source where I get my MR from.
> Separate release cylcles would be ok for me, though.
>
> > Perhaps more importantly, access to new features in MRUnit should not
> > require upgrading one's entire Hadoop deployment; this is a client
> library
> > that depends only on Hadoop's public APIs.
>
> +1.
>
> > My primary concern is to move MRUnit to a place where the community can
> > derive the most benefit from it. The Apache Incubator could fulfill this
> > role; given the presence of individuals willing to mentor this project, I
> > believe this would be a successful way to release MRUnit more quickly and
> > continue to work to grow the MRUnit community.
>
> What are your expectations what MRUnit would become, software-wise?
> Wouldn't the MRUnit community be largely the same as the Hadoop-MR
> community?
>
>  Bernd
>

Hey Guys,

FYI on this: Eric has mentioned he is going to start the Incubator proposal for MRUnit. Let's start small and then grow big (as needed). It seems like we've achieved enough consensus for the required mentors and critical mass to make an MRUnit Incubator proposal and then to have the Incubator community weigh in. If that expands to include other testing projects/etc., we can address that over the Incubation process, and as needed.

Eric: as soon as that wiki page is up, I'd be happy to add my name to it as a mentor and /kick the can on this.

Cheers,
Chris

On Feb 17, 2011, at 11:11 AM, Aaron Kimball wrote:

> The MRUnit community is a specific subset of the Hadoop community: Engineers
> writing Java code running on Hadoop. The Hadoop community also includes
> IT/ops staff who maintain Hadoop clusters, data scientists who use tools
> such as Pig & Hive, as well as those written by the aforementioned
> engineers, etc.
>
> The Hadoop project has long recognized that tools aimed at a specific subset
> of the Hadoop community, with separate release cycles, can more successfully
> reach their aims by splitting into incubator projects. Hive, Pig, and HBase,
> for example, have all gone this path.
>
> A "current" version of MRUnit would need to compile against multiple
> versions of Hadoop itself. This is not possible if it is in the same source
> tree as Hadoop.
>
> - Aaron
>
> On Thu, Feb 17, 2011 at 5:31 AM, Bernd Fondermann <
> [EMAIL PROTECTED]> wrote:
>
>> On Fri, Feb 11, 2011 at 23:10, Aaron Kimball <[EMAIL PROTECTED]> wrote:
>>> The main reason I am interested in removing MRUnit from Hadoop is that I
>>> believe that MRUnit deserves its own release cycle. I think this is in
>> the
>>> best interest of its users.
>>
>> Not in mine, at least. (I'm writing MR unit tests.)
>> Many projects release more than one product. I'd rather get MRUnit
>> from the same source where I get my MR from.
>> Separate release cylcles would be ok for me, though.
>>
>>> Perhaps more importantly, access to new features in MRUnit should not
>>> require upgrading one's entire Hadoop deployment; this is a client
>> library
>>> that depends only on Hadoop's public APIs.
>>
>> +1.
>>
>>> My primary concern is to move MRUnit to a place where the community can
>>> derive the most benefit from it. The Apache Incubator could fulfill this
>>> role; given the presence of individuals willing to mentor this project, I
>>> believe this would be a successful way to release MRUnit more quickly and
>>> continue to work to grow the MRUnit community.
>>
>> What are your expectations what MRUnit would become, software-wise?
>> Wouldn't the MRUnit community be largely the same as the Hadoop-MR
>> community?
>>
>> Bernd
>>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Chris a page is up (still being created by Eric afaict):
http://wiki.apache.org/incubator/MRUnitProposal

I took the liberty of listing us as mentors.

Patrick

On Thu, Feb 17, 2011 at 11:31 AM, Mattmann, Chris A (388J)
<[EMAIL PROTECTED]> wrote:
> Hey Guys,
>
> FYI on this: Eric has mentioned he is going to start the Incubator proposal for MRUnit. Let's start small and then grow big (as needed). It seems like we've achieved enough consensus for the required mentors and critical mass to make an MRUnit Incubator proposal and then to have the Incubator community weigh in. If that expands to include other testing projects/etc., we can address that over the Incubation process, and as needed.
>
> Eric: as soon as that wiki page is up, I'd be happy to add my name to it as a mentor and /kick the can on this.
>
> Cheers,
> Chris
>
> On Feb 17, 2011, at 11:11 AM, Aaron Kimball wrote:
>
>> The MRUnit community is a specific subset of the Hadoop community: Engineers
>> writing Java code running on Hadoop. The Hadoop community also includes
>> IT/ops staff who maintain Hadoop clusters, data scientists who use tools
>> such as Pig & Hive, as well as those written by the aforementioned
>> engineers, etc.
>>
>> The Hadoop project has long recognized that tools aimed at a specific subset
>> of the Hadoop community, with separate release cycles, can more successfully
>> reach their aims by splitting into incubator projects. Hive, Pig, and HBase,
>> for example, have all gone this path.
>>
>> A "current" version of MRUnit would need to compile against multiple
>> versions of Hadoop itself. This is not possible if it is in the same source
>> tree as Hadoop.
>>
>> - Aaron
>>
>> On Thu, Feb 17, 2011 at 5:31 AM, Bernd Fondermann <
>> [EMAIL PROTECTED]> wrote:
>>
>>> On Fri, Feb 11, 2011 at 23:10, Aaron Kimball <[EMAIL PROTECTED]> wrote:
>>>> The main reason I am interested in removing MRUnit from Hadoop is that I
>>>> believe that MRUnit deserves its own release cycle. I think this is in
>>> the
>>>> best interest of its users.
>>>
>>> Not in mine, at least. (I'm writing MR unit tests.)
>>> Many projects release more than one product. I'd rather get MRUnit
>>> from the same source where I get my MR from.
>>> Separate release cylcles would be ok for me, though.
>>>
>>>> Perhaps more importantly, access to new features in MRUnit should not
>>>> require upgrading one's entire Hadoop deployment; this is a client
>>> library
>>>> that depends only on Hadoop's public APIs.
>>>
>>> +1.
>>>
>>>> My primary concern is to move MRUnit to a place where the community can
>>>> derive the most benefit from it. The Apache Incubator could fulfill this
>>>> role; given the presence of individuals willing to mentor this project, I
>>>> believe this would be a successful way to release MRUnit more quickly and
>>>> continue to work to grow the MRUnit community.
>>>
>>> What are your expectations what MRUnit would become, software-wise?
>>> Wouldn't the MRUnit community be largely the same as the Hadoop-MR
>>> community?
>>>
>>> Bernd
>>>
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: [EMAIL PROTECTED]
> WWW:   http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>

Thanks Patrick.

Sounds good. Besides mentors, we need a champion, who must be an ASF member. So, either me, you or Nigel can do it. I'm already championing Gora, and would be happy to champion MRUnit, but am open to either one of you guys doing it too (or even another ASF member). Just let me know. I'll head over to the wiki and add my info. Thanks for getting this started Eric+Patrick!

Cheers,
Chris

On Feb 17, 2011, at 1:00 PM, Patrick Hunt wrote:

> Chris a page is up (still being created by Eric afaict):
> http://wiki.apache.org/incubator/MRUnitProposal
>
> I took the liberty of listing us as mentors.
>
> Patrick
>
> On Thu, Feb 17, 2011 at 11:31 AM, Mattmann, Chris A (388J)
> <[EMAIL PROTECTED]> wrote:
>> Hey Guys,
>>
>> FYI on this: Eric has mentioned he is going to start the Incubator proposal for MRUnit. Let's start small and then grow big (as needed). It seems like we've achieved enough consensus for the required mentors and critical mass to make an MRUnit Incubator proposal and then to have the Incubator community weigh in. If that expands to include other testing projects/etc., we can address that over the Incubation process, and as needed.
>>
>> Eric: as soon as that wiki page is up, I'd be happy to add my name to it as a mentor and /kick the can on this.
>>
>> Cheers,
>> Chris
>>
>> On Feb 17, 2011, at 11:11 AM, Aaron Kimball wrote:
>>
>>> The MRUnit community is a specific subset of the Hadoop community: Engineers
>>> writing Java code running on Hadoop. The Hadoop community also includes
>>> IT/ops staff who maintain Hadoop clusters, data scientists who use tools
>>> such as Pig & Hive, as well as those written by the aforementioned
>>> engineers, etc.
>>>
>>> The Hadoop project has long recognized that tools aimed at a specific subset
>>> of the Hadoop community, with separate release cycles, can more successfully
>>> reach their aims by splitting into incubator projects. Hive, Pig, and HBase,
>>> for example, have all gone this path.
>>>
>>> A "current" version of MRUnit would need to compile against multiple
>>> versions of Hadoop itself. This is not possible if it is in the same source
>>> tree as Hadoop.
>>>
>>> - Aaron
>>>
>>> On Thu, Feb 17, 2011 at 5:31 AM, Bernd Fondermann <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> On Fri, Feb 11, 2011 at 23:10, Aaron Kimball <[EMAIL PROTECTED]> wrote:
>>>>> The main reason I am interested in removing MRUnit from Hadoop is that I
>>>>> believe that MRUnit deserves its own release cycle. I think this is in
>>>> the
>>>>> best interest of its users.
>>>>
>>>> Not in mine, at least. (I'm writing MR unit tests.)
>>>> Many projects release more than one product. I'd rather get MRUnit
>>>> from the same source where I get my MR from.
>>>> Separate release cylcles would be ok for me, though.
>>>>
>>>>> Perhaps more importantly, access to new features in MRUnit should not
>>>>> require upgrading one's entire Hadoop deployment; this is a client
>>>> library
>>>>> that depends only on Hadoop's public APIs.
>>>>
>>>> +1.
>>>>
>>>>> My primary concern is to move MRUnit to a place where the community can
>>>>> derive the most benefit from it. The Apache Incubator could fulfill this
>>>>> role; given the presence of individuals willing to mentor this project, I
>>>>> believe this would be a successful way to release MRUnit more quickly and
>>>>> continue to work to grow the MRUnit community.
>>>>
>>>> What are your expectations what MRUnit would become, software-wise?
>>>> Wouldn't the MRUnit community be largely the same as the Hadoop-MR
>>>> community?
>>>>
>>>> Bernd
>>>>
>>
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: [EMAIL PROTECTED]
>> WWW:   http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Hi Guys,

I took a pass at the MRUnit proposal [1]. It's far from complete, but just letting you know I took a swipe :)

Cheers,
Chris

[1] http://wiki.apache.org/incubator/MRUnitProposal

On Feb 17, 2011, at 1:52 PM, Mattmann, Chris A (388J) wrote:

> Thanks Patrick.
>
> Sounds good. Besides mentors, we need a champion, who must be an ASF member. So, either me, you or Nigel can do it. I'm already championing Gora, and would be happy to champion MRUnit, but am open to either one of you guys doing it too (or even another ASF member). Just let me know. I'll head over to the wiki and add my info. Thanks for getting this started Eric+Patrick!
>
> Cheers,
> Chris
>
> On Feb 17, 2011, at 1:00 PM, Patrick Hunt wrote:
>
>> Chris a page is up (still being created by Eric afaict):
>> http://wiki.apache.org/incubator/MRUnitProposal
>>
>> I took the liberty of listing us as mentors.
>>
>> Patrick
>>
>> On Thu, Feb 17, 2011 at 11:31 AM, Mattmann, Chris A (388J)
>> <[EMAIL PROTECTED]> wrote:
>>> Hey Guys,
>>>
>>> FYI on this: Eric has mentioned he is going to start the Incubator proposal for MRUnit. Let's start small and then grow big (as needed). It seems like we've achieved enough consensus for the required mentors and critical mass to make an MRUnit Incubator proposal and then to have the Incubator community weigh in. If that expands to include other testing projects/etc., we can address that over the Incubation process, and as needed.
>>>
>>> Eric: as soon as that wiki page is up, I'd be happy to add my name to it as a mentor and /kick the can on this.
>>>
>>> Cheers,
>>> Chris
>>>
>>> On Feb 17, 2011, at 11:11 AM, Aaron Kimball wrote:
>>>
>>>> The MRUnit community is a specific subset of the Hadoop community: Engineers
>>>> writing Java code running on Hadoop. The Hadoop community also includes
>>>> IT/ops staff who maintain Hadoop clusters, data scientists who use tools
>>>> such as Pig & Hive, as well as those written by the aforementioned
>>>> engineers, etc.
>>>>
>>>> The Hadoop project has long recognized that tools aimed at a specific subset
>>>> of the Hadoop community, with separate release cycles, can more successfully
>>>> reach their aims by splitting into incubator projects. Hive, Pig, and HBase,
>>>> for example, have all gone this path.
>>>>
>>>> A "current" version of MRUnit would need to compile against multiple
>>>> versions of Hadoop itself. This is not possible if it is in the same source
>>>> tree as Hadoop.
>>>>
>>>> - Aaron
>>>>
>>>> On Thu, Feb 17, 2011 at 5:31 AM, Bernd Fondermann <
>>>> [EMAIL PROTECTED]> wrote:
>>>>
>>>>> On Fri, Feb 11, 2011 at 23:10, Aaron Kimball <[EMAIL PROTECTED]> wrote:
>>>>>> The main reason I am interested in removing MRUnit from Hadoop is that I
>>>>>> believe that MRUnit deserves its own release cycle. I think this is in
>>>>> the
>>>>>> best interest of its users.
>>>>>
>>>>> Not in mine, at least. (I'm writing MR unit tests.)
>>>>> Many projects release more than one product. I'd rather get MRUnit
>>>>> from the same source where I get my MR from.
>>>>> Separate release cylcles would be ok for me, though.
>>>>>
>>>>>> Perhaps more importantly, access to new features in MRUnit should not
>>>>>> require upgrading one's entire Hadoop deployment; this is a client
>>>>> library
>>>>>> that depends only on Hadoop's public APIs.
>>>>>
>>>>> +1.
>>>>>
>>>>>> My primary concern is to move MRUnit to a place where the community can
>>>>>> derive the most benefit from it. The Apache Incubator could fulfill this
>>>>>> role; given the presence of individuals willing to mentor this project, I
>>>>>> believe this would be a successful way to release MRUnit more quickly and
>>>>>> continue to work to grow the MRUnit community.
>>>>>
>>>>> What are your expectations what MRUnit would become, software-wise?
>>>>> Wouldn't the MRUnit community be largely the same as the Hadoop-MR
>>>>> community?
>>>>>
>>>>> Bernd
>>>>>
>>>
>>>
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

I have made some revisions and improvements here too..
- Aaron

On Sat, Feb 19, 2011 at 6:52 PM, Mattmann, Chris A (388J) <
[EMAIL PROTECTED]> wrote:

> Hi Guys,
>
> I took a pass at the MRUnit proposal [1]. It's far from complete, but just
> letting you know I took a swipe :)
>
> Cheers,
> Chris
>
> [1] http://wiki.apache.org/incubator/MRUnitProposal
>
> On Feb 17, 2011, at 1:52 PM, Mattmann, Chris A (388J) wrote:
>
> > Thanks Patrick.
> >
> > Sounds good. Besides mentors, we need a champion, who must be an ASF
> member. So, either me, you or Nigel can do it. I'm already championing Gora,
> and would be happy to champion MRUnit, but am open to either one of you guys
> doing it too (or even another ASF member). Just let me know. I'll head over
> to the wiki and add my info. Thanks for getting this started Eric+Patrick!
> >
> > Cheers,
> > Chris
> >
> > On Feb 17, 2011, at 1:00 PM, Patrick Hunt wrote:
> >
> >> Chris a page is up (still being created by Eric afaict):
> >> http://wiki.apache.org/incubator/MRUnitProposal
> >>
> >> I took the liberty of listing us as mentors.
> >>
> >> Patrick
> >>
> >> On Thu, Feb 17, 2011 at 11:31 AM, Mattmann, Chris A (388J)
> >> <[EMAIL PROTECTED]> wrote:
> >>> Hey Guys,
> >>>
> >>> FYI on this: Eric has mentioned he is going to start the Incubator
> proposal for MRUnit. Let's start small and then grow big (as needed). It
> seems like we've achieved enough consensus for the required mentors and
> critical mass to make an MRUnit Incubator proposal and then to have the
> Incubator community weigh in. If that expands to include other testing
> projects/etc., we can address that over the Incubation process, and as
> needed.
> >>>
> >>> Eric: as soon as that wiki page is up, I'd be happy to add my name to
> it as a mentor and /kick the can on this.
> >>>
> >>> Cheers,
> >>> Chris
> >>>
> >>> On Feb 17, 2011, at 11:11 AM, Aaron Kimball wrote:
> >>>
> >>>> The MRUnit community is a specific subset of the Hadoop community:
> Engineers
> >>>> writing Java code running on Hadoop. The Hadoop community also
> includes
> >>>> IT/ops staff who maintain Hadoop clusters, data scientists who use
> tools
> >>>> such as Pig & Hive, as well as those written by the aforementioned
> >>>> engineers, etc.
> >>>>
> >>>> The Hadoop project has long recognized that tools aimed at a specific
> subset
> >>>> of the Hadoop community, with separate release cycles, can more
> successfully
> >>>> reach their aims by splitting into incubator projects. Hive, Pig, and
> HBase,
> >>>> for example, have all gone this path.
> >>>>
> >>>> A "current" version of MRUnit would need to compile against multiple
> >>>> versions of Hadoop itself. This is not possible if it is in the same
> source
> >>>> tree as Hadoop.
> >>>>
> >>>> - Aaron
> >>>>
> >>>> On Thu, Feb 17, 2011 at 5:31 AM, Bernd Fondermann <
> >>>> [EMAIL PROTECTED]> wrote:
> >>>>
> >>>>> On Fri, Feb 11, 2011 at 23:10, Aaron Kimball <[EMAIL PROTECTED]>
> wrote:
> >>>>>> The main reason I am interested in removing MRUnit from Hadoop is
> that I
> >>>>>> believe that MRUnit deserves its own release cycle. I think this is
> in
> >>>>> the
> >>>>>> best interest of its users.
> >>>>>
> >>>>> Not in mine, at least. (I'm writing MR unit tests.)
> >>>>> Many projects release more than one product. I'd rather get MRUnit
> >>>>> from the same source where I get my MR from.
> >>>>> Separate release cylcles would be ok for me, though.
> >>>>>
> >>>>>> Perhaps more importantly, access to new features in MRUnit should
> not
> >>>>>> require upgrading one's entire Hadoop deployment; this is a client
> >>>>> library
> >>>>>> that depends only on Hadoop's public APIs.
> >>>>>
> >>>>> +1.
> >>>>>
> >>>>>> My primary concern is to move MRUnit to a place where the community
> can
> >>>>>> derive the most benefit from it. The Apache Incubator could fulfill
> this
> >>>>>> role; given the presence of individuals willing to mentor this
> project, I
> >>>>>> believe this would be a successful way to release MRUnit more

mrunit has been moved to the incubator:
https://svn.apache.org/repos/asf/incubator/mrunit/

I'll file an issue to remove the code from Hadoop SVN.

Nige

On Feb 11, 2011, at 9:57 PM, Nigel Daley wrote:

> This is great!  So we'll leave mrunit in contrib until it can be moved to incubator.
>
> Nige
>
> On Feb 11, 2011, at 2:26 PM, Eric Sammer wrote:
>
>> Just to add to the option of going to incubator, I'm fine with that as well.
>> Github was an easy thing to get started and I was under the impression we
>> needed some greater degree of committer diversity and, frankly, a bigger
>> project. If mrunit is a candidate, keeping this under the ASF umbrella is
>> more than fine with me.
>>
>> On Fri, Feb 11, 2011 at 5:10 PM, Aaron Kimball <[EMAIL PROTECTED]> wrote:
>>
>>> The main reason I am interested in removing MRUnit from Hadoop is that I
>>> believe that MRUnit deserves its own release cycle. I think this is in the
>>> best interest of its users.
>>>
>>> MRUnit is valuable to users of several different versions of Hadoop. But
>>> MRUnit has only ever been committed to version 0.21 and above -- even
>>> though
>>> in practice, the majority (dare I say--all) of its users are running on
>>> 0.20. The only place today to get a version of MRUnit compatible with 0.20
>>> has been through a Cloudera release, which backported the entire MRUnit
>>> patchset.
>>>
>>> My thoughts on MRUnit in 0.20.100 resonate with Eric's. There will be
>>> further fixes to MRUnit and its lightweight codebase can be released far
>>> more rapidly than whenever the next 0.20.1xx release of Hadoop would occur.
>>> Given that MRUnit has already been in the repository since April 2009 (see
>>> https://issues.apache.org/jira/browse/HADOOP-5518) and has yet to see an
>>> Apache 0.20-based release, I do not think it is in the best interest of the
>>> library's userbase to couple MRUnit's release cycle to that of Hadoop
>>> itself.
>>>
>>> Perhaps more importantly, access to new features in MRUnit should not
>>> require upgrading one's entire Hadoop deployment; this is a client library
>>> that depends only on Hadoop's public APIs.
>>>
>>> My primary concern is to move MRUnit to a place where the community can
>>> derive the most benefit from it. The Apache Incubator could fulfill this
>>> role; given the presence of individuals willing to mentor this project, I
>>> believe this would be a successful way to release MRUnit more quickly and
>>> continue to work to grow the MRUnit community.
>>>
>>> Regards,
>>> - Aaron
>>>
>>>
>>> On Fri, Feb 11, 2011 at 11:57 AM, Mattmann, Chris A (388J) <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> Awesome Patrick, we'd probably need one more active mentor. Any takers?
>>>>
>>>> After we get that, then we cook up a proposal on the Incubator wiki here
>>>> [1], and follow the process here [2] to get started...
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>> [1] http://wiki.apache.org/incubator/MRUnitProposal
>>>> [2] http://incubator.apache.org/guides/proposal.html
>>>>
>>>> On Feb 11, 2011, at 11:52 AM, Patrick Hunt wrote:
>>>>
>>>>> On Fri, Feb 11, 2011 at 9:44 AM, Mattmann, Chris A (388J)
>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>> Guys, BTW, if you need help or a mentor in Apache Incubator-ville for
>>>> MRUnit, I would be happy to help.
>>>>>
>>>>> I was going to suggest the same thing (mrunit to incubator). I would
>>>>> also be happy to be a mentor.
>>>>>
>>>>> Patrick
>>>>>
>>>>>>
>>>>>> On Feb 11, 2011, at 9:04 AM, Eric Sammer wrote:
>>>>>>
>>>>>>> On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley <[EMAIL PROTECTED]>
>>>> wrote:
>>>>>>>
>>>>>>>> On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:
>>>>>>>>
>>>>>>>> - allow mrunit to have its own release cycle. This is, I think, the
>>>> most
>>>>>>>>>
>>>>>>>>
>>>>>>>> important.
>>>>>>>>>
>>>>>>>>
>>>>>>>> If you submit your work to Apache we can evaluate it for inclusion
>>> in
>>>> the
>>>>>>>> 0.20.100 branch to get your changes released in a timely manner.