Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> [DISCUSS] Move common, hdfs, mapreduce contrib components to apache-extras.org or elsewhere


Copy link to this message
-
Re: [DISCUSS] Move common, hdfs, mapreduce contrib components to apache-extras.org or elsewhere
On Fri, Feb 11, 2011 at 1:03 PM, Aaron Kimball <[EMAIL PROTECTED]> wrote:
> Tom,
>
> How do these contrib components get released then?

In source form - this is what ZooKeeper does.

> If the intent of having
> the code is to eventually produce release artifacts that people can use,
> then allowing them to further degrade in releasability seems antithetical to
> the point of keeping the source around. I think users who download Hadoop
> would be surprised to find that tools/projects under contrib do not operate
> as advertised.
>
> If the point of retaining the code would be instead to have it as an example
> for future developers to reference, then maybe it would be better to move
> them into an "attic" or "unmaintaned" directory so it is clear that we do
> not expect these to stay up to date. If we do expect them to work, and
> believe that they belong in the project, then I think we need to keep them
> building -- and yes, that adds to the burden associated with the release of
> Hadoop as a whole.

Maybe. We haven't always done a great job of this though.

>
> - Aaron
>
> On Fri, Feb 11, 2011 at 10:12 AM, Tom White <[EMAIL PROTECTED]> wrote:
>
>> For contrib components that we elect not to remove, I suggest that we
>> remove them from the main build, so that failures in a contrib
>> component don't hinder the main build and release. This is what
>> ZooKeeper does, for example.
>>
>> Tom
>>
>> On Fri, Feb 11, 2011 at 2:03 AM, Bernd Fondermann
>> <[EMAIL PROTECTED]> wrote:
>> > -1.
>> >
>> > Move it away from TRUNK so it doesn't affect builds is a much better
>> > (and simpler) solution. If someone wants to revive it, he can within
>> > the bounds of Apache Hadoop and will become a part of the Hadoop
>> > community, which would be good.
>> > If you'd move it off-site, if the code ever gets new contributors,
>> > it's hard to integrate them (code and contributors) into Hadoop again.
>> >
>> > AFAIUI, apache-extras is for placing non-Apache code closer to the
>> > related Apache projects, not for moving our code away from our own
>> > premises.
>> >
>> >  Bernd
>> >
>> > On Mon, Jan 31, 2011 at 04:42, Nigel Daley <[EMAIL PROTECTED]> wrote:
>> >> Folks,
>> >>
>> >> Now that http://apache-extras.org is launched (
>> https://blogs.apache.org/foundation/entry/the_apache_software_foundation_launches)
>> I'd like to start a discussion on moving contrib components out of common,
>> mapreduce, and hdfs.
>> >>
>> >> These contrib components complicate the builds, cause test failures that
>> nobody seems to care about, have releases that are tied to Hadoop's long
>> release cycles, etc.  Most folks I've talked with agree that these contrib
>> components would be better served by being pulled out of Hadoop and hosted
>> elsewhere. The new apache-extras code hosting site seems like a natural
>> *default* location for migrating these contrib projects.  Perhaps some
>> should graduate from contrib to src (ie from contrib to core of the project
>> they're included in).  If folks agree, we'll need to come up with a mapping
>> of contrib component to it's final destination and file a jira.
>> >>
>> >> Here are the contrib components by project (hopefully I didn't miss
>> any).
>> >>
>> >> Common Contrib:
>> >>  failmon
>> >>  hod
>> >>  test
>> >>
>> >>
>> >> MapReduce Contrib:
>> >>  capacity-scheduler -- move to MR core?
>> >>  data_join
>> >>  dynamic-scheduler
>> >>  eclipse-plugin
>> >>  fairscheduler -- move to MR core?
>> >>  gridmix
>> >>  index
>> >>  mrunit
>> >>  mumak
>> >>  raid
>> >>  sqoop
>> >>  streaming -- move to MR core?
>> >>  vaidya
>> >>  vertica
>> >>
>> >>
>> >> HDFS Contrib:
>> >>  fuse-dfs
>> >>  hdfsproxy
>> >>  thriftfs
>> >>
>> >>
>> >> Cheers,
>> >> Nige
>> >>
>> >
>>
>