Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - where do side-projects go in trunk now that contrib/ is gone?


Copy link to this message
-
Re: where do side-projects go in trunk now that contrib/ is gone?
Eli Collins 2013-02-12, 21:35
"extension code" is separate from "contrib". There's nothing
preventing extension code from being part of the project. Removing
contrib is about saying removing not-yet-baked code, or un-maintained
code, which causes problems with users (who don't know what contrib
is, ie they can't tell what's supposed to work and what doesn't).

IMO Hadoop extension code is often a better fit for either Apache
extras (apache-extras.org, the community of open source projects
related to Apache Software Foundation projects or based on their
technology) or the project that's being integrated with (often it will
be maintained by those developers, and therefore better to have it in
a repo where they can more easily monitor/commit). In some cases I
could see it living in the Hadoop source as well, but I don't think
there's one right (TM) place for extension code.

To answer your original question, there's already precedent for
extension code in the project, eg file system extensions like s3 live
in o.a.h.fs. The directory depends on the type of extension (eg 3rd
party codec integration lives in o.a.h.io.compress). Probably best to
discuss the particular case.

On Tue, Feb 12, 2013 at 12:55 AM, Steve Loughran
<[EMAIL PROTECTED]> wrote:
> On 11 February 2013 21:36, Eli Collins <[EMAIL PROTECTED]> wrote:
>
>> The idea of removing contrib was that this source would no longer go into
>> the Hadoop project. Where it goes is really up to the contributor. Some
>> people have created separate incubator projects (eg MRUnit), some people
>> are using Apache extras (hosted on Google), some could bake on github and
>> then get contributed to Hadoop when they're ready to be fully supported in
>> the code base (eg Hadoop auth and Httpfs).
>>
>>
> I understand, and I recognise a limit of contrib/ was its low maintenance
>
> However, what the "no contrib" policy is saying is "no apache community
> around any extension code". I can -and do- have stuff in github, but I'm
> left choreographing merging with others -Rackspace, Mirantis -and as well
> as having the merge grief that comes from not having a shared repository,
> we are effectively excluding everyone else from participating.
>
> similarly: google code hosting != ASF.
>
> When you consider that Hadoop was a spin-off from lucene, Ant a spin-off
> from Tomcat, I think xalan started off in a corner of Xerces, etc, forcing
> things out of the ASF stops this incremental growth, not until things are
> already at the stage where it's considered ready for incubation -which also
> implies that this will be a long-lived project with independence from
> everything else.
>
> I can certainly see the value in having independent projects -but making it
> the starting place for all contributions to the hadoop codebase is too high
> a barrier.
>
> we need something less formal that is still part of the ASF
>
> -steve