Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> [DISCUSS] Spin out MR, HDFS and YARN as their own TLPs and disband Hadoop umbrella project


Copy link to this message
-
Re: [DISCUSS] Spin out MR, HDFS and YARN as their own TLPs and disband Hadoop umbrella project
On Wed, Aug 29, 2012 at 11:19PM, Mattmann, Chris A (388J) wrote:
> Hi Eli,
>
> On Aug 29, 2012, at 11:41 AM, Eli Collins wrote:
>
> > Thanks for writing up a proposal Chris.
>
> NP.
>
> >
> > I think it makes sense to have Common live in HDFS at least for now,
> > since it's at the bottom of the stack / dependency chain and it's code
> > is the most intertwined with common, and, per Arun, we tend to work on
> > common stuff more than MR people. The HDFS project is really a lot
> > more than HDFS, eg has all the hadoop commands, non-HDFS file system
> > source, etc but that seems like an OK starting point. We need to
> > figure out the committers and PMC though since the goal is to just
> > have the HDFS community (vs the current Hadoop people) but the project
> > will contain non-HDFS stuff. I'd like to hear from the current Hadoop
> > committers and PMC members that mostly work on MR and YARN - are you
> > guys OK losing your current privileges on the HDFS repo?
>
> Rather than ask the former question that way, I would just simply put up
> a list of proposed HDFS PMC folks (yes, I keep using PMC ^_^). Then,
> iterate on that.
>
> > Otherwise we
> > haven't made much progress (ie HDFS still has multiple communities).
>
> ACK.
>
> >
> > We also need to address the areas where it's not so cut and dry, eg
> > where there is a single Hadoop project:
> > - The Hadoop trademark, assume this lives in the HDFS project if Common does?
>
> Apache owns the Hadoop trademark, and the PMC helps to enforce it. Projects
> don't own trademarks.
>
> > - The user community, eg the users lists that we *just* merged, shall
> > we still keep one list?
>
> That's a good question -- maybe ask users to opt-in. Yes, this is intrusive, but
> I bet you'd find the real users of the specific projects if they have to resubscribe.
> Just my 2c.
>
> > - We should move the global stuff like "how to get started" docs to
> > Bigtop, which can point to individual projects resources
>
> Sounds cool to me.
>
> > - Hadoop 1.x is is maintenance mode, though it still actively gets
> > patches so we need to consider it. The surgery necessary to split v1
> > Hadoop is probably not suitable for a sustaining release and not worth
> > it at this point in the lifetime of this branch. I assume the HDFS
> > project will then host the Hadoop 1.x branches?  This implies only
> > members of the HDFS project can commit and release.
>
> Why not put the 1.x stuff in Bigtop since it's global or whatever?

Wearing my BigTop hat now, I encourage this audience to rush something like
this to BigTop. If I am reading you correctly, you are asking BigTop to host
1.x branches of Hadoop, aren't you? I don't see how it fits in there,
actually. But this is a separate issue that needs to involve BigTop community.

Cos

> Cheers,
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: [EMAIL PROTECTED]
> WWW:   http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>