Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> [DISCUSS] Spin out MR, HDFS and YARN as their own TLPs and disband Hadoop umbrella project


Copy link to this message
-
Re: [DISCUSS] Spin out MR, HDFS and YARN as their own TLPs and disband Hadoop umbrella project
Hi Eli,

On Aug 29, 2012, at 11:41 AM, Eli Collins wrote:

> Thanks for writing up a proposal Chris.

NP.

>
> I think it makes sense to have Common live in HDFS at least for now,
> since it's at the bottom of the stack / dependency chain and it's code
> is the most intertwined with common, and, per Arun, we tend to work on
> common stuff more than MR people. The HDFS project is really a lot
> more than HDFS, eg has all the hadoop commands, non-HDFS file system
> source, etc but that seems like an OK starting point. We need to
> figure out the committers and PMC though since the goal is to just
> have the HDFS community (vs the current Hadoop people) but the project
> will contain non-HDFS stuff. I'd like to hear from the current Hadoop
> committers and PMC members that mostly work on MR and YARN - are you
> guys OK losing your current privileges on the HDFS repo?

Rather than ask the former question that way, I would just simply put up
a list of proposed HDFS PMC folks (yes, I keep using PMC ^_^). Then,
iterate on that.

> Otherwise we
> haven't made much progress (ie HDFS still has multiple communities).

ACK.

>
> We also need to address the areas where it's not so cut and dry, eg
> where there is a single Hadoop project:
> - The Hadoop trademark, assume this lives in the HDFS project if Common does?

Apache owns the Hadoop trademark, and the PMC helps to enforce it. Projects
don't own trademarks.

> - The user community, eg the users lists that we *just* merged, shall
> we still keep one list?

That's a good question -- maybe ask users to opt-in. Yes, this is intrusive, but
I bet you'd find the real users of the specific projects if they have to resubscribe.
Just my 2c.

> - We should move the global stuff like "how to get started" docs to
> Bigtop, which can point to individual projects resources

Sounds cool to me.

> - Hadoop 1.x is is maintenance mode, though it still actively gets
> patches so we need to consider it. The surgery necessary to split v1
> Hadoop is probably not suitable for a sustaining release and not worth
> it at this point in the lifetime of this branch. I assume the HDFS
> project will then host the Hadoop 1.x branches?  This implies only
> members of the HDFS project can commit and release.

Why not put the 1.x stuff in Bigtop since it's global or whatever?

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB