Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> [VOTE] - Establish YARN as a sub-project of Apache Hadoop


Copy link to this message
-
Re: [VOTE] - Establish YARN as a sub-project of Apache Hadoop
Hi Guys,

The existing discussion and conversation below is the precise reason that I suggested
Hadoop consider spinning out the rest of its *products* as *projects*. Folks have piped
up and listed technical reasons as the challenges behind this, and then responded
with clear community reasons either by their actions, or by other means.

Having distinct communities as indicated by distinct lists of committers isn't wrong -- it's
usually however exemplified by having a distinct Apache project. It sounds like those
folks that have been working on YARN for 1.5 years+ as stated would like to have their
own distinct Apache community.

Why not simply do that and stop beating around the bush? Sorry, just keeping it real
here. I don't buy the technical issue rationale either. You guys are among the smartest
people in the world. I think we can get around technical issues (e.g., replication of pieces
of code that are hard to pull apart in YARN rather than trying to solve complex dependencies,
whatever). I'm sure you can figure it out.

It really seems like YARN should establish a precedent for a better and IMHO more "Apache"
way to do this rather than to debate committer lists endlessly. And citing "this is the way we've
done it in Hadoop to date, blah blah" isn't a good enough reason to continue doing it. So what.
Change now :) Make it better now so you don't have to keep doing this when YARN-NEXT++
or whatever comes up.

Just my 2c as a member of the Foundation who has seen this before and who has seen it
fixed before (see TLP spinouts in Lucene; in Nutch; and even in your own community,
Hadoop).

Cheers,
Chris

On Aug 16, 2012, at 11:27 AM, Suresh Srinivas wrote:

> +1 for Arun's proposal. It seems reasonable to me.
>
> Here are my thoughts on this thread:
>
> Some have voiced opinion that they would not have supported YARN becoming a
> sub-project if they knew committer list is going to change. YARN was voted
> as a sub-project on the merit of it being a distinct functionality from
> rest of existing Hadoop. Your vote for YARN becoming a sub-project should
> have nothing to do with the commiter list.
>
> A team of people have worked for a long time (more than 1.5 years) on YARN.
> They understand the code and the design context well enough to be
> committers. I am not sure how having a long inclusive list of committers
> helps YARN. Do we expect the folks who have not contributed to YARN and
> might not understand YARN code well enough to do a good job as committers?
> That is the reason why I do not mind not being included in the list of
> committers. Further Arun has been more inclusive by incorporating Tom's
> feedback.
>
> We, as the Hadoop PMC, decided to vote committers to sub-projects instead
> of whole of Hadoop. To ensure someone who works heavily on one sub-project
> from committing code the core parts of other sub-projects. Why do we want
> to make an exception for YARN?
>
> People interested in becoming committers can always contribute to YARN and
> become committers anytime. It is not like the list is closed forever.
>
> Regards,
> Suresh
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB