As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.

Specifically,
# Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
# Create a separate YARN jira project and yarn-dev@ mailing-list.
# Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.

Please vote, the vote will run the normal 7 days. Obviously, I'm +1.

thanks,
Arun

+1. Non-binding.

Thanks,

Junping

----- Original Message -----
From: "Arun C Murthy" <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Friday, July 27, 2012 11:24:44 AM
Subject: [VOTE] - Establish YARN as a sub-project of Apache Hadoop

As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.

Specifically,
# Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
# Create a separate YARN jira project and yarn-dev@ mailing-list.
# Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.

Please vote, the vote will run the normal 7 days. Obviously, I'm +1.

thanks,
Arun

+1.

On Thu, Jul 26, 2012 at 8:24 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> As we discussed in the previous thread, I'd like to call a vote to
> establish YARN as a sub-project (it's the best *term* we have to describe
> it in our current nomenclature) of Apache Hadoop along with Common, HDFS &
> MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of
> hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN &
> MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>
>

+1

Thanks,

Todd

On Jul 26, 2012, at 8:24 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>

+1

-- Hitesh

On Jul 26, 2012, at 8:24 PM, Arun C Murthy wrote:

> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>

+1...

thanks
mahadev

On Thu, Jul 26, 2012 at 8:24 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>

+1

On Fri, Jul 27, 2012 at 12:53 PM, Mahadev Konar <[EMAIL PROTECTED]> wrote:
> +1...
>
> thanks
> mahadev
>
> On Thu, Jul 26, 2012 at 8:24 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>>
>> Specifically,
>> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
>> # Create a separate YARN jira project and yarn-dev@ mailing-list.
>> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>>
>> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>>
>> thanks,
>> Arun
>>

--
Best Regards, Edward J. Yoon
@eddieyoon

+1

Tsz Wo

________________________________
From: Edward J. Yoon <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Thursday, July 26, 2012 8:56 PM
Subject: Re: [VOTE] - Establish YARN as a sub-project of Apache Hadoop

+1

On Fri, Jul 27, 2012 at 12:53 PM, Mahadev Konar <[EMAIL PROTECTED]> wrote:
> +1...
>
> thanks
> mahadev
>
> On Thu, Jul 26, 2012 at 8:24 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>>
>> Specifically,
>> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
>> # Create a separate YARN jira project and yarn-dev@ mailing-list.
>> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>>
>> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>>
>> thanks,
>> Arun
>>

--
Best Regards, Edward J. Yoon
@eddieyoon

+1

On Thu, Jul 26, 2012 at 8:24 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> As we discussed in the previous thread, I'd like to call a vote to
> establish YARN as a sub-project (it's the best *term* we have to describe
> it in our current nomenclature) of Apache Hadoop along with Common, HDFS &
> MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of
> hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN &
> MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>
>
--
http://hortonworks.com/download/

+1

On Fri, Jul 27, 2012 at 2:26 PM, Suresh Srinivas <[EMAIL PROTECTED]>wrote:

> +1
>
> On Thu, Jul 26, 2012 at 8:24 PM, Arun C Murthy <[EMAIL PROTECTED]>
> wrote:
>
> > As we discussed in the previous thread, I'd like to call a vote to
> > establish YARN as a sub-project (it's the best *term* we have to describe
> > it in our current nomenclature) of Apache Hadoop along with Common, HDFS
> &
> > MapReduce.
> >
> > Specifically,
> > # Create a separate top-level src folder hadoop-yarn, move it out of
> > hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> > # Create a separate YARN jira project and yarn-dev@ mailing-list.
> > # Continue to co-release a single Hadoop release with Common, HDFS, YARN
> &
> > MapReduce.
> >
> > Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
> >
> > thanks,
> > Arun
> >
> >
>
>
> --
> http://hortonworks.com/download/
>

+1

On Fri, Jul 27, 2012 at 8:54 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>

--
Harsh J

+1

Aaron

On Jul 26, 2012, at 8:24 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>

+1

Tom

On Thu, Jul 26, 2012 at 11:24 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>

+1

+1

On Thu, Jul 26, 2012 at 10:24 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> As we discussed in the previous thread, I'd like to call a vote to
> establish YARN as a sub-project (it's the best *term* we have to describe
> it in our current nomenclature) of Apache Hadoop along with Common, HDFS &
> MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of
> hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN &
> MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>
>

+1

On Thu, Jul 26, 2012 at 8:24 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>

+1

Daryn
On Jul 26, 2012, at 10:24 PM, Arun C Murthy wrote:

> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>

+1

>
> On Jul 26, 2012, at 10:24 PM, Arun C Murthy wrote:
>
>> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>>
>> Specifically,
>> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
>> # Create a separate YARN jira project and yarn-dev@ mailing-list.
>> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>>
>> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>>
>> thanks,
>> Arun
>>
>

+1

________________________________________
From: Devaraj Das [[EMAIL PROTECTED]]
Sent: Wednesday, August 01, 2012 3:16 AM
To: [EMAIL PROTECTED]
Subject: Re: [VOTE] - Establish YARN as a sub-project of Apache Hadoop

+1

>
> On Jul 26, 2012, at 10:24 PM, Arun C Murthy wrote:
>
>> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>>
>> Specifically,
>> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
>> # Create a separate YARN jira project and yarn-dev@ mailing-list.
>> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>>
>> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>>
>> thanks,
>> Arun
>>
>

+1

On 8/1/12, Uma Maheswara Rao G <[EMAIL PROTECTED]> wrote:
> +1
>
> ________________________________________
> From: Devaraj Das [[EMAIL PROTECTED]]
> Sent: Wednesday, August 01, 2012 3:16 AM
> To: [EMAIL PROTECTED]
> Subject: Re: [VOTE] - Establish YARN as a sub-project of Apache Hadoop
>
> +1
>
>>
>> On Jul 26, 2012, at 10:24 PM, Arun C Murthy wrote:
>>
>>> As we discussed in the previous thread, I'd like to call a vote to
>>> establish YARN as a sub-project (it's the best *term* we have to describe
>>> it in our current nomenclature) of Apache Hadoop along with Common, HDFS
>>> & MapReduce.
>>>
>>> Specifically,
>>> # Create a separate top-level src folder hadoop-yarn, move it out of
>>> hadoop-mapreduce-project in subversion at hadoop/common/trunk.
>>> # Create a separate YARN jira project and yarn-dev@ mailing-list.
>>> # Continue to co-release a single Hadoop release with Common, HDFS, YARN
>>> & MapReduce.
>>>
>>> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>>>
>>> thanks,
>>> Arun
>>>
>>

--
Sent from my mobile device

With 22 +1s (9 binding) and no -1s the vote passes unanimously. Thanks to all who voted.

I'll start the necessary process to move on this.

thanks,
Arun

On Jul 26, 2012, at 8:24 PM, Arun C Murthy wrote:

> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

Ok, we are nearly there.

# The new YARN jira is functional now (albeit with some teething issues with notifications/workflow which should be resolved soon): https://issues.apache.org/jira/browse/YARN
# I've committed YARN-1 to move YARN out of hadoop-mapreduce-project onto hadoop-yarn-project in subversion.
# The yarn-dev@ mailing list is active.
# All jira notifications are going to yarn-issues@ and all subversion commit notifications are going to yarn-commits@.

Please do subscribe to the relevant mailing lists.
Also note that there is *no* yarn-user@ list since all user interactions are now via [EMAIL PROTECTED].

If you see issues, please let me know and I'll try to respond ASAP.

thanks,
Arun

On Aug 3, 2012, at 12:51 AM, Arun C Murthy wrote:

> With 22 +1s (9 binding) and no -1s the vote passes unanimously. Thanks to all who voted.
>
> I'll start the necessary process to move on this.
>
> thanks,
> Arun
>
> On Jul 26, 2012, at 8:24 PM, Arun C Murthy wrote:
>
>> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>>
>> Specifically,
>> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
>> # Create a separate YARN jira project and yarn-dev@ mailing-list.
>> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>>
>> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>>
>> thanks,
>> Arun
>>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

On Wed, Aug 8, 2012 at 3:29 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>
> Ok, we are nearly there.
>
> # The new YARN jira is functional now (albeit with some teething issues
> with notifications/workflow which should be resolved soon):
> https://issues.apache.org/jira/browse/YARN
> # I've committed YARN-1 to move YARN out of hadoop-mapreduce-project onto
> hadoop-yarn-project in subversion.
> # The yarn-dev@ mailing list is active.
> # All jira notifications are going to yarn-issues@ and all subversion
> commit notifications are going to yarn-commits@.

Thanks for setting this up Arun. The committer list needs to be set up
by editing https://svn.apache.org/repos/infra/infrastructure/trunk/subversion/authorization/asf-authorization-template.
It should have the initial contents of the MR committer list. When we
split the committer lists for Hadoop into Common, HDFS, MR we made
them all start out with the Hadoop list, so we should do the same for
splitting YARN out of MR.

Cheers,
Tom

>
> Please do subscribe to the relevant mailing lists.
> Also note that there is *no* yarn-user@ list since all user interactions
> are now via [EMAIL PROTECTED].
>
> If you see issues, please let me know and I'll try to respond ASAP.
>
> thanks,
> Arun
>
> On Aug 3, 2012, at 12:51 AM, Arun C Murthy wrote:
>
> > With 22 +1s (9 binding) and no -1s the vote passes unanimously. Thanks
> > to all who voted.
> >
> > I'll start the necessary process to move on this.
> >
> > thanks,
> > Arun
> >
> > On Jul 26, 2012, at 8:24 PM, Arun C Murthy wrote:
> >
> >> As we discussed in the previous thread, I'd like to call a vote to
> >> establish YARN as a sub-project (it's the best *term* we have to describe it
> >> in our current nomenclature) of Apache Hadoop along with Common, HDFS &
> >> MapReduce.
> >>
> >> Specifically,
> >> # Create a separate top-level src folder hadoop-yarn, move it out of
> >> hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> >> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> >> # Continue to co-release a single Hadoop release with Common, HDFS,
> >> YARN & MapReduce.
> >>
> >> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
> >>
> >> thanks,
> >> Arun
> >>
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>

Tom,

On Aug 13, 2012, at 2:01 PM, Tom White wrote:
> The committer list needs to be set up
> by editing https://svn.apache.org/repos/infra/infrastructure/trunk/subversion/authorization/asf-authorization-template.

Good point, thanks for pointing it out.

> It should have the initial contents of the MR committer list. When we split the committer lists for Hadoop into Common, HDFS, MR we made
> them all start out with the Hadoop list, so we should do the same for splitting YARN out of MR.
For a while now, we maintain distinct committer lists for HDFS & MapReduce and nominations for new committers are based on their contributions to the individual sub-project.

Following that I would like to propose the following.

Given that:
# YARN is a substantial (~100K LoC) and new body of work which has been around for nearly 18 months now.
# It is also very distinct from Hadoop MapReduce itself, as discussed in this thread, and wasn't derived from any of the existing Hadoop sub-projects.
I propose we define the list as the committers who have been active and have made significant contributions to the project.

A quick review of subversion leads me to suggest the following list (in lexical order).
If there is anyone who has contributed significantly to YARN and is not on that list, please accept my apologies in advance and let me know, I'll add you in.

* Alejandro Abdelnur
* Arun C. Murthy
* Chris Douglas
* Hitesh Shah
* Jonathan Eagles
* Luke Lu
* Mahadev Konar
* Robert Evans
* Siddharth Seth
* Sharad Agarwal
* Thomas Graves
* Vinod Kumar V

This approach seems consistent with what we have done historically with other new sub-projects such as Hive, Pig, HBase, ZooKeeper, etc. and with the Apache principle of Meritocracy.
Also, since we're starting with folks from 6 companies we already meet the Incubator criteria for diversity, which seems healthy.  

Obviously, additional contributors will be encouraged to join the sub-project via the normal mechanisms.  

Last, not least, most people on the MapReduce committer list are already part of the Hadoop PMC and hence they have svn rights if necessary - this doesn't change that and we continue to trust them.

Thoughts?

thanks,
Arun

I personally am +1 for only pulling over committers from MR that have been
active on YARN.  Like Arun said YARN is new code and not everything in MR
applies directly to YARN or vise versa.

--Bobby

On 8/14/12 1:19 PM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:

>Tom,
>
>On Aug 13, 2012, at 2:01 PM, Tom White wrote:
>> The committer list needs to be set up
>> by editing
>>https://svn.apache.org/repos/infra/infrastructure/trunk/subversion/author
>>ization/asf-authorization-template.
>
>Good point, thanks for pointing it out.
>
>> It should have the initial contents of the MR committer list. When we
>>split the committer lists for Hadoop into Common, HDFS, MR we made
>> them all start out with the Hadoop list, so we should do the same for
>>splitting YARN out of MR.
>
>
>For a while now, we maintain distinct committer lists for HDFS &
>MapReduce and nominations for new committers are based on their
>contributions to the individual sub-project.
>
>Following that I would like to propose the following.
>
>Given that:
># YARN is a substantial (~100K LoC) and new body of work which has been
>around for nearly 18 months now.
># It is also very distinct from Hadoop MapReduce itself, as discussed in
>this thread, and wasn't derived from any of the existing Hadoop
>sub-projects.
>I propose we define the list as the committers who have been active and
>have made significant contributions to the project.
>
>A quick review of subversion leads me to suggest the following list (in
>lexical order).
>If there is anyone who has contributed significantly to YARN and is not
>on that list, please accept my apologies in advance and let me know, I'll
>add you in.
>
>* Alejandro Abdelnur
>* Arun C. Murthy
>* Chris Douglas
>* Hitesh Shah
>* Jonathan Eagles
>* Luke Lu
>* Mahadev Konar
>* Robert Evans
>* Siddharth Seth
>* Sharad Agarwal
>* Thomas Graves
>* Vinod Kumar V
>
>This approach seems consistent with what we have done historically with
>other new sub-projects such as Hive, Pig, HBase, ZooKeeper, etc. and with
>the Apache principle of Meritocracy.
>Also, since we're starting with folks from 6 companies we already meet
>the Incubator criteria for diversity, which seems healthy.
>
>Obviously, additional contributors will be encouraged to join the
>sub-project via the normal mechanisms.
>
>Last, not least, most people on the MapReduce committer list are already
>part of the Hadoop PMC and hence they have svn rights if necessary - this
>doesn't change that and we continue to trust them.
>
>Thoughts?
>
>thanks,
>Arun
>

I think we need to be careful not to exclude people. All existing MR
committers have been trusted to commit to YARN so they should continue
to be trusted to commit to it. The simplest way would be to include
all existing MR committers. Alternatively, if it's generally
considered important to start with active committers, add all MR
committers who have previously committed to the YARN tree.

Cheers,
Tom

On Tue, Aug 14, 2012 at 2:27 PM, Robert Evans <[EMAIL PROTECTED]> wrote:
> I personally am +1 for only pulling over committers from MR that have been
> active on YARN.  Like Arun said YARN is new code and not everything in MR
> applies directly to YARN or vise versa.
>
> --Bobby
>
> On 8/14/12 1:19 PM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:
>
>>Tom,
>>
>>On Aug 13, 2012, at 2:01 PM, Tom White wrote:
>>> The committer list needs to be set up
>>> by editing
>>>https://svn.apache.org/repos/infra/infrastructure/trunk/subversion/author
>>>ization/asf-authorization-template.
>>
>>Good point, thanks for pointing it out.
>>
>>> It should have the initial contents of the MR committer list. When we
>>>split the committer lists for Hadoop into Common, HDFS, MR we made
>>> them all start out with the Hadoop list, so we should do the same for
>>>splitting YARN out of MR.
>>
>>
>>For a while now, we maintain distinct committer lists for HDFS &
>>MapReduce and nominations for new committers are based on their
>>contributions to the individual sub-project.
>>
>>Following that I would like to propose the following.
>>
>>Given that:
>># YARN is a substantial (~100K LoC) and new body of work which has been
>>around for nearly 18 months now.
>># It is also very distinct from Hadoop MapReduce itself, as discussed in
>>this thread, and wasn't derived from any of the existing Hadoop
>>sub-projects.
>>I propose we define the list as the committers who have been active and
>>have made significant contributions to the project.
>>
>>A quick review of subversion leads me to suggest the following list (in
>>lexical order).
>>If there is anyone who has contributed significantly to YARN and is not
>>on that list, please accept my apologies in advance and let me know, I'll
>>add you in.
>>
>>* Alejandro Abdelnur
>>* Arun C. Murthy
>>* Chris Douglas
>>* Hitesh Shah
>>* Jonathan Eagles
>>* Luke Lu
>>* Mahadev Konar
>>* Robert Evans
>>* Siddharth Seth
>>* Sharad Agarwal
>>* Thomas Graves
>>* Vinod Kumar V
>>
>>This approach seems consistent with what we have done historically with
>>other new sub-projects such as Hive, Pig, HBase, ZooKeeper, etc. and with
>>the Apache principle of Meritocracy.
>>Also, since we're starting with folks from 6 companies we already meet
>>the Incubator criteria for diversity, which seems healthy.
>>
>>Obviously, additional contributors will be encouraged to join the
>>sub-project via the normal mechanisms.
>>
>>Last, not least, most people on the MapReduce committer list are already
>>part of the Hadoop PMC and hence they have svn rights if necessary - this
>>doesn't change that and we continue to trust them.
>>
>>Thoughts?
>>
>>thanks,
>>Arun
>>
>

Tom,

On Aug 14, 2012, at 5:44 PM, Tom White wrote:

> Alternatively, if it's generally
> considered important to start with active committers, add all MR
> committers who have previously committed to the YARN tree.

Good point. As I said, my personal preference is different; but if you feel strongly I'm happy to get to consensus and preempt a long discussion.

So, here is the list of folks who have either contributed or reviewed-and-committed (a handful of) YARN patches in the last 18 months:

* Alejandro Abdelnur
* Arun C. Murthy
* Chris Douglas
* Devaraj Das
* Hitesh Shah
* Jonathan Eagles
* Luke Lu
* Mahadev Konar
* Robert Evans
* Siddharth Seth
* Sharad Agarwal
* Thomas Graves
* Tom White
* Vinod Kumar V

Hope that it reasonable and we can get back to our caves to crank out code. *smile*

thanks,
Arun

> Cheers,
> Tom
>
> On Tue, Aug 14, 2012 at 2:27 PM, Robert Evans <[EMAIL PROTECTED]> wrote:
>> I personally am +1 for only pulling over committers from MR that have been
>> active on YARN.  Like Arun said YARN is new code and not everything in MR
>> applies directly to YARN or vise versa.
>>
>> --Bobby
>>
>> On 8/14/12 1:19 PM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:
>>
>>> Tom,
>>>
>>> On Aug 13, 2012, at 2:01 PM, Tom White wrote:
>>>> The committer list needs to be set up
>>>> by editing
>>>> https://svn.apache.org/repos/infra/infrastructure/trunk/subversion/author
>>>> ization/asf-authorization-template.
>>>
>>> Good point, thanks for pointing it out.
>>>
>>>> It should have the initial contents of the MR committer list. When we
>>>> split the committer lists for Hadoop into Common, HDFS, MR we made
>>>> them all start out with the Hadoop list, so we should do the same for
>>>> splitting YARN out of MR.
>>>
>>>
>>> For a while now, we maintain distinct committer lists for HDFS &
>>> MapReduce and nominations for new committers are based on their
>>> contributions to the individual sub-project.
>>>
>>> Following that I would like to propose the following.
>>>
>>> Given that:
>>> # YARN is a substantial (~100K LoC) and new body of work which has been
>>> around for nearly 18 months now.
>>> # It is also very distinct from Hadoop MapReduce itself, as discussed in
>>> this thread, and wasn't derived from any of the existing Hadoop
>>> sub-projects.
>>> I propose we define the list as the committers who have been active and
>>> have made significant contributions to the project.
>>>
>>> A quick review of subversion leads me to suggest the following list (in
>>> lexical order).
>>> If there is anyone who has contributed significantly to YARN and is not
>>> on that list, please accept my apologies in advance and let me know, I'll
>>> add you in.
>>>
>>> * Alejandro Abdelnur
>>> * Arun C. Murthy
>>> * Chris Douglas
>>> * Hitesh Shah
>>> * Jonathan Eagles
>>> * Luke Lu
>>> * Mahadev Konar
>>> * Robert Evans
>>> * Siddharth Seth
>>> * Sharad Agarwal
>>> * Thomas Graves
>>> * Vinod Kumar V
>>>
>>> This approach seems consistent with what we have done historically with
>>> other new sub-projects such as Hive, Pig, HBase, ZooKeeper, etc. and with
>>> the Apache principle of Meritocracy.
>>> Also, since we're starting with folks from 6 companies we already meet
>>> the Incubator criteria for diversity, which seems healthy.
>>>
>>> Obviously, additional contributors will be encouraged to join the
>>> sub-project via the normal mechanisms.
>>>
>>> Last, not least, most people on the MapReduce committer list are already
>>> part of the Hadoop PMC and hence they have svn rights if necessary - this
>>> doesn't change that and we continue to trust them.
>>>
>>> Thoughts?
>>>
>>> thanks,
>>> Arun
>>>
>>

Arun,

Creating a list of people without specifying the precise criteria is
problematic (particularly since the criteria weren't discussed before
the vote), and it's awkward for those folks who have committed code to
YARN in the past but are not on the list to have to ask to be
included. It's far better to be inclusive and welcoming, and in
practice it isn't a problem at all if we are "overly" inclusive in
adding MR committers since folks are aware of their limitations and
commit appropriately as ATM pointed out.

I think it would be simpler to add all the MR committers and deal with
emeritus questions separately, e.g. at the point that the subprojects
become TLPs.

Thanks,
Tom

On Wed, Aug 15, 2012 at 12:52 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> Tom,
>
> On Aug 14, 2012, at 5:44 PM, Tom White wrote:
>
>> Alternatively, if it's generally
>> considered important to start with active committers, add all MR
>> committers who have previously committed to the YARN tree.
>
> Good point. As I said, my personal preference is different; but if you feel strongly I'm happy to get to consensus and preempt a long discussion.
>
> So, here is the list of folks who have either contributed or reviewed-and-committed (a handful of) YARN patches in the last 18 months:
>
> * Alejandro Abdelnur
> * Arun C. Murthy
> * Chris Douglas
> * Devaraj Das
> * Hitesh Shah
> * Jonathan Eagles
> * Luke Lu
> * Mahadev Konar
> * Robert Evans
> * Siddharth Seth
> * Sharad Agarwal
> * Thomas Graves
> * Tom White
> * Vinod Kumar V
>
> Hope that it reasonable and we can get back to our caves to crank out code. *smile*
>
> thanks,
> Arun
>
>> Cheers,
>> Tom
>>
>> On Tue, Aug 14, 2012 at 2:27 PM, Robert Evans <[EMAIL PROTECTED]> wrote:
>>> I personally am +1 for only pulling over committers from MR that have been
>>> active on YARN.  Like Arun said YARN is new code and not everything in MR
>>> applies directly to YARN or vise versa.
>>>
>>> --Bobby
>>>
>>> On 8/14/12 1:19 PM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:
>>>
>>>> Tom,
>>>>
>>>> On Aug 13, 2012, at 2:01 PM, Tom White wrote:
>>>>> The committer list needs to be set up
>>>>> by editing
>>>>> https://svn.apache.org/repos/infra/infrastructure/trunk/subversion/author
>>>>> ization/asf-authorization-template.
>>>>
>>>> Good point, thanks for pointing it out.
>>>>
>>>>> It should have the initial contents of the MR committer list. When we
>>>>> split the committer lists for Hadoop into Common, HDFS, MR we made
>>>>> them all start out with the Hadoop list, so we should do the same for
>>>>> splitting YARN out of MR.
>>>>
>>>>
>>>> For a while now, we maintain distinct committer lists for HDFS &
>>>> MapReduce and nominations for new committers are based on their
>>>> contributions to the individual sub-project.
>>>>
>>>> Following that I would like to propose the following.
>>>>
>>>> Given that:
>>>> # YARN is a substantial (~100K LoC) and new body of work which has been
>>>> around for nearly 18 months now.
>>>> # It is also very distinct from Hadoop MapReduce itself, as discussed in
>>>> this thread, and wasn't derived from any of the existing Hadoop
>>>> sub-projects.
>>>> I propose we define the list as the committers who have been active and
>>>> have made significant contributions to the project.
>>>>
>>>> A quick review of subversion leads me to suggest the following list (in
>>>> lexical order).
>>>> If there is anyone who has contributed significantly to YARN and is not
>>>> on that list, please accept my apologies in advance and let me know, I'll
>>>> add you in.
>>>>
>>>> * Alejandro Abdelnur
>>>> * Arun C. Murthy
>>>> * Chris Douglas
>>>> * Hitesh Shah
>>>> * Jonathan Eagles
>>>> * Luke Lu
>>>> * Mahadev Konar
>>>> * Robert Evans
>>>> * Siddharth Seth
>>>> * Sharad Agarwal
>>>> * Thomas Graves
>>>> * Vinod Kumar V
>>>>
>>>> This approach seems consistent with what we have done historically with
>>>> other new sub-projects such as Hive, Pig, HBase, ZooKeeper, etc. and with
>>>> the Apache principle of Meritocracy.

Agree with Tom.  No reason moving the yarn directory up one level in
the repo means we need to revisit all this now. Let's get back to work
and revisit this at TLP time.

On Wed, Aug 15, 2012 at 2:24 PM, Tom White <[EMAIL PROTECTED]> wrote:
> Arun,
>
> Creating a list of people without specifying the precise criteria is
> problematic (particularly since the criteria weren't discussed before
> the vote), and it's awkward for those folks who have committed code to
> YARN in the past but are not on the list to have to ask to be
> included. It's far better to be inclusive and welcoming, and in
> practice it isn't a problem at all if we are "overly" inclusive in
> adding MR committers since folks are aware of their limitations and
> commit appropriately as ATM pointed out.
>
> I think it would be simpler to add all the MR committers and deal with
> emeritus questions separately, e.g. at the point that the subprojects
> become TLPs.
>
> Thanks,
> Tom
>
> On Wed, Aug 15, 2012 at 12:52 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>> Tom,
>>
>> On Aug 14, 2012, at 5:44 PM, Tom White wrote:
>>
>>> Alternatively, if it's generally
>>> considered important to start with active committers, add all MR
>>> committers who have previously committed to the YARN tree.
>>
>> Good point. As I said, my personal preference is different; but if you feel strongly I'm happy to get to consensus and preempt a long discussion.
>>
>> So, here is the list of folks who have either contributed or reviewed-and-committed (a handful of) YARN patches in the last 18 months:
>>
>> * Alejandro Abdelnur
>> * Arun C. Murthy
>> * Chris Douglas
>> * Devaraj Das
>> * Hitesh Shah
>> * Jonathan Eagles
>> * Luke Lu
>> * Mahadev Konar
>> * Robert Evans
>> * Siddharth Seth
>> * Sharad Agarwal
>> * Thomas Graves
>> * Tom White
>> * Vinod Kumar V
>>
>> Hope that it reasonable and we can get back to our caves to crank out code. *smile*
>>
>> thanks,
>> Arun
>>
>>> Cheers,
>>> Tom
>>>
>>> On Tue, Aug 14, 2012 at 2:27 PM, Robert Evans <[EMAIL PROTECTED]> wrote:
>>>> I personally am +1 for only pulling over committers from MR that have been
>>>> active on YARN.  Like Arun said YARN is new code and not everything in MR
>>>> applies directly to YARN or vise versa.
>>>>
>>>> --Bobby
>>>>
>>>> On 8/14/12 1:19 PM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> Tom,
>>>>>
>>>>> On Aug 13, 2012, at 2:01 PM, Tom White wrote:
>>>>>> The committer list needs to be set up
>>>>>> by editing
>>>>>> https://svn.apache.org/repos/infra/infrastructure/trunk/subversion/author
>>>>>> ization/asf-authorization-template.
>>>>>
>>>>> Good point, thanks for pointing it out.
>>>>>
>>>>>> It should have the initial contents of the MR committer list. When we
>>>>>> split the committer lists for Hadoop into Common, HDFS, MR we made
>>>>>> them all start out with the Hadoop list, so we should do the same for
>>>>>> splitting YARN out of MR.
>>>>>
>>>>>
>>>>> For a while now, we maintain distinct committer lists for HDFS &
>>>>> MapReduce and nominations for new committers are based on their
>>>>> contributions to the individual sub-project.
>>>>>
>>>>> Following that I would like to propose the following.
>>>>>
>>>>> Given that:
>>>>> # YARN is a substantial (~100K LoC) and new body of work which has been
>>>>> around for nearly 18 months now.
>>>>> # It is also very distinct from Hadoop MapReduce itself, as discussed in
>>>>> this thread, and wasn't derived from any of the existing Hadoop
>>>>> sub-projects.
>>>>> I propose we define the list as the committers who have been active and
>>>>> have made significant contributions to the project.
>>>>>
>>>>> A quick review of subversion leads me to suggest the following list (in
>>>>> lexical order).
>>>>> If there is anyone who has contributed significantly to YARN and is not
>>>>> on that list, please accept my apologies in advance and let me know, I'll
>>>>> add you in.
>>>>>
>>>>> * Alejandro Abdelnur
>>>>> * Arun C. Murthy
>>>>> * Chris Douglas
>

+1 for Arun's proposal. It seems reasonable to me.

Here are my thoughts on this thread:

Some have voiced opinion that they would not have supported YARN becoming a
sub-project if they knew committer list is going to change. YARN was voted
as a sub-project on the merit of it being a distinct functionality from
rest of existing Hadoop. Your vote for YARN becoming a sub-project should
have nothing to do with the commiter list.

A team of people have worked for a long time (more than 1.5 years) on YARN.
They understand the code and the design context well enough to be
committers. I am not sure how having a long inclusive list of committers
helps YARN. Do we expect the folks who have not contributed to YARN and
might not understand YARN code well enough to do a good job as committers?
That is the reason why I do not mind not being included in the list of
committers. Further Arun has been more inclusive by incorporating Tom's
feedback.

We, as the Hadoop PMC, decided to vote committers to sub-projects instead
of whole of Hadoop. To ensure someone who works heavily on one sub-project
from committing code the core parts of other sub-projects. Why do we want
to make an exception for YARN?

People interested in becoming committers can always contribute to YARN and
become committers anytime. It is not like the list is closed forever.

Regards,
Suresh

On Thu, Aug 16, 2012 at 11:27 AM, Suresh Srinivas <[EMAIL PROTECTED]>wrote:

> Some have voiced opinion that they would not have supported YARN becoming a
> sub-project if they knew committer list is going to change. YARN was voted
> as a sub-project on the merit of it being a distinct functionality from
> rest of existing Hadoop. Your vote for YARN becoming a sub-project should
> have nothing to do with the commiter list.
>

Take a look at the original DISCUSS thread on this subject. In it, Arun
attempted to enumerate all of the changes that this proposal would include.
Nowhere in it did it say that the committer list of YARN would be pared
down in an exclusive fashion. Given that this is an issue for some, I think
it should have been mentioned.
>
> A team of people have worked for a long time (more than 1.5 years) on YARN.
> They understand the code and the design context well enough to be
> committers. I am not sure how having a long inclusive list of committers
> helps YARN. Do we expect the folks who have not contributed to YARN and
> might not understand YARN code well enough to do a good job as committers?
> That is the reason why I do not mind not being included in the list of
> committers. Further Arun has been more inclusive by incorporating Tom's
> feedback.
>

How will having an inclusive list of grandfathered-in MR committers hurt
YARN? The only reason I've heard so far is the concern that someone may
make changes to an area of the codebase that they're not very familiar
with. In my experience, this has never been a problem in the Hadoop
community, so I don't understand why we're trying to account for this
hypothetical problematic situation.
>
> We, as the Hadoop PMC, decided to vote committers to sub-projects instead
> of whole of Hadoop. To ensure someone who works heavily on one sub-project
> from committing code the core parts of other sub-projects. Why do we want
> to make an exception for YARN?
>

No one is suggesting that the committer lists should be permanently the
same. The only question is the composition of the initial YARN committer
list. When we split the project into Common, HDFS, and MR, we grandfathered
in all existing committers to be committers on those subprojects. I don't
know why we should make an exception for YARN in this respect.

I think Eli summed up the issue very well: "No reason moving the yarn
directory up one level in the repo means we need to revisit all this now.
Let's get back to work and revisit this at TLP time."

--
Aaron T. Myers
Software Engineer, Cloudera

Hi Guys,

The existing discussion and conversation below is the precise reason that I suggested
Hadoop consider spinning out the rest of its *products* as *projects*. Folks have piped
up and listed technical reasons as the challenges behind this, and then responded
with clear community reasons either by their actions, or by other means.

Having distinct communities as indicated by distinct lists of committers isn't wrong -- it's
usually however exemplified by having a distinct Apache project. It sounds like those
folks that have been working on YARN for 1.5 years+ as stated would like to have their
own distinct Apache community.

Why not simply do that and stop beating around the bush? Sorry, just keeping it real
here. I don't buy the technical issue rationale either. You guys are among the smartest
people in the world. I think we can get around technical issues (e.g., replication of pieces
of code that are hard to pull apart in YARN rather than trying to solve complex dependencies,
whatever). I'm sure you can figure it out.

It really seems like YARN should establish a precedent for a better and IMHO more "Apache"
way to do this rather than to debate committer lists endlessly. And citing "this is the way we've
done it in Hadoop to date, blah blah" isn't a good enough reason to continue doing it. So what.
Change now :) Make it better now so you don't have to keep doing this when YARN-NEXT++
or whatever comes up.

Just my 2c as a member of the Foundation who has seen this before and who has seen it
fixed before (see TLP spinouts in Lucene; in Nutch; and even in your own community,
Hadoop).

Cheers,
Chris

On Aug 16, 2012, at 11:27 AM, Suresh Srinivas wrote:

> +1 for Arun's proposal. It seems reasonable to me.
>
> Here are my thoughts on this thread:
>
> Some have voiced opinion that they would not have supported YARN becoming a
> sub-project if they knew committer list is going to change. YARN was voted
> as a sub-project on the merit of it being a distinct functionality from
> rest of existing Hadoop. Your vote for YARN becoming a sub-project should
> have nothing to do with the commiter list.
>
> A team of people have worked for a long time (more than 1.5 years) on YARN.
> They understand the code and the design context well enough to be
> committers. I am not sure how having a long inclusive list of committers
> helps YARN. Do we expect the folks who have not contributed to YARN and
> might not understand YARN code well enough to do a good job as committers?
> That is the reason why I do not mind not being included in the list of
> committers. Further Arun has been more inclusive by incorporating Tom's
> feedback.
>
> We, as the Hadoop PMC, decided to vote committers to sub-projects instead
> of whole of Hadoop. To ensure someone who works heavily on one sub-project
> from committing code the core parts of other sub-projects. Why do we want
> to make an exception for YARN?
>
> People interested in becoming committers can always contribute to YARN and
> become committers anytime. It is not like the list is closed forever.
>
> Regards,
> Suresh
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

On Thu, Aug 16, 2012 at 12:59 PM, Mattmann, Chris A (388J)
<[EMAIL PROTECTED]> wrote:
> Hi Guys,
>
> The existing discussion and conversation below is the precise reason that I suggested
> Hadoop consider spinning out the rest of its *products* as *projects*. Folks have piped
> up and listed technical reasons as the challenges behind this, and then responded
> with clear community reasons either by their actions, or by other means.
>
> Having distinct communities as indicated by distinct lists of committers isn't wrong -- it's
> usually however exemplified by having a distinct Apache project. It sounds like those
> folks that have been working on YARN for 1.5 years+ as stated would like to have their
> own distinct Apache community.

Not sure that's the case, eg I think we all want to keep yarn in the
same code repository, allow patches that update it along with other
hadoop subprojects, co-design it with MR, test/release it together as
well.  That's not a sign of a distinct community.

I agree though, if people want a distinct community, then making it a
TLP is the right approach.

Thanks,
Eli

Hi Eli,

On Aug 16, 2012, at 1:08 PM, Eli Collins wrote:

> On Thu, Aug 16, 2012 at 12:59 PM, Mattmann, Chris A (388J)
> <[EMAIL PROTECTED]> wrote:
>> Hi Guys,
>>
>> The existing discussion and conversation below is the precise reason that I suggested
>> Hadoop consider spinning out the rest of its *products* as *projects*. Folks have piped
>> up and listed technical reasons as the challenges behind this, and then responded
>> with clear community reasons either by their actions, or by other means.
>>
>> Having distinct communities as indicated by distinct lists of committers isn't wrong -- it's
>> usually however exemplified by having a distinct Apache project. It sounds like those
>> folks that have been working on YARN for 1.5 years+ as stated would like to have their
>> own distinct Apache community.
>
> Not sure that's the case, eg I think we all want to keep yarn in the
> same code repository, allow patches that update it along with other
> hadoop subprojects, co-design it with MR, test/release it together as
> well.  That's not a sign of a distinct community.

Keeping code in the same repository with a PMC with different sets of
permissions in that repository *is* the sign of a distinct community.
Doesn't matter if you want the code there together, and allowing patches,
and testing and releasing and whatever. Those are technical issues.
Having code with *different* rules for *the same* community members
is not what I know as "community over code" and the Apache way.
And ultimately it's the reason why this email thread won't die. There's
an elephant in the room here (pun intended).

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

On Thu, Aug 16, 2012 at 1:11 PM, Mattmann, Chris A (388J)
<[EMAIL PROTECTED]> wrote:
> Hi Eli,
>
> On Aug 16, 2012, at 1:08 PM, Eli Collins wrote:
>
>> On Thu, Aug 16, 2012 at 12:59 PM, Mattmann, Chris A (388J)
>> <[EMAIL PROTECTED]> wrote:
>>> Hi Guys,
>>>
>>> The existing discussion and conversation below is the precise reason that I suggested
>>> Hadoop consider spinning out the rest of its *products* as *projects*. Folks have piped
>>> up and listed technical reasons as the challenges behind this, and then responded
>>> with clear community reasons either by their actions, or by other means.
>>>
>>> Having distinct communities as indicated by distinct lists of committers isn't wrong -- it's
>>> usually however exemplified by having a distinct Apache project. It sounds like those
>>> folks that have been working on YARN for 1.5 years+ as stated would like to have their
>>> own distinct Apache community.
>>
>> Not sure that's the case, eg I think we all want to keep yarn in the
>> same code repository, allow patches that update it along with other
>> hadoop subprojects, co-design it with MR, test/release it together as
>> well.  That's not a sign of a distinct community.
>
> Keeping code in the same repository with a PMC with different sets of
> permissions in that repository *is* the sign of a distinct community.
> Doesn't matter if you want the code there together, and allowing patches,
> and testing and releasing and whatever. Those are technical issues.
> Having code with *different* rules for *the same* community members
> is not what I know as "community over code" and the Apache way.
> And ultimately it's the reason why this email thread won't die. There's
> an elephant in the room here (pun intended).

For the PMC the permissions are the same across all the subprojects
and it is the same community. The question here is around committers.

We already have a working model where the various subprojects have
their own committers and people are trusted to commit across project
boundaries when it makes sense. I think everyone is in agreement that
yarn, like the other subprojects, will have it's own set of
committers, the only open question here as I understand it is whether
or not we grandfather in the existing MR committers (since that's
where YARN used to live).  Ie I don't see this as a major elephant in
the room, just need to decide between Arun's proposal where we
actively exclude these people or Tom's proposal where we don't.
Perhaps we should just vote.

Thanks,
Eli

>
> Cheers,
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: [EMAIL PROTECTED]
> WWW:   http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>

Hi Eli,

On Aug 16, 2012, at 1:21 PM, Eli Collins wrote:
>>> [...snip...]
>>
>> Keeping code in the same repository with a PMC with different sets of
>> permissions in that repository *is* the sign of a distinct community.
>> Doesn't matter if you want the code there together, and allowing patches,
>> and testing and releasing and whatever. Those are technical issues.
>> Having code with *different* rules for *the same* community members
>> is not what I know as "community over code" and the Apache way.
>> And ultimately it's the reason why this email thread won't die. There's
>> an elephant in the room here (pun intended).
>
> For the PMC the permissions are the same across all the subprojects
> and it is the same community. The question here is around committers.

Trying to distinguish between the 2 (PMC and committers) leads you to this.
I guess the Hadoop PMC is interested in the overhead of managing different
lists of people with different permissions who release different things. That
seems like an awfully lot of worthless overhead to me, and discussions
around topics that pretty much wouldn't matter at all if you adopted a flatter
organizational structure.

See 2 of my favorite quotes from former ASF Director Hen Yandell:

https://twitter.com/flamefew/statuses/36352411593351168
https://twitter.com/flamefew/statuses/36352484263858176

So whether you are talking about "PMC" or "committers" here, is really
moot on whether or not this is a community issue. It is. No matter how
you spin it.

>
> We already have a working model where the various subprojects have
> their own committers and people are trusted to commit across project
> boundaries when it makes sense.

Depends on what you consider to be working. Sure if the Hadoop PMC
wants to take on the overhead in a centralized fashion that is normally
distributed amongst projects and their own distinct PMCs that's fine.
Just doesn't seem to be a scalable solution that's all. Because if you
look back at most of the Apache projects that did this (e.g., umbrella
projects) they don't work out. That's b/c having a single PMC that has
merit across all of the sub-projects usually doesn't work because not
every person on that PMC rightly should have an equal VOTE on
those sub-projects that they have no merit in. This can easily be
the case when you have things like e.g., YARN, or new things come
along that you guys yourself try and bless instead of just realizing
that it's a distinct community.
> I think everyone is in agreement that
> yarn, like the other subprojects, will have it's own set of
> committers, the only open question here as I understand it is whether
> or not we grandfather in the existing MR committers (since that's
> where YARN used to live).  

If there's even a discussion about this, there's an issue. You guys
think this discussion is new and what you arent' realizing is that
we've had it before. In MANY Apache projects. Over MANY years.

So what's new to you is not new to me. And what's frustrating is watching
you guys waste time debating and discussing something that's already
been solved before. MANY times.

> Ie I don't see this as a major elephant in
> the room, just need to decide between Arun's proposal where we
> actively exclude these people or Tom's proposal where we don't.
> Perhaps we should just vote.

VOTEing to solve issues at Apache is also unfortunately the sign
of a community issue. Most healthy communities I see at the Foundation
discuss, reach consensus, and VOTEs are merely a formality putting pen
and paper to an existing consensus.

Anyhoo, I'll return back to the sidelines. I just thought after a few days
of this, I'd pipe up b/c I think you guys can VOTE however many times
you want, and create as many sub-whosamuwwhatsits as many times
as you want with different committers and whatever else, I just think
there could be an easier way. And it doesn't have to be re-invented here.

//back to sidelines

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

A very interesting point, Chris (and great quotes!).

As a non-PMC 3-years old committer of Hadoop I'd say that the split-line of
'PMC vs. committers' was always a very touchy-feely issue for the community. It
almost felt in times that only those who behave along a "party" lines (in some
definition of a party) are allowed into the sacred temple of PMC.

And I am totally agree with Hen's point: flatter the ratio between the two -
more healthier the community and more vibrant the project. I am coming from a
slightly different perspective here, because I never seen a truly successful
anything that is lead by a central authority.

At any rate, I would like to voice my opinion here and ask not to include me
to the list of initial YARN committers (if the grandfather'ing is accepted
instead of say incubating), because I haven't contributed to the project at
all. I argut that it isn't an honest way of becoming an official project
committer just because you happen to be around of the grandfathering
community.

Just my opinion, non binding apparently.

Cos

 On Thu, Aug 16, 2012 at 08:41PM, Mattmann, Chris A (388J) wrote:
> Hi Eli,
>
> On Aug 16, 2012, at 1:21 PM, Eli Collins wrote:
> >>> [...snip...]
> >>
> >> Keeping code in the same repository with a PMC with different sets of
> >> permissions in that repository *is* the sign of a distinct community.
> >> Doesn't matter if you want the code there together, and allowing patches,
> >> and testing and releasing and whatever. Those are technical issues.
> >> Having code with *different* rules for *the same* community members
> >> is not what I know as "community over code" and the Apache way.
> >> And ultimately it's the reason why this email thread won't die. There's
> >> an elephant in the room here (pun intended).
> >
> > For the PMC the permissions are the same across all the subprojects
> > and it is the same community. The question here is around committers.
>
> Trying to distinguish between the 2 (PMC and committers) leads you to this.
> I guess the Hadoop PMC is interested in the overhead of managing different
> lists of people with different permissions who release different things. That
> seems like an awfully lot of worthless overhead to me, and discussions
> around topics that pretty much wouldn't matter at all if you adopted a flatter
> organizational structure.
>
> See 2 of my favorite quotes from former ASF Director Hen Yandell:
>
> https://twitter.com/flamefew/statuses/36352411593351168
> https://twitter.com/flamefew/statuses/36352484263858176
>
> So whether you are talking about "PMC" or "committers" here, is really
> moot on whether or not this is a community issue. It is. No matter how
> you spin it.
>
> >
> > We already have a working model where the various subprojects have
> > their own committers and people are trusted to commit across project
> > boundaries when it makes sense.
>
> Depends on what you consider to be working. Sure if the Hadoop PMC
> wants to take on the overhead in a centralized fashion that is normally
> distributed amongst projects and their own distinct PMCs that's fine.
> Just doesn't seem to be a scalable solution that's all. Because if you
> look back at most of the Apache projects that did this (e.g., umbrella
> projects) they don't work out. That's b/c having a single PMC that has
> merit across all of the sub-projects usually doesn't work because not
> every person on that PMC rightly should have an equal VOTE on
> those sub-projects that they have no merit in. This can easily be
> the case when you have things like e.g., YARN, or new things come
> along that you guys yourself try and bless instead of just realizing
> that it's a distinct community.
>
>
> > I think everyone is in agreement that
> > yarn, like the other subprojects, will have it's own set of
> > committers, the only open question here as I understand it is whether
> > or not we grandfather in the existing MR committers (since that's
> > where YARN used to live).  

The HDFS and MapReduce committer roles were split to ease the
transition to TLPs, where both projects could manage and release their
code independently. However, after an initial burst of enthusiasm,
progress has been trending toward merging the projects back together.
The original rationale for splitting committer roles does not hold and
will not be realized in the foreseeable future, so the two lists
should merge again.

Similarly, either Yarn has reached escape velocity and should be a TLP
(with a sensible subset of its contributors) or it will be developed,
released, and managed concurrently with HDFS and MapReduce. We should
stop pretending that we're close to managing and releasing subprojects
independently in opposition to reality. That fiction is drawing
*meaningful, but irrelevant* distinctions between contributors.
Anticipating a split with a separate committer list has not been a
successful pattern, as measured in pending patches,
administrative/developer overhead, and distracting email threads like
this one.

However, contrary appeals to emotional reasoning citing "exclusion" or
a "lack of trust in contributors" are lazy and invalid. The proposal
is a dry partitioning for a specific, administrative purpose: to be
transparent about criteria to be included in a Yarn TLP while that
project is still managed here. But if that promotion is not imminent-
if, for the foreseeable future, Yarn will be developed and released
with HDFS and MapReduce- then the cost of preparing for a split that
doesn't happen is all overhead. There is no value in incurring it.

Neither is there value in discussing this as anything but an
administrative action with concrete criteria and consequences for
success (TLP status in a reasonable, set timeframe) and failure
(continued dependency and co-development, roles are merged back).
Define these and drop the drama.

tl;dr: I agree with Chris Mattmann: start making subprojects into TLPs
or flatten the roles. These half-measures are expensive drains on
attention. -C

On Thu, Aug 16, 2012 at 1:21 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
> On Thu, Aug 16, 2012 at 1:11 PM, Mattmann, Chris A (388J)
> <[EMAIL PROTECTED]> wrote:
>> Hi Eli,
>>
>> On Aug 16, 2012, at 1:08 PM, Eli Collins wrote:
>>
>>> On Thu, Aug 16, 2012 at 12:59 PM, Mattmann, Chris A (388J)
>>> <[EMAIL PROTECTED]> wrote:
>>>> Hi Guys,
>>>>
>>>> The existing discussion and conversation below is the precise reason that I suggested
>>>> Hadoop consider spinning out the rest of its *products* as *projects*. Folks have piped
>>>> up and listed technical reasons as the challenges behind this, and then responded
>>>> with clear community reasons either by their actions, or by other means.
>>>>
>>>> Having distinct communities as indicated by distinct lists of committers isn't wrong -- it's
>>>> usually however exemplified by having a distinct Apache project. It sounds like those
>>>> folks that have been working on YARN for 1.5 years+ as stated would like to have their
>>>> own distinct Apache community.
>>>
>>> Not sure that's the case, eg I think we all want to keep yarn in the
>>> same code repository, allow patches that update it along with other
>>> hadoop subprojects, co-design it with MR, test/release it together as
>>> well.  That's not a sign of a distinct community.
>>
>> Keeping code in the same repository with a PMC with different sets of
>> permissions in that repository *is* the sign of a distinct community.
>> Doesn't matter if you want the code there together, and allowing patches,
>> and testing and releasing and whatever. Those are technical issues.
>> Having code with *different* rules for *the same* community members
>> is not what I know as "community over code" and the Apache way.
>> And ultimately it's the reason why this email thread won't die. There's
>> an elephant in the room here (pun intended).
>
> For the PMC the permissions are the same across all the subprojects
> and it is the same community. The question here is around committers.

Woah, I'm gone for one day and miss out a lot of fun - thank god for little mercies! :)

A couple of quick responses before I need to run again...

> However, contrary appeals to emotional reasoning citing "exclusion" or
> a "lack of trust in contributors" are lazy and invalid. The proposal
> is a dry partitioning for a specific, administrative purpose: to be
> transparent about criteria to be included in a Yarn
I thought I already did, but my bad - apologies. Looks like a couple of people asked, here you go:

I included everyone with 20 patches and then, on Tom's suggestion, anyone with 10 commits - both for a period of 18 months.
(The list doesn't change if we go with a filter of 15 patches for the same period.)
Is thought it was a reasonably small bar, no? Should we go higher?

Regarding why this wasn't discussed before, not sure - but we are discussing now after Tom asked to make up.
And boy, are we *discussing*! :)

Arun

As someone who has contributed major code to YARN, I would prefer to have
the committer list to be inclusive of all MR committers.
I quickly browsed the list of MR and HDFS committers. Lot of HDFS
committers are really folks who have worked on only MR side (like me) who
have rarely contributed to HDFS. Same holds true for some MR committers.
And there is a huge overlap even with the "common" sub project.

I understand giving committer status is primarily based on
"familiarity/contribution to a particular sub-project" + "trust a person
gains by working on other parts of project". IMO the second point is much
more important and in past we have given committership based on this.
Contribution in one part of the project earns someone trust.I don't think
we have seen instances where people have misused that in the past.

After all, it is all part of the same TLP Hadoop. Then why to create walls
by creating so many lists and the management overhead too.

IMO we should look at merging the other lists too but that anyway is a
separate discussion.

On Thu, Aug 16, 2012 at 1:21 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
> I think everyone is in agreement that
> yarn, like the other subprojects, will have it's own set of
> committers

Yes, that is unfortunately the status quo in Hadoop, since the vote to
combine these sets failed in August of 2010:

http://s.apache.org/15r

Perhaps it's time to revisit that decision?

Doug

On Tue, Aug 21, 2012 at 3:46 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:
> On Thu, Aug 16, 2012 at 1:21 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
>> I think everyone is in agreement that
>> yarn, like the other subprojects, will have it's own set of
>> committers
>
> Yes, that is unfortunately the status quo in Hadoop, since the vote to
> combine these sets failed in August of 2010:
>
> http://s.apache.org/15r
>
> Perhaps it's time to revisit that decision?
>

Sure, how about a vote between:

1. The status quo (distinct MR and HDFS committers where MR includes YARN)
2. Distinct MR, HDFS and YARN committers
3. Combine MR, YARN, HDFS committers

On Tue, Aug 21, 2012 at 4:13 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
> Sure, how about a vote between:
>
> 1. The status quo (distinct MR and HDFS committers where MR includes YARN)
> 2. Distinct MR, HDFS and YARN committers
> 3. Combine MR, YARN, HDFS committers

(3) is a change to the bylaws, getting rid of the separate committer
lists for subprojects described there:

http://hadoop.apache.org/bylaws.html

The August 2010 vote pre-dated our bylaws and was done under 2/3
majority rules (http://s.apache.org/BP5), but our bylaws now state
that bylaw changes only require a simple majority.

Getting a majority for one of 3 options is not always possible,
leaving us without a decision.  So we might better just vote on (2)
versus (3)?

Doug

On Wed, Aug 22, 2012 at 11:38 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:
> On Tue, Aug 21, 2012 at 4:13 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
>> Sure, how about a vote between:
>>
>> 1. The status quo (distinct MR and HDFS committers where MR includes YARN)
>> 2. Distinct MR, HDFS and YARN committers
>> 3. Combine MR, YARN, HDFS committers
>
> (3) is a change to the bylaws, getting rid of the separate committer
> lists for subprojects described there:
>
> http://hadoop.apache.org/bylaws.html
>
> The August 2010 vote pre-dated our bylaws and was done under 2/3
> majority rules (http://s.apache.org/BP5), but our bylaws now state
> that bylaw changes only require a simple majority.
>
> Getting a majority for one of 3 options is not always possible,
> leaving us without a decision.  So we might better just vote on (2)
> versus (3)?

Works for me.  What do others think?

On Aug 22, 2012, at 11:43 AM, Eli Collins wrote:

> On Wed, Aug 22, 2012 at 11:38 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>>> 2. Distinct MR, HDFS and YARN committers
>>> 3. Combine MR, YARN, HDFS committers
>>
>> So we might better just vote on (2)
>> versus (3)?
>
> Works for me.  What do others think?

Can you clarify more on what you are proposing we vote on?

What does (2) mean? Since (1) was dropped, does it mean we seed the YARN list with folks who have been contributing/reviewing patches on YARN?

Thanks,
+Vinod

On Thu, Aug 23, 2012 at 11:13 AM, Vinod Kumar Vavilapalli
<[EMAIL PROTECTED]> wrote:
>
> On Aug 22, 2012, at 11:43 AM, Eli Collins wrote:
>
>> On Wed, Aug 22, 2012 at 11:38 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>>>> 2. Distinct MR, HDFS and YARN committers
>>>> 3. Combine MR, YARN, HDFS committers
>>>
>>> So we might better just vote on (2)
>>> versus (3)?
>>
>> Works for me.  What do others think?
>
> Can you clarify more on what you are proposing we vote on?
>
> What does (2) mean?

"Distinct MR, HDFS and YARN committers" means that MR, HDFS, and YARN
each have their own set of committers. Today we have two sets (MR and
HDFS) so under this option we would have three sets of committers.
(And technically a 4th set which is the PMC which is shared across
sub-projects and can commit to all of them).

> Since (1) was dropped, does it mean we seed the YARN list with folks who have been contributing/reviewing patches on YARN?

The vote would need to spell out how we would seed the YARN list. Per
above I'd suggest seeding it with the current MR committers - ie the
people who can commit to YARN today - there's no need to actively
exclude people we already trust to commit to this code, and there's
obvious downside to excluding them, for example, some patches need to
span sub-projects (same reason all HDFS/MR committers can commit to
Common).  If/when the sub-projects become TLPs (ie when there are real
distinct boundaries between the projects) seems like a good time to
divvy things up.

Personally I'm in favor of #3, I liked Chris D's original proposal to
just merge all the committer lists and call it a day!

Thanks,
Eli

Thanks for the clarification.

Seems like we can start a vote on those lines. But let's do that separately on a fresh thread. This one has stretched for far too long after the original vote thread concluded.

Thanks,
+Vinod

On Aug 23, 2012, at 11:33 AM, Eli Collins wrote:

> On Thu, Aug 23, 2012 at 11:13 AM, Vinod Kumar Vavilapalli
> <[EMAIL PROTECTED]> wrote:
>>
>> On Aug 22, 2012, at 11:43 AM, Eli Collins wrote:
>>
>>> On Wed, Aug 22, 2012 at 11:38 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>>>>> 2. Distinct MR, HDFS and YARN committers
>>>>> 3. Combine MR, YARN, HDFS committers
>>>>
>>>> So we might better just vote on (2)
>>>> versus (3)?
>>>
>>> Works for me.  What do others think?
>>
>> Can you clarify more on what you are proposing we vote on?
>>
>> What does (2) mean?
>
> "Distinct MR, HDFS and YARN committers" means that MR, HDFS, and YARN
> each have their own set of committers. Today we have two sets (MR and
> HDFS) so under this option we would have three sets of committers.
> (And technically a 4th set which is the PMC which is shared across
> sub-projects and can commit to all of them).
>
>> Since (1) was dropped, does it mean we seed the YARN list with folks who have been contributing/reviewing patches on YARN?
>
> The vote would need to spell out how we would seed the YARN list. Per
> above I'd suggest seeding it with the current MR committers - ie the
> people who can commit to YARN today - there's no need to actively
> exclude people we already trust to commit to this code, and there's
> obvious downside to excluding them, for example, some patches need to
> span sub-projects (same reason all HDFS/MR committers can commit to
> Common).  If/when the sub-projects become TLPs (ie when there are real
> distinct boundaries between the projects) seems like a good time to
> divvy things up.
>
> Personally I'm in favor of #3, I liked Chris D's original proposal to
> just merge all the committer lists and call it a day!
>
> Thanks,
> Eli

Will do.  Not sure we can do a single vote with multiple options since
it represents a change to the bylaws and it would be possible for both
votes to pass.  I'll start a vote for #3 and if that fails someone can
start a vote for #2.

Thanks,
Eli

On Thu, Aug 23, 2012 at 7:11 PM, Vinod Kumar Vavilapalli
<[EMAIL PROTECTED]> wrote:
>
> Thanks for the clarification.
>
> Seems like we can start a vote on those lines. But let's do that separately on a fresh thread. This one has stretched for far too long after the original vote thread concluded.
>
> Thanks,
> +Vinod
>
> On Aug 23, 2012, at 11:33 AM, Eli Collins wrote:
>
>> On Thu, Aug 23, 2012 at 11:13 AM, Vinod Kumar Vavilapalli
>> <[EMAIL PROTECTED]> wrote:
>>>
>>> On Aug 22, 2012, at 11:43 AM, Eli Collins wrote:
>>>
>>>> On Wed, Aug 22, 2012 at 11:38 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>>>>>> 2. Distinct MR, HDFS and YARN committers
>>>>>> 3. Combine MR, YARN, HDFS committers
>>>>>
>>>>> So we might better just vote on (2)
>>>>> versus (3)?
>>>>
>>>> Works for me.  What do others think?
>>>
>>> Can you clarify more on what you are proposing we vote on?
>>>
>>> What does (2) mean?
>>
>> "Distinct MR, HDFS and YARN committers" means that MR, HDFS, and YARN
>> each have their own set of committers. Today we have two sets (MR and
>> HDFS) so under this option we would have three sets of committers.
>> (And technically a 4th set which is the PMC which is shared across
>> sub-projects and can commit to all of them).
>>
>>> Since (1) was dropped, does it mean we seed the YARN list with folks who have been contributing/reviewing patches on YARN?
>>
>> The vote would need to spell out how we would seed the YARN list. Per
>> above I'd suggest seeding it with the current MR committers - ie the
>> people who can commit to YARN today - there's no need to actively
>> exclude people we already trust to commit to this code, and there's
>> obvious downside to excluding them, for example, some patches need to
>> span sub-projects (same reason all HDFS/MR committers can commit to
>> Common).  If/when the sub-projects become TLPs (ie when there are real
>> distinct boundaries between the projects) seems like a good time to
>> divvy things up.
>>
>> Personally I'm in favor of #3, I liked Chris D's original proposal to
>> just merge all the committer lists and call it a day!
>>
>> Thanks,
>> Eli
>

Vote already started now?
Is this also runs for 7 days?

-thx

On Fri, Aug 24, 2012 at 12:03 AM, Eli Collins <[EMAIL PROTECTED]> wrote:

> On Thu, Aug 23, 2012 at 11:13 AM, Vinod Kumar Vavilapalli
> <[EMAIL PROTECTED]> wrote:
> >
> > On Aug 22, 2012, at 11:43 AM, Eli Collins wrote:
> >
> >> On Wed, Aug 22, 2012 at 11:38 AM, Doug Cutting <[EMAIL PROTECTED]>
> wrote:
> >>>> 2. Distinct MR, HDFS and YARN committers
> >>>> 3. Combine MR, YARN, HDFS committers
> >>>
> >>> So we might better just vote on (2)
> >>> versus (3)?
> >>
> >> Works for me.  What do others think?
> >
> > Can you clarify more on what you are proposing we vote on?
> >
> > What does (2) mean?
>
> "Distinct MR, HDFS and YARN committers" means that MR, HDFS, and YARN
> each have their own set of committers. Today we have two sets (MR and
> HDFS) so under this option we would have three sets of committers.
> (And technically a 4th set which is the PMC which is shared across
> sub-projects and can commit to all of them).
>
> > Since (1) was dropped, does it mean we seed the YARN list with folks who
> have been contributing/reviewing patches on YARN?
>
> The vote would need to spell out how we would seed the YARN list. Per
> above I'd suggest seeding it with the current MR committers - ie the
> people who can commit to YARN today - there's no need to actively
> exclude people we already trust to commit to this code, and there's
> obvious downside to excluding them, for example, some patches need to
> span sub-projects (same reason all HDFS/MR committers can commit to
> Common).  If/when the sub-projects become TLPs (ie when there are real
> distinct boundaries between the projects) seems like a good time to
> divvy things up.
>
> Personally I'm in favor of #3, I liked Chris D's original proposal to
> just merge all the committer lists and call it a day!
>
> Thanks,
> Eli
>

Chris has seen more of these sorts of things than any of the other
participants in this discussion.

His experience here is very valuable.

On Thu, Aug 16, 2012 at 1:11 PM, Mattmann, Chris A (388J) <
[EMAIL PROTECTED]> wrote:

> > Not sure that's the case, eg I think we all want to keep yarn in the
> > same code repository, allow patches that update it along with other
> > hadoop subprojects, co-design it with MR, test/release it together as
> > well.  That's not a sign of a distinct community.
>
> Keeping code in the same repository with a PMC with different sets of
> permissions in that repository *is* the sign of a distinct community.
> Doesn't matter if you want the code there together, and allowing patches,
> and testing and releasing and whatever. Those are technical issues.
> Having code with *different* rules for *the same* community members
> is not what I know as "community over code" and the Apache way.
> And ultimately it's the reason why this email thread won't die. There's
> an elephant in the room here (pun intended).
>

Can we update the mailing list page with the new ones and combined users:
http://hadoop.apache.org/mailing_lists.html?

Thanks
Tom
On 8/8/12 2:29 PM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:

> Ok, we are nearly there.
>
> # The new YARN jira is functional now (albeit with some teething issues with
> notifications/workflow which should be resolved soon):
> https://issues.apache.org/jira/browse/YARN
> # I've committed YARN-1 to move YARN out of hadoop-mapreduce-project onto
> hadoop-yarn-project in subversion.
> # The yarn-dev@ mailing list is active.
> # All jira notifications are going to yarn-issues@ and all subversion commit
> notifications are going to yarn-commits@.
>
> Please do subscribe to the relevant mailing lists.
> Also note that there is *no* yarn-user@ list since all user interactions are
> now via [EMAIL PROTECTED].
>
> If you see issues, please let me know and I'll try to respond ASAP.
>
> thanks,
> Arun
>
> On Aug 3, 2012, at 12:51 AM, Arun C Murthy wrote:
>
>> With 22 +1s (9 binding) and no -1s the vote passes unanimously. Thanks to all
>> who voted.
>>
>> I'll start the necessary process to move on this.
>>
>> thanks,
>> Arun
>>
>> On Jul 26, 2012, at 8:24 PM, Arun C Murthy wrote:
>>
>>> As we discussed in the previous thread, I'd like to call a vote to establish
>>> YARN as a sub-project (it's the best *term* we have to describe it in our
>>> current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>>>
>>> Specifically,
>>> # Create a separate top-level src folder hadoop-yarn, move it out of
>>> hadoop-mapreduce-project in subversion at hadoop/common/trunk.
>>> # Create a separate YARN jira project and yarn-dev@ mailing-list.
>>> # Continue to co-release a single Hadoop release with Common, HDFS, YARN &
>>> MapReduce.
>>>
>>> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>>>
>>> thanks,
>>> Arun
>>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>

I think this was being done via HADOOP-8652, but it looks like there
were some issues updating the site after the commit.  I'm assuming
they're still working through those issues.

Jason

On 08/13/2012 09:16 AM, Thomas Graves wrote:
> Can we update the mailing list page with the new ones and combined users:
> http://hadoop.apache.org/mailing_lists.html?
>
> Thanks
> Tom
>
>
> On 8/8/12 2:29 PM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:
>
>> Ok, we are nearly there.
>>
>> # The new YARN jira is functional now (albeit with some teething issues with
>> notifications/workflow which should be resolved soon):
>> https://issues.apache.org/jira/browse/YARN
>> # I've committed YARN-1 to move YARN out of hadoop-mapreduce-project onto
>> hadoop-yarn-project in subversion.
>> # The yarn-dev@ mailing list is active.
>> # All jira notifications are going to yarn-issues@ and all subversion commit
>> notifications are going to yarn-commits@.
>>
>> Please do subscribe to the relevant mailing lists.
>> Also note that there is *no* yarn-user@ list since all user interactions are
>> now via [EMAIL PROTECTED].
>>
>> If you see issues, please let me know and I'll try to respond ASAP.
>>
>> thanks,
>> Arun
>>
>> On Aug 3, 2012, at 12:51 AM, Arun C Murthy wrote:
>>
>>> With 22 +1s (9 binding) and no -1s the vote passes unanimously. Thanks to all
>>> who voted.
>>>
>>> I'll start the necessary process to move on this.
>>>
>>> thanks,
>>> Arun
>>>
>>> On Jul 26, 2012, at 8:24 PM, Arun C Murthy wrote:
>>>
>>>> As we discussed in the previous thread, I'd like to call a vote to establish
>>>> YARN as a sub-project (it's the best *term* we have to describe it in our
>>>> current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>>>>
>>>> Specifically,
>>>> # Create a separate top-level src folder hadoop-yarn, move it out of
>>>> hadoop-mapreduce-project in subversion at hadoop/common/trunk.
>>>> # Create a separate YARN jira project and yarn-dev@ mailing-list.
>>>> # Continue to co-release a single Hadoop release with Common, HDFS, YARN &
>>>> MapReduce.
>>>>
>>>> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>>>>
>>>> thanks,
>>>> Arun
>>>>
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>>
>>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>

Please remove me from this email chain.
Ryan Grymes
Business Development Manager
410-534-4800 ext 260
Connect With Me

-----Original Message-----
From: Jason Lowe [mailto:[EMAIL PROTECTED]]
Sent: Monday, August 13, 2012 11:38 AM
To: [EMAIL PROTECTED]
Subject: Re: [VOTE] - Establish YARN as a sub-project of Apache Hadoop

I think this was being done via HADOOP-8652, but it looks like there
were some issues updating the site after the commit.  I'm assuming
they're still working through those issues.

Jason

On 08/13/2012 09:16 AM, Thomas Graves wrote:
> Can we update the mailing list page with the new ones and combined users:
> http://hadoop.apache.org/mailing_lists.html?
>
> Thanks
> Tom
>
>
> On 8/8/12 2:29 PM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:
>
>> Ok, we are nearly there.
>>
>> # The new YARN jira is functional now (albeit with some teething issues with
>> notifications/workflow which should be resolved soon):
>> https://issues.apache.org/jira/browse/YARN
>> # I've committed YARN-1 to move YARN out of hadoop-mapreduce-project onto
>> hadoop-yarn-project in subversion.
>> # The yarn-dev@ mailing list is active.
>> # All jira notifications are going to yarn-issues@ and all subversion commit
>> notifications are going to yarn-commits@.
>>
>> Please do subscribe to the relevant mailing lists.
>> Also note that there is *no* yarn-user@ list since all user interactions are
>> now via [EMAIL PROTECTED].
>>
>> If you see issues, please let me know and I'll try to respond ASAP.
>>
>> thanks,
>> Arun
>>
>> On Aug 3, 2012, at 12:51 AM, Arun C Murthy wrote:
>>
>>> With 22 +1s (9 binding) and no -1s the vote passes unanimously. Thanks to all
>>> who voted.
>>>
>>> I'll start the necessary process to move on this.
>>>
>>> thanks,
>>> Arun
>>>
>>> On Jul 26, 2012, at 8:24 PM, Arun C Murthy wrote:
>>>
>>>> As we discussed in the previous thread, I'd like to call a vote to establish
>>>> YARN as a sub-project (it's the best *term* we have to describe it in our
>>>> current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>>>>
>>>> Specifically,
>>>> # Create a separate top-level src folder hadoop-yarn, move it out of
>>>> hadoop-mapreduce-project in subversion at hadoop/common/trunk.
>>>> # Create a separate YARN jira project and yarn-dev@ mailing-list.
>>>> # Continue to co-release a single Hadoop release with Common, HDFS, YARN &
>>>> MapReduce.
>>>>
>>>> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>>>>
>>>> thanks,
>>>> Arun
>>>>
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>>
>>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>

+1, after the fact. Was off the grid.

Thanks,
+Vinod

On Jul 26, 2012, at 8:24 PM, Arun C Murthy wrote:

> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>

Hi Tom,

Speaking as someone who is being excluded from the new list, this seems fair to me.  It seems like you are worrying about a hypothetical.  Anyone who feels they have contributed significantly to YARN need only identify themselves.  Others are welcome to do so and become committers via the normal process.  Seems meritocratic / appropriate.

E14

On Jul 26, 2012, at 8:24 PM, Arun C Murthy wrote:

> As we discussed in the previous thread, I'd like to call a vote to establish YARN as a sub-project (it's the best *term* we have to describe it in our current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN & MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>

Hello,

I agree with Tom on this one. Seeding the initial committer list of a split
off project with those committers on the previous project is what we did
during the Common/HDFS/MR project split, and I see no reason to deviate
from that practice in this case.

In matters such as who should be on initial committer lists, when to move
folks to emeritus status, etc., I've always been of the opinion that we
should err on the side of being overly inclusive. We trust all of our
committers to know their limitations, and not make modifications to the
code base in areas where they are unfamiliar. If you're concerned with the
hypothetical situation that some committer might make hasty changes to an
area of YARN that they are unfamiliar with, then that person shouldn't be
an MR committer either. The fact that an MR committer has earned our trust
in that project indicates to me that they should be similarly trusted in
the YARN project. In general, we should want more people, not fewer, who
are able to review patches and fix up issues found in YARN/MR/etc.

At the very least, if we were not going to seed the initial committer list
with those from the MR project, we should have discussed that before
creating a separate sub-project for YARN. If that had been a condition of
YARN becoming a separate sub-project, I'm not sure I would have supported
it.

--
Aaron T. Myers
Software Engineer, Cloudera

On Tue, Aug 14, 2012 at 8:00 PM, Eric Baldeschwieler <[EMAIL PROTECTED]
> wrote:

> Hi Tom,
>
> Speaking as someone who is being excluded from the new list, this seems
> fair to me.  It seems like you are worrying about a hypothetical.  Anyone
> who feels they have contributed significantly to YARN need only identify
> themselves.  Others are welcome to do so and become committers via the
> normal process.  Seems meritocratic / appropriate.
>
> E14
>
>
>
> On Jul 26, 2012, at 8:24 PM, Arun C Murthy wrote:
>
> > As we discussed in the previous thread, I'd like to call a vote to
> establish YARN as a sub-project (it's the best *term* we have to describe
> it in our current nomenclature) of Apache Hadoop along with Common, HDFS &
> MapReduce.
> >
> > Specifically,
> > # Create a separate top-level src folder hadoop-yarn, move it out of
> hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> > # Create a separate YARN jira project and yarn-dev@ mailing-list.
> > # Continue to co-release a single Hadoop release with Common, HDFS, YARN
> & MapReduce.
> >
> > Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
> >
> > thanks,
> > Arun
> >
>
>

+1

--Bobby Evans

On 7/26/12 10:24 PM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:

>As we discussed in the previous thread, I'd like to call a vote to
>establish YARN as a sub-project (it's the best *term* we have to describe
>it in our current nomenclature) of Apache Hadoop along with Common, HDFS
>& MapReduce.
>
>Specifically,
># Create a separate top-level src folder hadoop-yarn, move it out of
>hadoop-mapreduce-project in subversion at hadoop/common/trunk.
># Create a separate YARN jira project and yarn-dev@ mailing-list.
># Continue to co-release a single Hadoop release with Common, HDFS, YARN
>& MapReduce.
>
>Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
>thanks,
>Arun
>

+1

Tom
On 7/26/12 10:24 PM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:

> As we discussed in the previous thread, I'd like to call a vote to establish
> YARN as a sub-project (it's the best *term* we have to describe it in our
> current nomenclature) of Apache Hadoop along with Common, HDFS & MapReduce.
>
> Specifically,
> # Create a separate top-level src folder hadoop-yarn, move it out of
> hadoop-mapreduce-project in subversion at hadoop/common/trunk.
> # Create a separate YARN jira project and yarn-dev@ mailing-list.
> # Continue to co-release a single Hadoop release with Common, HDFS, YARN &
> MapReduce.
>
> Please vote, the vote will run the normal 7 days. Obviously, I'm +1.
>
> thanks,
> Arun
>