Per this thread [1] should we have a single set of committers for the
entire Hadoop project, ie all subprojects?

If the vote passes, current and future committers will gain commit
rights in all current (Common, HDFS, MapReduce, YARN) and future
Hadoop subprojects. The specific change to the bylaws follows:

Index: main/author/src/documentation/content/xdocs/bylaws.xml
==================================================================--- main/author/src/documentation/content/xdocs/bylaws.xml (revision 1376823)
+++ main/author/src/documentation/content/xdocs/bylaws.xml (working copy)
@@ -64,10 +64,9 @@
    <li> <strong>Committers</strong>

         <p>The project's Committers are responsible for the project's
-        technical management. Committers have access to a specified
-        set of subprojects' subversion repositories. Committers on
-        subprojects may cast binding votes on any technical discussion
-        regarding that subproject.</p>
+        technical management. Committers have access to all subproject
+        subversion repositories. Committers may cast binding votes on
+        any technical discussion regarding that subproject.</p>
This vote will run for 7 days. Since the vote represents a
modification of the bylawys [2] it requires a lazy majority of active
PMC members.

1. http://s.apache.org/WW1
2. http://hadoop.apache.org/bylaws.html

Here's my +1

+1 (binding).

On Thu, Aug 23, 2012 at 9:20 PM, Eli Collins <[EMAIL PROTECTED]> wrote:

> Per this thread [1] should we have a single set of committers for the
> entire Hadoop project, ie all subprojects?
>
> If the vote passes, current and future committers will gain commit
> rights in all current (Common, HDFS, MapReduce, YARN) and future
> Hadoop subprojects. The specific change to the bylaws follows:
>
> Index: main/author/src/documentation/content/xdocs/bylaws.xml
> ==================================================================> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision
> 1376823)
> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working
> copy)
> @@ -64,10 +64,9 @@
>     <li> <strong>Committers</strong>
>
>          <p>The project's Committers are responsible for the project's
> -        technical management. Committers have access to a specified
> -        set of subprojects' subversion repositories. Committers on
> -        subprojects may cast binding votes on any technical discussion
> -        regarding that subproject.</p>
> +        technical management. Committers have access to all subproject
> +        subversion repositories. Committers may cast binding votes on
> +        any technical discussion regarding that subproject.</p>
>
>
> This vote will run for 7 days. Since the vote represents a
> modification of the bylawys [2] it requires a lazy majority of active
> PMC members.
>
> 1. http://s.apache.org/WW1
> 2. http://hadoop.apache.org/bylaws.html
>
> Here's my +1
>

+1

Doug

On Fri, Aug 24, 2012 at 10:55 AM, Jakob Homan <[EMAIL PROTECTED]> wrote:
> +1 (binding).
>
> On Thu, Aug 23, 2012 at 9:20 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
>
>> Per this thread [1] should we have a single set of committers for the
>> entire Hadoop project, ie all subprojects?
>>
>> If the vote passes, current and future committers will gain commit
>> rights in all current (Common, HDFS, MapReduce, YARN) and future
>> Hadoop subprojects. The specific change to the bylaws follows:
>>
>> Index: main/author/src/documentation/content/xdocs/bylaws.xml
>> ==================================================================>> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision
>> 1376823)
>> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working
>> copy)
>> @@ -64,10 +64,9 @@
>>     <li> <strong>Committers</strong>
>>
>>          <p>The project's Committers are responsible for the project's
>> -        technical management. Committers have access to a specified
>> -        set of subprojects' subversion repositories. Committers on
>> -        subprojects may cast binding votes on any technical discussion
>> -        regarding that subproject.</p>
>> +        technical management. Committers have access to all subproject
>> +        subversion repositories. Committers may cast binding votes on
>> +        any technical discussion regarding that subproject.</p>
>>
>>
>> This vote will run for 7 days. Since the vote represents a
>> modification of the bylawys [2] it requires a lazy majority of active
>> PMC members.
>>
>> 1. http://s.apache.org/WW1
>> 2. http://hadoop.apache.org/bylaws.html
>>
>> Here's my +1
>>

On Thu, Aug 23, 2012 at 9:20 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
> Per this thread [1] should we have a single set of committers for the
> entire Hadoop project, ie all subprojects?
>
> If the vote passes, current and future committers will gain commit
> rights in all current (Common, HDFS, MapReduce, YARN) and future
> Hadoop subprojects. The specific change to the bylaws follows:
>
> Index: main/author/src/documentation/content/xdocs/bylaws.xml
> ==================================================================> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision 1376823)
> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working copy)
> @@ -64,10 +64,9 @@
>     <li> <strong>Committers</strong>
>
>          <p>The project's Committers are responsible for the project's
> -        technical management. Committers have access to a specified
> -        set of subprojects' subversion repositories. Committers on
> -        subprojects may cast binding votes on any technical discussion
> -        regarding that subproject.</p>
> +        technical management. Committers have access to all subproject
> +        subversion repositories. Committers may cast binding votes on
> +        any technical discussion regarding that subproject.</p>
>

Andy Isaacson pointed out that I should replace "that" with "any"
above in "regarding that subproject" to make this more clear.  Here's
the updated diff, same intent.

site $ svn diff
Index: main/author/src/documentation/content/xdocs/bylaws.xml
==================================================================--- main/author/src/documentation/content/xdocs/bylaws.xml (revision 1376823)
+++ main/author/src/documentation/content/xdocs/bylaws.xml (working copy)
@@ -64,10 +64,9 @@
    <li> <strong>Committers</strong>

         <p>The project's Committers are responsible for the project's
-        technical management. Committers have access to a specified
-        set of subprojects' subversion repositories. Committers on
-        subprojects may cast binding votes on any technical discussion
-        regarding that subproject.</p>
+        technical management. Committers have access to all subproject
+        subversion repositories. Committers may cast binding votes on
+        any technical discussion regarding any subproject.</p>

+1 -C

On Thu, Aug 23, 2012 at 9:20 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
> Per this thread [1] should we have a single set of committers for the
> entire Hadoop project, ie all subprojects?
>
> If the vote passes, current and future committers will gain commit
> rights in all current (Common, HDFS, MapReduce, YARN) and future
> Hadoop subprojects. The specific change to the bylaws follows:
>
> Index: main/author/src/documentation/content/xdocs/bylaws.xml
> ==================================================================> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision 1376823)
> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working copy)
> @@ -64,10 +64,9 @@
>     <li> <strong>Committers</strong>
>
>          <p>The project's Committers are responsible for the project's
> -        technical management. Committers have access to a specified
> -        set of subprojects' subversion repositories. Committers on
> -        subprojects may cast binding votes on any technical discussion
> -        regarding that subproject.</p>
> +        technical management. Committers have access to all subproject
> +        subversion repositories. Committers may cast binding votes on
> +        any technical discussion regarding that subproject.</p>
>
>
> This vote will run for 7 days. Since the vote represents a
> modification of the bylawys [2] it requires a lazy majority of active
> PMC members.
>
> 1. http://s.apache.org/WW1
> 2. http://hadoop.apache.org/bylaws.html
>
> Here's my +1

+1 (non binding)

On Sat, Aug 25, 2012 at 6:33 AM, Chris Douglas <[EMAIL PROTECTED]> wrote:
> +1 -C
>
> On Thu, Aug 23, 2012 at 9:20 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
>> Per this thread [1] should we have a single set of committers for the
>> entire Hadoop project, ie all subprojects?
>>
>> If the vote passes, current and future committers will gain commit
>> rights in all current (Common, HDFS, MapReduce, YARN) and future
>> Hadoop subprojects. The specific change to the bylaws follows:
>>
>> Index: main/author/src/documentation/content/xdocs/bylaws.xml
>> ==================================================================>> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision 1376823)
>> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working copy)
>> @@ -64,10 +64,9 @@
>>     <li> <strong>Committers</strong>
>>
>>          <p>The project's Committers are responsible for the project's
>> -        technical management. Committers have access to a specified
>> -        set of subprojects' subversion repositories. Committers on
>> -        subprojects may cast binding votes on any technical discussion
>> -        regarding that subproject.</p>
>> +        technical management. Committers have access to all subproject
>> +        subversion repositories. Committers may cast binding votes on
>> +        any technical discussion regarding that subproject.</p>
>>
>>
>> This vote will run for 7 days. Since the vote represents a
>> modification of the bylawys [2] it requires a lazy majority of active
>> PMC members.
>>
>> 1. http://s.apache.org/WW1
>> 2. http://hadoop.apache.org/bylaws.html
>>
>> Here's my +1

--
Harsh J

+1

On Fri, Aug 24, 2012 at 7:25 PM, Harsh J <[EMAIL PROTECTED]> wrote:
> +1 (non binding)
>
> On Sat, Aug 25, 2012 at 6:33 AM, Chris Douglas <[EMAIL PROTECTED]> wrote:
>> +1 -C
>>
>> On Thu, Aug 23, 2012 at 9:20 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
>>> Per this thread [1] should we have a single set of committers for the
>>> entire Hadoop project, ie all subprojects?
>>>
>>> If the vote passes, current and future committers will gain commit
>>> rights in all current (Common, HDFS, MapReduce, YARN) and future
>>> Hadoop subprojects. The specific change to the bylaws follows:
>>>
>>> Index: main/author/src/documentation/content/xdocs/bylaws.xml
>>> ==================================================================>>> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision 1376823)
>>> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working copy)
>>> @@ -64,10 +64,9 @@
>>>     <li> <strong>Committers</strong>
>>>
>>>          <p>The project's Committers are responsible for the project's
>>> -        technical management. Committers have access to a specified
>>> -        set of subprojects' subversion repositories. Committers on
>>> -        subprojects may cast binding votes on any technical discussion
>>> -        regarding that subproject.</p>
>>> +        technical management. Committers have access to all subproject
>>> +        subversion repositories. Committers may cast binding votes on
>>> +        any technical discussion regarding that subproject.</p>
>>>
>>>
>>> This vote will run for 7 days. Since the vote represents a
>>> modification of the bylawys [2] it requires a lazy majority of active
>>> PMC members.
>>>
>>> 1. http://s.apache.org/WW1
>>> 2. http://hadoop.apache.org/bylaws.html
>>>
>>> Here's my +1
>
>
>
> --
> Harsh J

--
Alejandro

+1

+1.

Tom Graves
On 8/23/12 11:20 PM, "Eli Collins" <[EMAIL PROTECTED]> wrote:

> Per this thread [1] should we have a single set of committers for the
> entire Hadoop project, ie all subprojects?
>
> If the vote passes, current and future committers will gain commit
> rights in all current (Common, HDFS, MapReduce, YARN) and future
> Hadoop subprojects. The specific change to the bylaws follows:
>
> Index: main/author/src/documentation/content/xdocs/bylaws.xml
> ==================================================================> --- main/author/src/documentation/content/xdocs/bylaws.xml (revision 1376823)
> +++ main/author/src/documentation/content/xdocs/bylaws.xml (working copy)
> @@ -64,10 +64,9 @@
>     <li> <strong>Committers</strong>
>
>          <p>The project's Committers are responsible for the project's
> -        technical management. Committers have access to a specified
> -        set of subprojects' subversion repositories. Committers on
> -        subprojects may cast binding votes on any technical discussion
> -        regarding that subproject.</p>
> +        technical management. Committers have access to all subproject
> +        subversion repositories. Committers may cast binding votes on
> +        any technical discussion regarding that subproject.</p>
>
>
> This vote will run for 7 days. Since the vote represents a
> modification of the bylawys [2] it requires a lazy majority of active
> PMC members.
>
> 1. http://s.apache.org/WW1
> 2. http://hadoop.apache.org/bylaws.html
>
> Here's my +1

+1
Makes sense.

My rational is that since "Any committer may serve as the manager of a release"
http://www.apache.org/dev/release-publishing.html#release_manager
then any committer needs access to the whole tree.

Thanks,
--Konstantin

On Thu, Aug 23, 2012 at 9:20 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
> Per this thread [1] should we have a single set of committers for the
> entire Hadoop project, ie all subprojects?
>
> If the vote passes, current and future committers will gain commit
> rights in all current (Common, HDFS, MapReduce, YARN) and future
> Hadoop subprojects. The specific change to the bylaws follows:
>
> Index: main/author/src/documentation/content/xdocs/bylaws.xml
> ==================================================================> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision 1376823)
> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working copy)
> @@ -64,10 +64,9 @@
>     <li> <strong>Committers</strong>
>
>          <p>The project's Committers are responsible for the project's
> -        technical management. Committers have access to a specified
> -        set of subprojects' subversion repositories. Committers on
> -        subprojects may cast binding votes on any technical discussion
> -        regarding that subproject.</p>
> +        technical management. Committers have access to all subproject
> +        subversion repositories. Committers may cast binding votes on
> +        any technical discussion regarding that subproject.</p>
>
>
> This vote will run for 7 days. Since the vote represents a
> modification of the bylawys [2] it requires a lazy majority of active
> PMC members.
>
> 1. http://s.apache.org/WW1
> 2. http://hadoop.apache.org/bylaws.html
>
> Here's my +1

+1

Make sense.
Committers knows their limits to commit/review the specific part of the code area where they are familiar in subprojects. So,  no issues in this proposal to make a single list.
IMO, ok to maintain single list, until we plan to make separate releases of each sub project.

Thanks,
Uma
________________________________________
From: Eli Collins [[EMAIL PROTECTED]]
Sent: Saturday, August 25, 2012 4:23 AM
To: [EMAIL PROTECTED]
Subject: Re: [VOTE] Maintain a single committer list for the Hadoop project

On Thu, Aug 23, 2012 at 9:20 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
> Per this thread [1] should we have a single set of committers for the
> entire Hadoop project, ie all subprojects?
>
> If the vote passes, current and future committers will gain commit
> rights in all current (Common, HDFS, MapReduce, YARN) and future
> Hadoop subprojects. The specific change to the bylaws follows:
>
> Index: main/author/src/documentation/content/xdocs/bylaws.xml
> ==================================================================> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision 1376823)
> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working copy)
> @@ -64,10 +64,9 @@
>     <li> <strong>Committers</strong>
>
>          <p>The project's Committers are responsible for the project's
> -        technical management. Committers have access to a specified
> -        set of subprojects' subversion repositories. Committers on
> -        subprojects may cast binding votes on any technical discussion
> -        regarding that subproject.</p>
> +        technical management. Committers have access to all subproject
> +        subversion repositories. Committers may cast binding votes on
> +        any technical discussion regarding that subproject.</p>
>

Andy Isaacson pointed out that I should replace "that" with "any"
above in "regarding that subproject" to make this more clear.  Here's
the updated diff, same intent.

site $ svn diff
Index: main/author/src/documentation/content/xdocs/bylaws.xml
==================================================================--- main/author/src/documentation/content/xdocs/bylaws.xml      (revision 1376823)
+++ main/author/src/documentation/content/xdocs/bylaws.xml      (working copy)
@@ -64,10 +64,9 @@
    <li> <strong>Committers</strong>

         <p>The project's Committers are responsible for the project's
-        technical management. Committers have access to a specified
-        set of subprojects' subversion repositories. Committers on
-        subprojects may cast binding votes on any technical discussion
-        regarding that subproject.</p>
+        technical management. Committers have access to all subproject
+        subversion repositories. Committers may cast binding votes on
+        any technical discussion regarding any subproject.</p>

+1

Tom

On Fri, Aug 24, 2012 at 5:20 AM, Eli Collins <[EMAIL PROTECTED]> wrote:
> Per this thread [1] should we have a single set of committers for the
> entire Hadoop project, ie all subprojects?
>
> If the vote passes, current and future committers will gain commit
> rights in all current (Common, HDFS, MapReduce, YARN) and future
> Hadoop subprojects. The specific change to the bylaws follows:
>
> Index: main/author/src/documentation/content/xdocs/bylaws.xml
> ==================================================================> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision 1376823)
> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working copy)
> @@ -64,10 +64,9 @@
>     <li> <strong>Committers</strong>
>
>          <p>The project's Committers are responsible for the project's
> -        technical management. Committers have access to a specified
> -        set of subprojects' subversion repositories. Committers on
> -        subprojects may cast binding votes on any technical discussion
> -        regarding that subproject.</p>
> +        technical management. Committers have access to all subproject
> +        subversion repositories. Committers may cast binding votes on
> +        any technical discussion regarding that subproject.</p>
>
>
> This vote will run for 7 days. Since the vote represents a
> modification of the bylawys [2] it requires a lazy majority of active
> PMC members.
>
> 1. http://s.apache.org/WW1
> 2. http://hadoop.apache.org/bylaws.html
>
> Here's my +1

I feel this proposal is taking us in the wrong direction.

Apache Board has encouraged us to move to TLPs. We did follow through partially with Pig, Hive, HBase. Chris M, in the other thread, has made good arguments, drawing from the larger apache experience,  on why the remaining hadoop sub projects should follow the Apache way and move to TLPs.  The communities for HDFS and MR are quite distinct when you examine the contributions of each committer or who participates in design discussions for each of the sub-projects.

I would like to see a discussion on a plan to get us to TLPs.
Hence my vote is -1;   lets have the TLP discussion first because the vote on the proposal in this thread is taking taking us in the opposite direction.

sanjay

On Aug 23, 2012, at 9:20 PM, Eli Collins wrote:

> Per this thread [1] should we have a single set of committers for the
> entire Hadoop project, ie all subprojects?
>
> If the vote passes, current and future committers will gain commit
> rights in all current (Common, HDFS, MapReduce, YARN) and future
> Hadoop subprojects. The specific change to the bylaws follows:
>
> Index: main/author/src/documentation/content/xdocs/bylaws.xml
> ==================================================================> --- main/author/src/documentation/content/xdocs/bylaws.xml (revision 1376823)
> +++ main/author/src/documentation/content/xdocs/bylaws.xml (working copy)
> @@ -64,10 +64,9 @@
>    <li> <strong>Committers</strong>
>
>         <p>The project's Committers are responsible for the project's
> -        technical management. Committers have access to a specified
> -        set of subprojects' subversion repositories. Committers on
> -        subprojects may cast binding votes on any technical discussion
> -        regarding that subproject.</p>
> +        technical management. Committers have access to all subproject
> +        subversion repositories. Committers may cast binding votes on
> +        any technical discussion regarding that subproject.</p>
>
>
> This vote will run for 7 days. Since the vote represents a
> modification of the bylawys [2] it requires a lazy majority of active
> PMC members.
>
> 1. http://s.apache.org/WW1
> 2. http://hadoop.apache.org/bylaws.html
>
> Here's my +1

It has been clear for a long time that HDFS & MapReduce were largely
disjoint developer communities. We set up the sub-projects to recognize
that fact and allow each community to focus on its own issues. I think
merging the committer lists is moving the project the wrong direction.

-1 to merging the committer lists

On Aug 23, 2012, at 9:20 PM, Eli Collins wrote:

> Per this thread [1] should we have a single set of committers for the
> entire Hadoop project, ie all subprojects?

I feel like we need to have a wider discussion here.

This discussion started when a diverse set of folks working on YARN for a year and a half wanted their own identity and an acknowledgement of the fact that they are a distinct community. In retrospect, I went about convincing the wider Hadoop community about this in the wrong way. My apologies.

Upon reflection, I think Chris Mattman has convinced me that we have an even wider issue at hand and that the right way to a better place, not just for YARN, but for all of Hadoop, is to expedite the process of graduating Hadoop sub-projects into TLPs. This is a mere reflection of the fact that Hadoop is not a single community.

Historically there have been at least 2 communities (HDFS, MapReduce) under the Hadoop umbrella; and there now 3 (HDFS, MapReduce, YARN).
At least for the last 3 years, if not more, the overwhelming majority of contributors to Hadoop have focussed exclusively on one of the sub-projects. That is a clear indicator.
This is exactly the thinking behind graduating former sub-projects like HBase, Hive & Pig graduating, upon the nudge received by the Hadoop PMC from the Board.

The good news is that, in principle, most seem to agree on the need for Hadoop sub-projects to stand alone and the path to get there. It could lead to several great outcomes such as ensuring HDFS pays equal attention to HBase as MapReduce, YARN pays attention to projects beyond MapReduce etc. by not tying them together.

Rather than sweep this under the carpet, I feel we are better off acknowledging this.

This is very much in keeping with the way the ASF and the Board wants to see communities - small and focussed on a single project.

A meta or umbrella community like Hadoop leads to issues which are well documented and understood in the ASF, something experienced Apache Members like Chris Mattman have repeatedly pointed out.

It is also fair, per Chris Douglas, to set a reasonable time frame. After due consideration, I think doing this before hadoop-2 is declared stable (GA) is the most reasonable option. It gives us necessary headroom hereupon and will ensure we don't confuse users further by doing it post-fact hadoop-2. Let's discuss the mechanics, timelines etc. further.

Yes, this is hard work and there are several technical challenges. But, the ASF is all about communities and I'm sure we can solve these technical issues for a better long-term health of these distinct communities.

Thoughts?

thanks,
Arun

On Tue, Aug 28, 2012 at 4:12 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> On Aug 23, 2012, at 9:20 PM, Eli Collins wrote:
>
>> Per this thread [1] should we have a single set of committers for the
>> entire Hadoop project, ie all subprojects?
>
> I feel like we need to have a wider discussion here.
>
> This discussion started when a diverse set of folks working on YARN for a year and a half wanted their own identity and an acknowledgement of the fact that they are a distinct community. In retrospect, I went about convincing the wider Hadoop community about this in the wrong way. My apologies.
>
> Upon reflection, I think Chris Mattman has convinced me that we have an even wider issue at hand and that the right way to a better place, not just for YARN, but for all of Hadoop, is to expedite the process of graduating Hadoop sub-projects into TLPs. This is a mere reflection of the fact that Hadoop is not a single community.
>
> Historically there have been at least 2 communities (HDFS, MapReduce) under the Hadoop umbrella; and there now 3 (HDFS, MapReduce, YARN).
> At least for the last 3 years, if not more, the overwhelming majority of contributors to Hadoop have focussed exclusively on one of the sub-projects. That is a clear indicator.
> This is exactly the thinking behind graduating former sub-projects like HBase, Hive & Pig graduating, upon the nudge received by the Hadoop PMC from the Board.
>
> The good news is that, in principle, most seem to agree on the need for Hadoop sub-projects to stand alone and the path to get there. It could lead to several great outcomes such as ensuring HDFS pays equal attention to HBase as MapReduce, YARN pays attention to projects beyond MapReduce etc. by not tying them together.
>
> Rather than sweep this under the carpet, I feel we are better off acknowledging this.
>
> This is very much in keeping with the way the ASF and the Board wants to see communities - small and focussed on a single project.
>
> A meta or umbrella community like Hadoop leads to issues which are well documented and understood in the ASF, something experienced Apache Members like Chris Mattman have repeatedly pointed out.
>
> It is also fair, per Chris Douglas, to set a reasonable time frame. After due consideration, I think doing this before hadoop-2 is declared stable (GA) is the most reasonable option. It gives us necessary headroom hereupon and will ensure we don't confuse users further by doing it post-fact hadoop-2. Let's discuss the mechanics, timelines etc. further.
>
> Yes, this is hard work and there are several technical challenges. But, the ASF is all about communities and I'm sure we can solve these technical issues for a better long-term health of these distinct communities.
>
> Thoughts?

I'd start a separate discussion thread or vote about moving some or
all of the sub-projects to TLPs. IMO we should resolve this issue
independently - there's no reason to block this decision on a possible
future direction for the project. For example if YARN spins out as a
TLP this issue still remains for the rest of the sub-projects, so I
don't want to stall progress on this on the larger more complex
discussion of whether all projects become TLPs. And if a sub-project
spins out as a TLP that's a great opportunity to figure out the right
set of committers. Ie the decision here doesn't prevent YARN from
establishing a new committer lists if/when it spins out.

Thanks,
Eli

Hi Eli,

On Aug 28, 2012, at 4:57 PM, Eli Collins wrote:

>> [...snip...]
>>
>> Thoughts?
>
> I'd start a separate discussion thread or vote about moving some or
> all of the sub-projects to TLPs. IMO we should resolve this issue
> independently - there's no reason to block this decision on a possible
> future direction for the project.

I think it's a mischaracterization to suggest that Arun's proposal is a future
direction for the project. The project is already there and has been there for
a while, during this time trying to identify itself as one project when in reality
it's been many. That's the cause of issues like this and all the email bandwidth.

> For example if YARN spins out as a
> TLP this issue still remains for the rest of the sub-projects, so I
> don't want to stall progress on this on the larger more complex
> discussion of whether all projects become TLPs.

Incrementally spinning out the projects is fine. Concretely, if each project, HDFS, MR, and YARN each did:

svn copy -m "HDFS TLP." https://svn.apache.org/repos/asf/hadoop/ https://svn.apache.org/repos/asf/<insert cool MR name>
svn copy -m "HDFS TLP." https://svn.apache.org/repos/asf/hadoop/ https://svn.apache.org/repos/asf/<insert cool YARN name>
svn copy -m "HDFS TLP." https://svn.apache.org/repos/asf/hadoop/ https://svn.apache.org/repos/asf/<insert cool HDFS name>

That would probably be a fine starting point for each project, so long as the goal is to turn
each one of those destination paths into distinct entities, and to remove code duplication.
IOW, I think it's perfectly agreeable to do the above, so long as there is the intention to get
to a point that's independent -- don't use independence as the required starting point b/c
I'm not sure you guys will ever fully get there.

> And if a sub-project
> spins out as a TLP that's a great opportunity to figure out the right
> set of committers.

Agreed, you can do that now, not needed in this VOTE - which seems to be
trying to deal with something implicitly again when there is an explicit means
to deal with it.

I would suggest a nominal process for creating TLP:

0. [DISCUSS] thread for <TLP name> in which you talk about #1 and #2 below, potentially draft resolution too.

1. Decide on an initial set of *PMC* members. I urge each new TLP to not drawn distinctions
between committers and PMC members -- in each and every Apache project over the years in which I've seen this done
all it does is create unnecessary drama, and all it does it put an extra chip on folks'
shoulders. Sure, VOTE them in as a committer -- they can modify the code but not have a
VOTE on adding new committers or on the bits they release. Huh? Make them wait another year, six months, before they get
that bit. Huh?

That just doesn't make sense to me. PMC==C makes people feel equal amongst their peers (which they are) --
and peers at Apache are really the people that are doing the work. To draw other distinctive
lines is artificial IMHO. This is a foundation built on trust and most of these communities that themselves
are distinct already have that trust -- it's just being walled up right now. Furthermore all the talk I've seen
in the past within Hadoop about being "worried" about having to deal with people's work in case they screw
up as a PMC member or committer, or having to track down bugs, etc., is exacerbated by the work that it
takes for you guys to banter back and forth in emails discussing committer lists, artificial project boundaries,
and in reviewing committer A's work, committer B's work (neither of whom can VOTE on the bits they release),
etc., etc.. Technical issues used to justify community segregation.

2. Decide on a chair. Try not to VOTE for this explicitly, see if can be discussed and consensus
can be reached (just a thought experiment). VOTE if necessary.

3. [VOTE] thread for <TLP name>

4. paste resolution from #0 to board@.

5. infrastructure set up.

6. TLPs proceed, collaborate, operate as distinct communities, and try to solve the code duplication/dependency
issues from there.

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

BTW, one amendment to my proposed process below:

On Aug 28, 2012, at 6:37 PM, Mattmann, Chris A (388J) wrote:

> [...snip...]
>
> 4. paste resolution from #0 to board@.

An alternative to #4 is to go through the Incubator, which might be good to get other
Foundation members on the PMC that are outside of Apache Hadoop and that themselves
have experience in other communities, and to see how those Foundation members may
help the new Hadoop projects learn about the way other communities at the Foundation
work.

This quite possibly could be great cross fertilization and tutelage that the
new (eventual) TLPs could benefit from.

Post facto this alternative #4, we jump to #6, except instead of TLPs, the new
projects operate as Apache Incubator podlings, use the Incubator for its mentorship,
metrics on healthy projects, etc.

This would also be a good way to go.

>
> 5. infrastructure set up.
>
> 6. TLPs proceed, collaborate, operate as distinct communities, and try to solve the code duplication/dependency
> issues from there.

Then there is a new step here in that scenario:

7. Graduate to TLP from Incubator via board resolution.

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Sorry just being complete here:

On Aug 28, 2012, at 6:37 PM, Mattmann, Chris A (388J) wrote:
>
>
>> For example if YARN spins out as a
>> TLP this issue still remains for the rest of the sub-projects, so I
>> don't want to stall progress on this on the larger more complex
>> discussion of whether all projects become TLPs.
>
> Incrementally spinning out the projects is fine. Concretely, if each project, HDFS, MR, and YARN each did:
>
> svn copy -m "HDFS TLP." https://svn.apache.org/repos/asf/hadoop/ https://svn.apache.org/repos/asf/<insert cool MR name>
> svn copy -m "HDFS TLP." https://svn.apache.org/repos/asf/hadoop/ https://svn.apache.org/repos/asf/<insert cool YARN name>
> svn copy -m "HDFS TLP." https://svn.apache.org/repos/asf/hadoop/ https://svn.apache.org/repos/asf/<insert cool HDFS name>
>

I meant to say:

> svn copy -m "MR TLP." https://svn.apache.org/repos/asf/hadoop/ https://svn.apache.org/repos/asf/<insert cool MR name>
> svn copy -m "YARN TLP." https://svn.apache.org/repos/asf/hadoop/ https://svn.apache.org/repos/asf/<insert cool YARN name>
> svn copy -m "HDFS TLP." https://svn.apache.org/repos/asf/hadoop/ https://svn.apache.org/repos/asf/<insert cool HDFS name>
And to finish it off with:

svn remove -m "Remove Hadoop umbrella TLP. Split into separate projects." https://svn.apache.org/repos/asf/hadoop

Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [EMAIL PROTECTED]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

On Tue, Aug 28, 2012 at 6:37 PM, Mattmann, Chris A (388J)
<[EMAIL PROTECTED]> wrote:
> Hi Eli,
>
> On Aug 28, 2012, at 4:57 PM, Eli Collins wrote:
>
>>> [...snip...]
>>>
>>> Thoughts?
>>
>> I'd start a separate discussion thread or vote about moving some or
>> all of the sub-projects to TLPs. IMO we should resolve this issue
>> independently - there's no reason to block this decision on a possible
>> future direction for the project.
>
> I think it's a mischaracterization to suggest that Arun's proposal is a future
> direction for the project. The project is already there and has been there for
> a while, during this time trying to identify itself as one project when in reality
> it's been many. That's the cause of issues like this and all the email bandwidth.
>

We've talked about eventually moving the sub-projects to TLPs, so
perhaps it's more of a "future state" than a "future direction".
However it's also not that cut and dry, after all the sub-project
discussion and the "project split" we've actually started going the
other direction in some ways, specifically, we've
- Merged all the code bases to a single repository, shared more source code.
- Decided to continue to release all components together.
- Merged the user community to a single list.
- Started YARN in the Hadoop project rather than the incubator

Anyway, seems like mincing words, and a vote thread isn't a great
place to discuss specifics of one particular TLP proposal. If you
think the time is right for the TLP and you've got a particular
proposal sounds like you should start a DISCUSS thread for that
proposal.

Thanks,
Eli

-1

Given the other discussion related to TLP and the proposal here is
essentially going in the opposite direction.

On Tue, Aug 28, 2012 at 7:09 PM, Eli Collins <[EMAIL PROTECTED]> wrote:

> On Tue, Aug 28, 2012 at 6:37 PM, Mattmann, Chris A (388J)
> <[EMAIL PROTECTED]> wrote:
> > Hi Eli,
> >
> > On Aug 28, 2012, at 4:57 PM, Eli Collins wrote:
> >
> >>> [...snip...]
> >>>
> >>> Thoughts?
> >>
> >> I'd start a separate discussion thread or vote about moving some or
> >> all of the sub-projects to TLPs. IMO we should resolve this issue
> >> independently - there's no reason to block this decision on a possible
> >> future direction for the project.
> >
> > I think it's a mischaracterization to suggest that Arun's proposal is a
> future
> > direction for the project. The project is already there and has been
> there for
> > a while, during this time trying to identify itself as one project when
> in reality
> > it's been many. That's the cause of issues like this and all the email
> bandwidth.
> >
>
> We've talked about eventually moving the sub-projects to TLPs, so
> perhaps it's more of a "future state" than a "future direction".
> However it's also not that cut and dry, after all the sub-project
> discussion and the "project split" we've actually started going the
> other direction in some ways, specifically, we've
> - Merged all the code bases to a single repository, shared more source
> code.
> - Decided to continue to release all components together.
> - Merged the user community to a single list.
> - Started YARN in the Hadoop project rather than the incubator
>
> Anyway, seems like mincing words, and a vote thread isn't a great
> place to discuss specifics of one particular TLP proposal. If you
> think the time is right for the TLP and you've got a particular
> proposal sounds like you should start a DISCUSS thread for that
> proposal.
>
> Thanks,
> Eli
>

--
http://hortonworks.com/download/

+1

--
Aaron T. Myers
Software Engineer, Cloudera

On Thu, Aug 23, 2012 at 9:20 PM, Eli Collins <[EMAIL PROTECTED]> wrote:

> Per this thread [1] should we have a single set of committers for the
> entire Hadoop project, ie all subprojects?
>
> If the vote passes, current and future committers will gain commit
> rights in all current (Common, HDFS, MapReduce, YARN) and future
> Hadoop subprojects. The specific change to the bylaws follows:
>
> Index: main/author/src/documentation/content/xdocs/bylaws.xml
> ==================================================================> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision
> 1376823)
> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working
> copy)
> @@ -64,10 +64,9 @@
>     <li> <strong>Committers</strong>
>
>          <p>The project's Committers are responsible for the project's
> -        technical management. Committers have access to a specified
> -        set of subprojects' subversion repositories. Committers on
> -        subprojects may cast binding votes on any technical discussion
> -        regarding that subproject.</p>
> +        technical management. Committers have access to all subproject
> +        subversion repositories. Committers may cast binding votes on
> +        any technical discussion regarding that subproject.</p>
>
>
> This vote will run for 7 days. Since the vote represents a
> modification of the bylawys [2] it requires a lazy majority of active
> PMC members.
>
> 1. http://s.apache.org/WW1
> 2. http://hadoop.apache.org/bylaws.html
>
> Here's my +1
>

Given that a discussion of a TLP proposal is happening in parallel, and that there seems to be a positive response there, is this vote still happening?

At any rate, here's my opinion: The earlier project split, however painful it was, has done a lot of good, both in terms of project management as well as community focus. Leaving common project aside, merging the committer lists back is going to submerge the underlying issue of communities, that have acquired sufficient independence, attempting to function together and is also likely to make us depend on informal behavior of trusting someone to not commit code that they aren't really responsible for.

This is definitely going in the opposite direction of our long term goals.

-1.

Thanks,
+Vinod

On Thu, Aug 30, 2012 at 9:20 AM, Vinod Kumar Vavilapalli
<[EMAIL PROTECTED]> wrote:
>
> Given that a discussion of a TLP proposal is happening in parallel, and that there seems to be a positive response there, is this vote still happening?
>

Yes, related discussions don't stop votes.  Plenty of people don't
want to see TLPs, and some people want to see a combined committer
list even if we eventually do go to the TLP route.

Thanks,
Eli

With 12 +1s (9 binding) and 3 -1s (binding) the vote passes.

I'll update the bylaws to reflect the merged committer lists
(http://s.apache.org/Owx).

Thanks,
Eli

On Thu, Aug 23, 2012 at 9:20 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
> Per this thread [1] should we have a single set of committers for the
> entire Hadoop project, ie all subprojects?
>
> If the vote passes, current and future committers will gain commit
> rights in all current (Common, HDFS, MapReduce, YARN) and future
> Hadoop subprojects. The specific change to the bylaws follows:
>
> Index: main/author/src/documentation/content/xdocs/bylaws.xml
> ==================================================================> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision 1376823)
> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working copy)
> @@ -64,10 +64,9 @@
>     <li> <strong>Committers</strong>
>
>          <p>The project's Committers are responsible for the project's
> -        technical management. Committers have access to a specified
> -        set of subprojects' subversion repositories. Committers on
> -        subprojects may cast binding votes on any technical discussion
> -        regarding that subproject.</p>
> +        technical management. Committers have access to all subproject
> +        subversion repositories. Committers may cast binding votes on
> +        any technical discussion regarding that subproject.</p>
>
>
> This vote will run for 7 days. Since the vote represents a
> modification of the bylawys [2] it requires a lazy majority of active
> PMC members.
>
> 1. http://s.apache.org/WW1
> 2. http://hadoop.apache.org/bylaws.html
>
> Here's my +1

On Aug 30, 2012, at 9:30 PM, Eli Collins wrote:

> With 12 +1s (9 binding) and 3 -1s (binding) the vote passes.
>
> I'll update the bylaws to reflect the merged committer lists
> (http://s.apache.org/Owx).
>

Also, could you please update the site to reflect this?

thanks,
Arun

> Thanks,
> Eli
>
> On Thu, Aug 23, 2012 at 9:20 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
>> Per this thread [1] should we have a single set of committers for the
>> entire Hadoop project, ie all subprojects?
>>
>> If the vote passes, current and future committers will gain commit
>> rights in all current (Common, HDFS, MapReduce, YARN) and future
>> Hadoop subprojects. The specific change to the bylaws follows:
>>
>> Index: main/author/src/documentation/content/xdocs/bylaws.xml
>> ==================================================================>> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision 1376823)
>> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working copy)
>> @@ -64,10 +64,9 @@
>>    <li> <strong>Committers</strong>
>>
>>         <p>The project's Committers are responsible for the project's
>> -        technical management. Committers have access to a specified
>> -        set of subprojects' subversion repositories. Committers on
>> -        subprojects may cast binding votes on any technical discussion
>> -        regarding that subproject.</p>
>> +        technical management. Committers have access to all subproject
>> +        subversion repositories. Committers may cast binding votes on
>> +        any technical discussion regarding that subproject.</p>
>>
>>
>> This vote will run for 7 days. Since the vote represents a
>> modification of the bylawys [2] it requires a lazy majority of active
>> PMC members.
>>
>> 1. http://s.apache.org/WW1
>> 2. http://hadoop.apache.org/bylaws.html
>>
>> Here's my +1

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

I've updated asf-authorization-template to reflect this:

$ svn log -r 830654
------------------------------------------------------------------------
r830654 | acmurthy | 2012-08-31 09:02:23 -0700 (Fri, 31 Aug 2012) | 1 line

Merge hadoop-hdfs & hadoop-mapreduce lists per http://s.apache.org/pn
------------------------------------------------------------------------

Arun

On Aug 30, 2012, at 9:30 PM, Eli Collins wrote:

> With 12 +1s (9 binding) and 3 -1s (binding) the vote passes.
>
> I'll update the bylaws to reflect the merged committer lists
> (http://s.apache.org/Owx).
>
> Thanks,
> Eli
>
> On Thu, Aug 23, 2012 at 9:20 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
>> Per this thread [1] should we have a single set of committers for the
>> entire Hadoop project, ie all subprojects?
>>
>> If the vote passes, current and future committers will gain commit
>> rights in all current (Common, HDFS, MapReduce, YARN) and future
>> Hadoop subprojects. The specific change to the bylaws follows:
>>
>> Index: main/author/src/documentation/content/xdocs/bylaws.xml
>> ==================================================================>> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision 1376823)
>> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working copy)
>> @@ -64,10 +64,9 @@
>>    <li> <strong>Committers</strong>
>>
>>         <p>The project's Committers are responsible for the project's
>> -        technical management. Committers have access to a specified
>> -        set of subprojects' subversion repositories. Committers on
>> -        subprojects may cast binding votes on any technical discussion
>> -        regarding that subproject.</p>
>> +        technical management. Committers have access to all subproject
>> +        subversion repositories. Committers may cast binding votes on
>> +        any technical discussion regarding that subproject.</p>
>>
>>
>> This vote will run for 7 days. Since the vote represents a
>> modification of the bylawys [2] it requires a lazy majority of active
>> PMC members.
>>
>> 1. http://s.apache.org/WW1
>> 2. http://hadoop.apache.org/bylaws.html
>>
>> Here's my +1

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

On Fri, Aug 31, 2012 at 9:04 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> On Aug 30, 2012, at 9:30 PM, Eli Collins wrote:
>
>> With 12 +1s (9 binding) and 3 -1s (binding) the vote passes.
>>
>> I'll update the bylaws to reflect the merged committer lists
>> (http://s.apache.org/Owx).
>>
>
> Also, could you please update the site to reflect this?
>

Yes, I filed HADOOP-8752.
> thanks,
> Arun
>
>> Thanks,
>> Eli
>>
>> On Thu, Aug 23, 2012 at 9:20 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
>>> Per this thread [1] should we have a single set of committers for the
>>> entire Hadoop project, ie all subprojects?
>>>
>>> If the vote passes, current and future committers will gain commit
>>> rights in all current (Common, HDFS, MapReduce, YARN) and future
>>> Hadoop subprojects. The specific change to the bylaws follows:
>>>
>>> Index: main/author/src/documentation/content/xdocs/bylaws.xml
>>> ==================================================================>>> --- main/author/src/documentation/content/xdocs/bylaws.xml      (revision 1376823)
>>> +++ main/author/src/documentation/content/xdocs/bylaws.xml      (working copy)
>>> @@ -64,10 +64,9 @@
>>>    <li> <strong>Committers</strong>
>>>
>>>         <p>The project's Committers are responsible for the project's
>>> -        technical management. Committers have access to a specified
>>> -        set of subprojects' subversion repositories. Committers on
>>> -        subprojects may cast binding votes on any technical discussion
>>> -        regarding that subproject.</p>
>>> +        technical management. Committers have access to all subproject
>>> +        subversion repositories. Committers may cast binding votes on
>>> +        any technical discussion regarding that subproject.</p>
>>>
>>>
>>> This vote will run for 7 days. Since the vote represents a
>>> modification of the bylawys [2] it requires a lazy majority of active
>>> PMC members.
>>>
>>> 1. http://s.apache.org/WW1
>>> 2. http://hadoop.apache.org/bylaws.html
>>>
>>> Here's my +1
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>