Per the discussion thread: http://s.apache.org/XkY

Should HDFS and MapReduce committers lists be combined and all
subsequent committers on either of these two projects be granted karma
in the other?

If the vote passes, current and future committers to MapReduce and
HDFS will gain commit rights in both projects. Commit rights to Common
are unaffected.

Without bylaws, a 2/3 majority for a committer import seems like a
reasonable bar, given that adding an individual committer requires
consensus.

----

Owen has started a separate voting thread, proposing to define the
Common committer list as the union of HDFS and MapReduce committers
(vote A), so I tried to write this (vote B) so it would not conflict.
As I'm reading it:

A passes, B passes: One can become a committer on HDFS or MapReduce.
Commit to either implies commit on HDFS, MR, and Common.
A passes, B fails: One can become a committer on HDFS or MapReduce.
Commit to either implies commit on Common, only.
A fails, B passes: One can become a committer on HDFS, MapReduce, or
Common. Commit to to HDFS/MR implies converse, but individual
appointments to Common continue.
A fails, B fails: Committers continue to be appointed individually to
HDFS, MapReduce, and Common.

In no scheme would commit rights to Common imply commit rights to
either HDFS or MapReduce, I guess. -C

On Tue, Aug 17, 2010 at 3:15 AM, Chris Douglas <[EMAIL PROTECTED]> wrote:
> Per the discussion thread: http://s.apache.org/XkY
>
> Should HDFS and MapReduce committers lists be combined and all
> subsequent committers on either of these two projects be granted karma
> in the other?

+1

Tom

On Tue, Aug 17, 2010 at 3:15 AM, Chris Douglas <[EMAIL PROTECTED]> wrote:
> Per the discussion thread: http://s.apache.org/XkY
>
> Should HDFS and MapReduce committers lists be combined and all
> subsequent committers on either of these two projects be granted karma
> in the other?

+1      (non-binding)

Thanks,
Eli

+1

On 08/17/2010 03:15 AM, Chris Douglas wrote:
> Per the discussion thread: http://s.apache.org/XkY
>
> Should HDFS and MapReduce committers lists be combined and all
> subsequent committers on either of these two projects be granted karma
> in the other?
>
> If the vote passes, current and future committers to MapReduce and
> HDFS will gain commit rights in both projects. Commit rights to Common
> are unaffected.
>
> Without bylaws, a 2/3 majority for a committer import seems like a
> reasonable bar, given that adding an individual committer requires
> consensus.
>
> ----
>
> Owen has started a separate voting thread, proposing to define the
> Common committer list as the union of HDFS and MapReduce committers
> (vote A), so I tried to write this (vote B) so it would not conflict.
> As I'm reading it:
>
> A passes, B passes: One can become a committer on HDFS or MapReduce.
> Commit to either implies commit on HDFS, MR, and Common.
> A passes, B fails: One can become a committer on HDFS or MapReduce.
> Commit to either implies commit on Common, only.
> A fails, B passes: One can become a committer on HDFS, MapReduce, or
> Common. Commit to to HDFS/MR implies converse, but individual
> appointments to Common continue.
> A fails, B fails: Committers continue to be appointed individually to
> HDFS, MapReduce, and Common.
>
> In no scheme would commit rights to Common imply commit rights to
> either HDFS or MapReduce, I guess. -C

-1

I think it is clear that separate communities are developing for HDFS
and MapReduce (unlike Common) and will further develop if we allow
them to. There will continue to be significant overlap between those
communities, but I think it is a useful distinction and matches the
way that users think about the projects.

-- Owen

-1

I agree, we need to allow HDFS and Map-Reduce to further develop into  
coherent, independent communities. This seems like a step backwards.

Arun

On Aug 17, 2010, at 12:39 PM, Owen O'Malley wrote:

> -1
>
> I think it is clear that separate communities are developing for HDFS
> and MapReduce (unlike Common) and will further develop if we allow
> them to. There will continue to be significant overlap between those
> communities, but I think it is a useful distinction and matches the
> way that users think about the projects.
>
> -- Owen

+1

On Aug 17, 2010, at 3:15 AM, Chris Douglas wrote:

> Per the discussion thread: http://s.apache.org/XkY
>
> Should HDFS and MapReduce committers lists be combined and all
> subsequent committers on either of these two projects be granted karma
> in the other?
>
> If the vote passes, current and future committers to MapReduce and
> HDFS will gain commit rights in both projects. Commit rights to Common
> are unaffected.
>
> Without bylaws, a 2/3 majority for a committer import seems like a
> reasonable bar, given that adding an individual committer requires
> consensus.
>
> ----
>
> Owen has started a separate voting thread, proposing to define the
> Common committer list as the union of HDFS and MapReduce committers
> (vote A), so I tried to write this (vote B) so it would not conflict.
> As I'm reading it:
>
> A passes, B passes: One can become a committer on HDFS or MapReduce.
> Commit to either implies commit on HDFS, MR, and Common.
> A passes, B fails: One can become a committer on HDFS or MapReduce.
> Commit to either implies commit on Common, only.
> A fails, B passes: One can become a committer on HDFS, MapReduce, or
> Common. Commit to to HDFS/MR implies converse, but individual
> appointments to Common continue.
> A fails, B fails: Committers continue to be appointed individually to
> HDFS, MapReduce, and Common.
>
> In no scheme would commit rights to Common imply commit rights to
> either HDFS or MapReduce, I guess. -C

-1. After thinking through, I do feel that there is and will be a
trend where more folks will focus on only one of the two projects. I
certainly know of several Map/Reduce committers who are not active on
HDFS and vice-versa, and therefore couldn't qualify to have commit
rights on both projects.

Thanks
Hemanth

On Tue, Aug 17, 2010 at 3:45 PM, Chris Douglas <[EMAIL PROTECTED]> wrote:
> Per the discussion thread: http://s.apache.org/XkY
>
> Should HDFS and MapReduce committers lists be combined and all
> subsequent committers on either of these two projects be granted karma
> in the other?
>
> If the vote passes, current and future committers to MapReduce and
> HDFS will gain commit rights in both projects. Commit rights to Common
> are unaffected.
>
> Without bylaws, a 2/3 majority for a committer import seems like a
> reasonable bar, given that adding an individual committer requires
> consensus.
>
> ----
>
> Owen has started a separate voting thread, proposing to define the
> Common committer list as the union of HDFS and MapReduce committers
> (vote A), so I tried to write this (vote B) so it would not conflict.
> As I'm reading it:
>
> A passes, B passes: One can become a committer on HDFS or MapReduce.
> Commit to either implies commit on HDFS, MR, and Common.
> A passes, B fails: One can become a committer on HDFS or MapReduce.
> Commit to either implies commit on Common, only.
> A fails, B passes: One can become a committer on HDFS, MapReduce, or
> Common. Commit to to HDFS/MR implies converse, but individual
> appointments to Common continue.
> A fails, B fails: Committers continue to be appointed individually to
> HDFS, MapReduce, and Common.
>
> In no scheme would commit rights to Common imply commit rights to
> either HDFS or MapReduce, I guess. -C
>

On 17/08/10 11:15, Chris Douglas wrote:
> Per the discussion thread: http://s.apache.org/XkY
>
> Should HDFS and MapReduce committers lists be combined and all
> subsequent committers on either of these two projects be granted karma
> in the other?

0

I don't know.

A big driver for the split was the volume of emails, for a dev you had
to track lots of jira issues across the board. Having the split has been
bad for me in that I've got lazy about tracking everything, which is
what you need to do to understand what's going on. Question is -how many
people need to understand all that's going on? And of those people who
are trying to do this, how do they get time for anything else?

I do think having some separate user groups where hdfs and mapreduce
issues can be discussed is good, so some separation there does seem to
help, though it's not clear for new users which list to go to.

-steve

-1

Most committers work on either HDFS or MapReduce in general.  For the cases that
contributors may provide patches on both projects, those people will gain
committer accesses on both projects.  For the occasional cases that some issues
may require patches to be committed on both projects, it totally makes sense to
find a committer on each project reviewing and committing those patches.

Nicholas

----- Original Message ----
> From: Chris Douglas <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Tue, August 17, 2010 3:15:47 AM
> Subject: [VOTE] Combine MapReduce/HDFS Committers
>
> Per the discussion thread: http://s.apache.org/XkY
>
> Should HDFS and  MapReduce committers lists be combined and all
> subsequent committers on  either of these two projects be granted karma
> in the other?

I have a technical question if this vote fails.  Since we still release MR and HDFS together as Hadoop, does this mean that only committers who can commit to both projects can then make a release candidate of Hadoop?  Do we end up with MR committers, HDFS committers, and Hadoop committers?

Nige

On Aug 18, 2010, at 8:31 AM, Tsz Wo (Nicholas), Sze wrote:

> -1
>
> Most committers work on either HDFS or MapReduce in general.  For the cases that
> contributors may provide patches on both projects, those people will gain
> committer accesses on both projects.  For the occasional cases that some issues
> may require patches to be committed on both projects, it totally makes sense to
> find a committer on each project reviewing and committing those patches.
>
> Nicholas
>
>
>
> ----- Original Message ----
>> From: Chris Douglas <[EMAIL PROTECTED]>
>> To: [EMAIL PROTECTED]
>> Sent: Tue, August 17, 2010 3:15:47 AM
>> Subject: [VOTE] Combine MapReduce/HDFS Committers
>>
>> Per the discussion thread: http://s.apache.org/XkY
>>
>> Should HDFS and  MapReduce committers lists be combined and all
>> subsequent committers on  either of these two projects be granted karma
>> in the other?
>

I tend to +1 on this because most of Hadoop committers wear both hats already.
Most of those who aren't should have for all I know.

The argument about enforcing projects separation by the means of splitting dev.
communities doesn't hold the critique IMO. True separation would come from
design independence and well defined contracts between components, not from
the perception of the policies.

Cos

On Tue, Aug 17, 2010 at 03:15AM, Chris Douglas wrote:
> Per the discussion thread: http://s.apache.org/XkY
>
> Should HDFS and MapReduce committers lists be combined and all
> subsequent committers on either of these two projects be granted karma
> in the other?
>
> If the vote passes, current and future committers to MapReduce and
> HDFS will gain commit rights in both projects. Commit rights to Common
> are unaffected.

+1

IMO, a combined committer list is less messy given how we currently
release where common+mr+hdfs are all bundled and released together.

Corner cases like the one postulated by Nigel above are the less
likely if contributors have access to all hadoop.

I do not  see how a combined contributor list could act as friction on
the ongoing break-up of the hadoop project -- something I'm in favor
of -- nor get in the way of the development of distinct mr/hdfs
user+dev communities; it seems to me that that project can progress
independent of who can commit where.

St.Ack
On Tue, Aug 17, 2010 at 3:15 AM, Chris Douglas <[EMAIL PROTECTED]> wrote:
> Per the discussion thread: http://s.apache.org/XkY
>
> Should HDFS and MapReduce committers lists be combined and all
> subsequent committers on either of these two projects be granted karma
> in the other?
>
> If the vote passes, current and future committers to MapReduce and
> HDFS will gain commit rights in both projects. Commit rights to Common
> are unaffected.
>
> Without bylaws, a 2/3 majority for a committer import seems like a
> reasonable bar, given that adding an individual committer requires
> consensus.
>
> ----
>
> Owen has started a separate voting thread, proposing to define the
> Common committer list as the union of HDFS and MapReduce committers
> (vote A), so I tried to write this (vote B) so it would not conflict.
> As I'm reading it:
>
> A passes, B passes: One can become a committer on HDFS or MapReduce.
> Commit to either implies commit on HDFS, MR, and Common.
> A passes, B fails: One can become a committer on HDFS or MapReduce.
> Commit to either implies commit on Common, only.
> A fails, B passes: One can become a committer on HDFS, MapReduce, or
> Common. Commit to to HDFS/MR implies converse, but individual
> appointments to Common continue.
> A fails, B fails: Committers continue to be appointed individually to
> HDFS, MapReduce, and Common.
>
> In no scheme would commit rights to Common imply commit rights to
> either HDFS or MapReduce, I guess. -C
>

On 08/19/2010 12:48 PM, Stack wrote:
> I do not  see how a combined contributor list could act as friction on
> the ongoing break-up of the hadoop project -- something I'm in favor
> of -- nor get in the way of the development of distinct mr/hdfs
> user+dev communities; it seems to me that that project can progress
> independent of who can commit where.

I agree.

Many projects manage such things with trust, not with rigid ACLs.  Some
committers are trusted to commit just test code, some just
documentation, and some deep implementation details in particular areas
of the code.  But should any committer wish to manage a release, or even
commit a patch outside of their normal domain of expertise (if it's been
reviewed by someone with expertise in that domain) they can.  Over time
they can gain trust in new areas of the project.  In such projects,
adding a committer merely implies that you trust someone not to overstep
their abilities.  If someone consistently suggests that patches are
ready for commit which others feel are not, then they won't earn that
trust and be invited to become a committer.  Thus committership is not
about deep technical abilities, but personal trust not to violate social
contracts.  Erecting walls within the community of trust via ACLs
doesn't seem productive.

Doug

On Aug 19, 2010, at 1:03 PM, Doug Cutting wrote:

> On 08/19/2010 12:48 PM, Stack wrote:
>> I do not  see how a combined contributor list could act as friction on
>> the ongoing break-up of the hadoop project -- something I'm in favor
>> of -- nor get in the way of the development of distinct mr/hdfs
>> user+dev communities; it seems to me that that project can progress
>> independent of who can commit where.
>
> I agree.
>
> Many projects manage such things with trust, not with rigid ACLs.  Some committers are trusted to commit just test code, some just documentation, and some deep implementation details in particular areas of the code.  But should any committer wish to manage a release, or even commit a patch outside of their normal domain of expertise (if it's been reviewed by someone with expertise in that domain) they can.  Over time they can gain trust in new areas of the project.  In such projects, adding a committer merely implies that you trust someone not to overstep their abilities.  If someone consistently suggests that patches are ready for commit which others feel are not, then they won't earn that trust and be invited to become a committer.  Thus committership is not about deep technical abilities, but personal trust not to violate social contracts.  Erecting walls within the community of trust via ACLs doesn't seem productive.
>
> Doug

Agreed.  Anyone want to reconsider their vote?

0

My thinking is that while MR and HDFS are the same product,
we can have joint committership.
If split happens there is no reason to have common committers,
like nobody is raising a question to merge e.g. tomcat and
subversion committers.
I think nobody does, but I don't know for sure of course.

--Konstantin

On 8/17/2010 3:15 AM, Chris Douglas wrote:
> Per the discussion thread: http://s.apache.org/XkY
>
> Should HDFS and MapReduce committers lists be combined and all
> subsequent committers on either of these two projects be granted karma
> in the other?
>
> If the vote passes, current and future committers to MapReduce and
> HDFS will gain commit rights in both projects. Commit rights to Common
> are unaffected.
>
> Without bylaws, a 2/3 majority for a committer import seems like a
> reasonable bar, given that adding an individual committer requires
> consensus.
>
> ----
>
> Owen has started a separate voting thread, proposing to define the
> Common committer list as the union of HDFS and MapReduce committers
> (vote A), so I tried to write this (vote B) so it would not conflict.
> As I'm reading it:
>
> A passes, B passes: One can become a committer on HDFS or MapReduce.
> Commit to either implies commit on HDFS, MR, and Common.
> A passes, B fails: One can become a committer on HDFS or MapReduce.
> Commit to either implies commit on Common, only.
> A fails, B passes: One can become a committer on HDFS, MapReduce, or
> Common. Commit to to HDFS/MR implies converse, but individual
> appointments to Common continue.
> A fails, B fails: Committers continue to be appointed individually to
> HDFS, MapReduce, and Common.
>
> In no scheme would commit rights to Common imply commit rights to
> either HDFS or MapReduce, I guess. -C
>

For the records, I am +1 on having a single set of committers for  
hdfs , mr and common. For me, all committers do not need to be equally  
good, or have to reach a grand bar; but rather I should trust that  
person to be a team player.

Dhruba

Sent from my iPhone

On Aug 19, 2010, at 4:00 PM, Nigel Daley <[EMAIL PROTECTED]> wrote:

>
> On Aug 19, 2010, at 1:03 PM, Doug Cutting wrote:
>
>> On 08/19/2010 12:48 PM, Stack wrote:
>>> I do not  see how a combined contributor list could act as  
>>> friction on
>>> the ongoing break-up of the hadoop project -- something I'm in favor
>>> of -- nor get in the way of the development of distinct mr/hdfs
>>> user+dev communities; it seems to me that that project can progress
>>> independent of who can commit where.
>>
>> I agree.
>>
>> Many projects manage such things with trust, not with rigid ACLs.  
>> Some committers are trusted to commit just test code, some just  
>> documentation, and some deep implementation details in particular  
>> areas of the code.  But should any committer wish to manage a  
>> release, or even commit a patch outside of their normal domain of  
>> expertise (if it's been reviewed by someone with expertise in that  
>> domain) they can.  Over time they can gain trust in new areas of  
>> the project.  In such projects, adding a committer merely implies  
>> that you trust someone not to overstep their abilities.  If someone  
>> consistently suggests that patches are ready for commit which  
>> others feel are not, then they won't earn that trust and be invited  
>> to become a committer.  Thus committership is not about deep  
>> technical abilities, but personal trust not to violate social  
>> contracts.  Erecting walls within the community of trust via ACLs  
>> doesn't seem productive.
>>
>> Doug
>
> Agreed.  Anyone want to reconsider their vote?
>

+1: Tom, Doug, Nigel, Stack, Dhruba, (Eli)
-1: Owen, Arun, Hemanth, Nicholas
+/-0: Konstantin, (Steve)

The vote did not pass.

Thanks to everyone who voted. -C

Sorry, I missed (Cos, +1) in my count, but the outcome is unchanged. -C

On Fri, Aug 20, 2010 at 2:38 PM, Chris Douglas <[EMAIL PROTECTED]> wrote:
> +1: Tom, Doug, Nigel, Stack, Dhruba, (Eli)
> -1: Owen, Arun, Hemanth, Nicholas
> +/-0: Konstantin, (Steve)
>
> The vote did not pass.
>
> Thanks to everyone who voted. -C
>

Chris,

Didn't you vote +1 too? (If I recall correctly from the discussion.)

Tom

On Fri, Aug 20, 2010 at 2:42 PM, Chris Douglas <[EMAIL PROTECTED]> wrote:
> Sorry, I missed (Cos, +1) in my count, but the outcome is unchanged. -C
>
> On Fri, Aug 20, 2010 at 2:38 PM, Chris Douglas <[EMAIL PROTECTED]> wrote:
>> +1: Tom, Doug, Nigel, Stack, Dhruba, (Eli)
>> -1: Owen, Arun, Hemanth, Nicholas
>> +/-0: Konstantin, (Steve)
>>
>> The vote did not pass.
>>
>> Thanks to everyone who voted. -C
>>
>

On Fri, Aug 20, 2010 at 2:52 PM, Tom White <[EMAIL PROTECTED]> wrote:
> Chris,
>
> Didn't you vote +1 too? (If I recall correctly from the discussion.)

At the start of the discussion I was leaning toward a +1, but elected
to abstain from the vote as I read the responses. I didn't add an
explicit +/-0, as I'd already added all the content I had in the
discussion thread, and would only repeat myself. Since you ask:

Vinod's point, i.e. "Where are the HDFS + MR issues?" gave me pause.
There really aren't that many. The only exception I can think of are
the RAID JIRAs, which aren't MR issues. Removing contrib is a better
fix. The projects really are separating, and I expect this will become
clearer as we cut more releases. If not, we should revisit this topic.

After these votes, "Hadoop committers" are defined as PMC members, or
contributors who independently earn commit status in both projects.
Most members of the current PMC required about two years of full-time
work in Hadoop before they were added. That's unusually steep, and the
reason I didn't vote -1.

There's a lack of data that makes this ambivalence hard to resolve.
The arguments in favor are too abstract (trust, coherency, etc.) to
make a definitive statements about the cost of separate lists.
Examples of the form, "I tried to do X, but was prevented and it
caused a costly, bureaucratic detour that took N days to resolve" (as
Vinod provided for lacking access to Common) are persuasive to me.
Hypothetical contributors that might be prevented from trying
something are compelling, but less so. I'd like to get more experience
working and releasing before I vote one way or the other. -C

Thanks for sharing this, Chris.

Cheers,
Tom

On Fri, Aug 20, 2010 at 5:37 PM, Chris Douglas <[EMAIL PROTECTED]> wrote:
> On Fri, Aug 20, 2010 at 2:52 PM, Tom White <[EMAIL PROTECTED]> wrote:
>> Chris,
>>
>> Didn't you vote +1 too? (If I recall correctly from the discussion.)
>
> At the start of the discussion I was leaning toward a +1, but elected
> to abstain from the vote as I read the responses. I didn't add an
> explicit +/-0, as I'd already added all the content I had in the
> discussion thread, and would only repeat myself. Since you ask:
>
> Vinod's point, i.e. "Where are the HDFS + MR issues?" gave me pause.
> There really aren't that many. The only exception I can think of are
> the RAID JIRAs, which aren't MR issues. Removing contrib is a better
> fix. The projects really are separating, and I expect this will become
> clearer as we cut more releases. If not, we should revisit this topic.
>
> After these votes, "Hadoop committers" are defined as PMC members, or
> contributors who independently earn commit status in both projects.
> Most members of the current PMC required about two years of full-time
> work in Hadoop before they were added. That's unusually steep, and the
> reason I didn't vote -1.
>
> There's a lack of data that makes this ambivalence hard to resolve.
> The arguments in favor are too abstract (trust, coherency, etc.) to
> make a definitive statements about the cost of separate lists.
> Examples of the form, "I tried to do X, but was prevented and it
> caused a costly, bureaucratic detour that took N days to resolve" (as
> Vinod provided for lacking access to Common) are persuasive to me.
> Hypothetical contributors that might be prevented from trying
> something are compelling, but less so. I'd like to get more experience
> working and releasing before I vote one way or the other. -C
>