This has been brought up in the past but we are here again.

We have a few large features that are hanging out and having a hard time
because trunk changes underneath it and in some cases because they are
being worked by folks without a commit bit.   (ex: snapshots w/ Jesse and
Matteo, and have some other potentially in the pipeline -- major assignment
manager changes with Jimmy and possibly me, HBASE-4120, HBASE-2600,
removing root)

Though I wasn't around yet, it seems like this is what we did for
coprocs/security, probably for the 0.90 master.
http://search-hadoop.com/m/byzZYZMktx1/hbase+windows&subj=Re+Proposed+feature+branch+for+HBase+security

Where the folks working on those features committers at the time?  What do
we do for contributions from folks who aren't committers yet?

This was proposed over on hadoop-general by Todd -- what do you all think
about doing something like this for the major changes?  (Github seems
easiest, svn seems "more official").

Here's one proposal, making use of git as an easy way to allow
non-committers to "commit" code while still tracking development in
the usual places:
- Upon anyone's request, we create a new "Version" tag in JIRA.
- The developers create an umbrella JIRA for the project, and file the
individual work items as subtasks (either up front, or as they are
developed if using a more iterative model)
- On the umbrella, they add a pointer to a git branch to be used as
the staging area for the branch. As they develop each subtask, they
can use the JIRA to discuss the development like they would with a
normally committed JIRA, but when they feel it is ready to go (not
requiring a +1 from any committer) they commit to their git branch
instead of the SVN repo.
- When the branch is ready to merge, they can call a merge vote, which
requires +1 from 3 committers, same as a branch being proposed by an
existing committer. A committer would then use git-svn to merge their
branch commit-by-commit, or if it is less extensive, simply generate a
single big patch to commit into SVN.

My thinking is that this would provide a low-friction way for people
to collaborate with the community and develop in the open, without
having to work closely with any committer to review every individual
subtask.

Another alternative, if people are reluctant to use git, would be to
add a "sandbox/" repository inside our SVN, and hand out commit bit to
branches inside there without any PMC vote. Anyone interested in
contributing could request a branch in the sandbox, and be granted
access as soon as they get an apache SVN account.

--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// [EMAIL PROTECTED]

+1 on git, either on github or closer to the linux model with real
distributed repos.

- I've been using it for just about all of my development and it works
pretty nicely.  I push everything to github as I'm working.  Then I
squash commits and create a diff to post on jira.
- I would suggest that since hbase's code base moves so rapidly, a
rebased branch should probably be a requirement before merging.
Otherwise the merge will get pretty interesting for very long lived
branches.

On Wed, Sep 5, 2012 at 11:38 AM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
> This has been brought up in the past but we are here again.
>
> We have a few large features that are hanging out and having a hard time
> because trunk changes underneath it and in some cases because they are
> being worked by folks without a commit bit.   (ex: snapshots w/ Jesse and
> Matteo, and have some other potentially in the pipeline -- major assignment
> manager changes with Jimmy and possibly me, HBASE-4120, HBASE-2600,
> removing root)
>
> Though I wasn't around yet, it seems like this is what we did for
> coprocs/security, probably for the 0.90 master.
> http://search-hadoop.com/m/byzZYZMktx1/hbase+windows&subj=Re+Proposed+feature+branch+for+HBase+security
>
> Where the folks working on those features committers at the time?  What do
> we do for contributions from folks who aren't committers yet?
>
> This was proposed over on hadoop-general by Todd -- what do you all think
> about doing something like this for the major changes?  (Github seems
> easiest, svn seems "more official").
>
> Here's one proposal, making use of git as an easy way to allow
> non-committers to "commit" code while still tracking development in
> the usual places:
> - Upon anyone's request, we create a new "Version" tag in JIRA.
> - The developers create an umbrella JIRA for the project, and file the
> individual work items as subtasks (either up front, or as they are
> developed if using a more iterative model)
> - On the umbrella, they add a pointer to a git branch to be used as
> the staging area for the branch. As they develop each subtask, they
> can use the JIRA to discuss the development like they would with a
> normally committed JIRA, but when they feel it is ready to go (not
> requiring a +1 from any committer) they commit to their git branch
> instead of the SVN repo.
> - When the branch is ready to merge, they can call a merge vote, which
> requires +1 from 3 committers, same as a branch being proposed by an
> existing committer. A committer would then use git-svn to merge their
> branch commit-by-commit, or if it is less extensive, simply generate a
> single big patch to commit into SVN.
>
> My thinking is that this would provide a low-friction way for people
> to collaborate with the community and develop in the open, without
> having to work closely with any committer to review every individual
> subtask.
>
> Another alternative, if people are reluctant to use git, would be to
> add a "sandbox/" repository inside our SVN, and hand out commit bit to
> branches inside there without any PMC vote. Anyone interested in
> contributing could request a branch in the sandbox, and be granted
> access as soon as they get an apache SVN account.
>
>
>
> --
> // Jonathan Hsieh (shay)
> // Software Engineer, Cloudera
> // [EMAIL PROTECTED]

On Wed, Sep 5, 2012 at 3:58 PM, Elliott Clark <[EMAIL PROTECTED]>wrote:

> +1 on git, either on github or closer to the linux model with real
> distributed repos.
>
> - I've been using it for just about all of my development and it works
> pretty nicely.  I push everything to github as I'm working.  Then I
> squash commits and create a diff to post on jira.
>

I do the same, just locally. Solid model.
> - I would suggest that since hbase's code base moves so rapidly, a
> rebased branch should probably be a requirement before merging.
> Otherwise the merge will get pretty interesting for very long lived
> branches.
>

IIRC when Todd was working on some large stuff for HDFS he was doing this
in a feature branch every few days. Seriously helps with when things are
actually finished in terms of rolling it back in.

Using github to keep a constantly rebased version (every few days) would be
a reasonble, super-low friction way of solving the problem for
non-committers. Further, for big changes, it would ensure that if the
people go away we aren't left with a bunch of dangling branches in the svn.
Problem here is also establishing the 'master' branch in github, though
that can be established on a case-by-case basis with the people involved.

>
> On Wed, Sep 5, 2012 at 11:38 AM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
> > This has been brought up in the past but we are here again.
> >
> > We have a few large features that are hanging out and having a hard time
> > because trunk changes underneath it and in some cases because they are
> > being worked by folks without a commit bit.   (ex: snapshots w/ Jesse and
> > Matteo, and have some other potentially in the pipeline -- major
> assignment
>

I'm generally opposed to doing feature branches for a variety of reasons
(left behind functionality, hard to roll back in, difficulty of testing,
etc) and further don't really feel its really necessary for the snapshot
code given that the code doesn't touch all that much of the current
codebase.

A lot of the pain with it right now is that the code has been broken into 5
patches, making it hard to build a version of HBase that has snapshots 'in
its current form'. This gets even worse as I'm planning on doing a bit more
refactoring into a couple more patches to help make it more digestable
(e.g. see latest patch for 3PC https://reviews.apache.org/r/6592/ which
pulls out a lot of the coordination functionality)). This helps with
reviews, etc, but makes it a bit of a pain for people who want to do
advanced testing on the feature - hard to justify doing a lot of that work
though as if the code is changing a lot, then testing doesn't make much
sense.

In terms of how the work is breaking down, with Matteo doing restore on top
of the taking that I'm working on, his part clearly depends on the taking
of snapshots. However, the filesystem layout hasn't changed at all in
nearly the last two months, meaning the work can proceed pretty much
independently (more or less).
> > manager changes with Jimmy and possibly me,
>

This is a lot more high-touch with the codebase, making a branch (either in
sandbox or otherwise) more feasible.
>  HBASE-4120, HBASE-2600,
> > removing root)
>

Salesforce is planning on tackling at least the latter two in the next few
months, so this is something that we need to figure out :)
>  >
> > Though I wasn't around yet, it seems like this is what we did for
> > coprocs/security, probably for the 0.90 master.
> >
> http://search-hadoop.com/m/byzZYZMktx1/hbase+windows&subj=Re+Proposed+feature+branch+for+HBase+security
> >
> > Where the folks working on those features committers at the time?  What
> do
> > we do for contributions from folks who aren't committers yet?
> >
> > This was proposed over on hadoop-general by Todd -- what do you all think
> > about doing something like this for the major changes?  (Github seems
> > easiest, svn seems "more official").
> >
> > Here's one proposal, making use of git as an easy way to allow

Overall, this seems reasonable. I can imagine the work to merge back in
being a huge pain. It would be great to see if we can break down these big
changes into smaller patches and roll them in one at a time. Both in terms
of ease on a single committer as helping to ensure code quality of each
sub-piece; its easier to enforce good testing on smaller pieces and helps
with code reuse.

My comments above obviously contradict this a little bit - its a huge pain
to work on the end functionality when the sub-pieces that you are building
on shift due to code reviews. In the end it leads to a better foundation,
but can be headache to keep everything in sync.

The latter goes away a bit if we have a single branch with the majority of
the code then progressive commits to fix things, but still is terrible to
review (pot calling the kettle black here) that first massive code drop.

TL;DR prefer smaller, independently useful patches that build to the bigger
change. Its may not be possible for some features, but should make it
easier to review, roll in, and in the end merge the final change while
being more generally useful.

This seems a little excessive. It would be nice for the more 'official'
status this confers,  but seems to create more friction than its worth
(IMO).
TL;DR github with 'official' branches per umbrella JIRA seems a
low-friction way to do feature branches without the possiblitly of cruft in
the main repository. We should really be sure that we need a branch though
and still favoring smaller patches along the same branch for generally
useful features.

Jesse Yates
@jesse_yates
jyates.github.com

Hope to have time to write up some more thoughts later, but some
interesting reading is this document from Linux on how to contribute
to that project:
https://github.com/mirrors/linux-2.6/blob/master/Documentation/SubmittingPatches

Worth looking at other projects' guidelines to form our own if we're
thinking of going this route.

-Todd

On Wed, Sep 5, 2012 at 4:43 PM, Jesse Yates <[EMAIL PROTECTED]> wrote:
> On Wed, Sep 5, 2012 at 3:58 PM, Elliott Clark <[EMAIL PROTECTED]>wrote:
>
>> +1 on git, either on github or closer to the linux model with real
>> distributed repos.
>>
>> - I've been using it for just about all of my development and it works
>> pretty nicely.  I push everything to github as I'm working.  Then I
>> squash commits and create a diff to post on jira.
>>
>
> I do the same, just locally. Solid model.
>
>
>> - I would suggest that since hbase's code base moves so rapidly, a
>> rebased branch should probably be a requirement before merging.
>> Otherwise the merge will get pretty interesting for very long lived
>> branches.
>>
>
> IIRC when Todd was working on some large stuff for HDFS he was doing this
> in a feature branch every few days. Seriously helps with when things are
> actually finished in terms of rolling it back in.
>
> Using github to keep a constantly rebased version (every few days) would be
> a reasonble, super-low friction way of solving the problem for
> non-committers. Further, for big changes, it would ensure that if the
> people go away we aren't left with a bunch of dangling branches in the svn.
> Problem here is also establishing the 'master' branch in github, though
> that can be established on a case-by-case basis with the people involved.
>
>>
>> On Wed, Sep 5, 2012 at 11:38 AM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
>> > This has been brought up in the past but we are here again.
>> >
>> > We have a few large features that are hanging out and having a hard time
>> > because trunk changes underneath it and in some cases because they are
>> > being worked by folks without a commit bit.   (ex: snapshots w/ Jesse and
>> > Matteo, and have some other potentially in the pipeline -- major
>> assignment
>>
>
> I'm generally opposed to doing feature branches for a variety of reasons
> (left behind functionality, hard to roll back in, difficulty of testing,
> etc) and further don't really feel its really necessary for the snapshot
> code given that the code doesn't touch all that much of the current
> codebase.
>
> A lot of the pain with it right now is that the code has been broken into 5
> patches, making it hard to build a version of HBase that has snapshots 'in
> its current form'. This gets even worse as I'm planning on doing a bit more
> refactoring into a couple more patches to help make it more digestable
> (e.g. see latest patch for 3PC https://reviews.apache.org/r/6592/ which
> pulls out a lot of the coordination functionality)). This helps with
> reviews, etc, but makes it a bit of a pain for people who want to do
> advanced testing on the feature - hard to justify doing a lot of that work
> though as if the code is changing a lot, then testing doesn't make much
> sense.
>
> In terms of how the work is breaking down, with Matteo doing restore on top
> of the taking that I'm working on, his part clearly depends on the taking
> of snapshots. However, the filesystem layout hasn't changed at all in
> nearly the last two months, meaning the work can proceed pretty much
> independently (more or less).
>
>
>> > manager changes with Jimmy and possibly me,
>>
>
> This is a lot more high-touch with the codebase, making a branch (either in
> sandbox or otherwise) more feasible.
>
>
>>  HBASE-4120, HBASE-2600,
>> > removing root)
>>
>
> Salesforce is planning on tackling at least the latter two in the next few
> months, so this is something that we need to figure out :)
>
>
>>  >
>> > Though I wasn't around yet, it seems like this is what we did for
>> > coprocs/security, probably for the 0.90 master.

Todd Lipcon
Software Engineer, Cloudera

On Wed, Sep 5, 2012 at 4:43 PM, Jesse Yates <[EMAIL PROTECTED]> wrote:

> On Wed, Sep 5, 2012 at 3:58 PM, Elliott Clark <[EMAIL PROTECTED]
> >wrote:
>
> > - I would suggest that since hbase's code base moves so rapidly, a
> > rebased branch should probably be a requirement before merging.
> > Otherwise the merge will get pretty interesting for very long lived
> > branches.
> >
>
> IIRC when Todd was working on some large stuff for HDFS he was doing this
> in a feature branch every few days. Seriously helps with when things are
> actually finished in terms of rolling it back in.
>
> Using github to keep a constantly rebased version (every few days) would be
> a reasonble, super-low friction way of solving the problem for
> non-committers. Further, for big changes, it would ensure that if the
> people go away we aren't left with a bunch of dangling branches in the svn.
> Problem here is also establishing the 'master' branch in github, though
> that can be established on a case-by-case basis with the people involved.
>
> Have a work branch and then have the "committed" branch in github?
>  >
> > On Wed, Sep 5, 2012 at 11:38 AM, Jonathan Hsieh <[EMAIL PROTECTED]>
> wrote:
> > > This has been brought up in the past but we are here again.
> > >
> > > We have a few large features that are hanging out and having a hard
> time
> > > because trunk changes underneath it and in some cases because they are
> > > being worked by folks without a commit bit.   (ex: snapshots w/ Jesse
> and
> > > Matteo, and have some other potentially in the pipeline -- major
> > assignment
> >
>
> I'm generally opposed to doing feature branches for a variety of reasons
> (left behind functionality, hard to roll back in, difficulty of testing,
> etc) and further don't really feel its really necessary for the snapshot
> code given that the code doesn't touch all that much of the current
> codebase.
>
> I realize it is hard to split all the code up but it is also hard and time
consuming to review the design and implementation with a single 300k patch
that has many modes.

I'll post some other more concrete suggestions there.
> A lot of the pain with it right now is that the code has been broken into 5
> patches, making it hard to build a version of HBase that has snapshots 'in
> its current form'. This gets even worse as I'm planning on doing a bit more
> refactoring into a couple more patches to help make it more digestable
> (e.g. see latest patch for 3PC https://reviews.apache.org/r/6592/ which
> pulls out a lot of the coordination functionality)). This helps with
> reviews, etc, but makes it a bit of a pain for people who want to do
> advanced testing on the feature - hard to justify doing a lot of that work
> though as if the code is changing a lot, then testing doesn't make much
> sense.
>
> My hope with splitting parts of it was that we could peal off and commit
portions we are agree upon and have something solid and committed to build
the rest on top of.  If also keeps the subsystems decoupled which is a good
thing; this limits the scope of what needs to be understood, so that
reviewing/modifying it only requires localized knowledge instead of global
knowledge. This process would likely add tests at the internals layers and
make these subsystems testable.

>From a reviewer's point of view -- I'm going to ignore the things built on
top until we get the building blocks to a place where it is easy to follow.
> In terms of how the work is breaking down, with Matteo doing restore on top
> of the taking that I'm working on, his part clearly depends on the taking
> of snapshots. However, the filesystem layout hasn't changed at all in
> nearly the last two months, meaning the work can proceed pretty much
> independently (more or less).
>
> This is one case where the design docs did a really good job of defining
the structures ahead of time.  The spec is laid out and we just need to
confirm that the code puts things in those places. :)
> > > manager changes with Jimmy and possibly me,

There is another philosophy out there -- commit it but have the feature
disabled by default until it has been stabilized.  We've tried this in a
few places (wal compress, off heap cache, instant schema change) and these
features unfortunately haven't gone through the "stabilitzation".  Maybe
after Lars's push for more frequent releases proves it self,  we could go
into that camp.  I personally don't like the idea of features that are in
but doesn't always work or doesn't work in conjunction with other perfectly
reasonable seemingly orthogonal features.
be things we can't really do incrementally.   These should be rare.
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// [EMAIL PROTECTED]

On Wed, Sep 5, 2012 at 11:38 AM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
> Though I wasn't around yet, it seems like this is what we did for
> coprocs/security, probably for the 0.90 master.
> http://search-hadoop.com/m/byzZYZMktx1/hbase+windows&subj=Re+Proposed+feature+branch+for+HBase+security
>

Yes.

> Where the folks working on those features committers at the time?  What do
> we do for contributions from folks who aren't committers yet?
>

Yes.

For folks not yet committers, lets look at them.  If they are working
on big features for HBase, they probably should be committers?

> Here's one proposal, making use of git as an easy way to allow
> non-committers to "commit" code while still tracking development in
> the usual places:
> - Upon anyone's request, we create a new "Version" tag in JIRA.
> - The developers create an umbrella JIRA for the project, and file the
> individual work items as subtasks (either up front, or as they are
> developed if using a more iterative model)
> - On the umbrella, they add a pointer to a git branch to be used as

Do you mean github when you say git above?  So their work is
accessible by others?

> the staging area for the branch. As they develop each subtask, they
> can use the JIRA to discuss the development like they would with a
> normally committed JIRA, but when they feel it is ready to go (not
> requiring a +1 from any committer) they commit to their git branch
> instead of the SVN repo.
> - When the branch is ready to merge, they can call a merge vote, which
> requires +1 from 3 committers, same as a branch being proposed by an
> existing committer. A committer would then use git-svn to merge their
> branch commit-by-commit, or if it is less extensive, simply generate a
> single big patch to commit into SVN.
>

Your proposal looks great to me.

> Another alternative, if people are reluctant to use git, would be to
> add a "sandbox/" repository inside our SVN, and hand out commit bit to
> branches inside there without any PMC vote. Anyone interested in
> contributing could request a branch in the sandbox, and be granted
> access as soon as they get an apache SVN account.
>

We could do this but more overhead that your first suggestion.

Good on you Jon,
St.Ack

On Wed, Sep 5, 2012 at 3:49 PM, Stack <[EMAIL PROTECTED]> wrote:

> On Wed, Sep 5, 2012 at 11:38 AM, Jonathan Hsieh <[EMAIL PROTECTED]>
> wrote:> Where the folks working on those features committers at the time?
>  What do
> > we do for contributions from folks who aren't committers yet?
> >
>
> Yes.
>
> For folks not yet committers, lets look at them.  If they are working
> on big features for HBase, they probably should be committers?
>
> This is probably case by case.
> > Here's one proposal, making use of git as an easy way to allow
> > non-committers to "commit" code while still tracking development in
> > the usual places:
> > - Upon anyone's request, we create a new "Version" tag in JIRA.
> > - The developers create an umbrella JIRA for the project, and file the
> > individual work items as subtasks (either up front, or as they are
> > developed if using a more iterative model)
> > - On the umbrella, they add a pointer to a git branch to be used as
>
> Do you mean github when you say git above?  So their work is
> accessible by others?
>
> Yes, github.  Personally, I'm not familiar with github's review interface
and prefer doing reviews on reviewboard.  Fewer systems for me to figure
out, and fewer places for me to look. :)
> > the staging area for the branch. As they develop each subtask, they
> > can use the JIRA to discuss the development like they would with a
> > normally committed JIRA, but when they feel it is ready to go (not
> > requiring a +1 from any committer) they commit to their git branch
> > instead of the SVN repo.
> > - When the branch is ready to merge, they can call a merge vote, which
> > requires +1 from 3 committers, same as a branch being proposed by an
> > existing committer. A committer would then use git-svn to merge their
> > branch commit-by-commit, or if it is less extensive, simply generate a
> > single big patch to commit into SVN.
> >
>
> Your proposal looks great to me.
>
> > Another alternative, if people are reluctant to use git, would be to
> > add a "sandbox/" repository inside our SVN, and hand out commit bit to
> > branches inside there without any PMC vote. Anyone interested in
> > contributing could request a branch in the sandbox, and be granted
> > access as soon as they get an apache SVN account.
> >
>
> We could do this but more overhead that your first suggestion.
>
> I agree this is more work but svn is harder to change and feels more
"official".  I guess this is part of the apache legacy.
> Good on you Jon,
> St.Ack
>

--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// [EMAIL PROTECTED]

On Thu, Sep 6, 2012 at 3:33 AM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
>> For folks not yet committers, lets look at them.  If they are working
>> on big features for HBase, they probably should be committers?
>>
>> This is probably case by case.
>

Agree
St.Ack

On Wed, Sep 5, 2012 at 11:38 AM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
> Though I wasn't around yet, it seems like this is what we did for
> coprocs/security, probably for the 0.90 master.
>
http://search-hadoop.com/m/byzZYZMktx1/hbase+windows&subj=Re+Proposed+feature+branch+for+HBase+security

Though we had multiple contributors, not all committers, I could have
committed any of the work at any time and Gary soon was a committer as well.

> Here's one proposal, making use of git as an easy way to allow
> non-committers to "commit" code while still tracking development in
> the usual places:
> [...]
> My thinking is that this would provide a low-friction way for people
> to collaborate with the community and develop in the open, without
> having to work closely with any committer to review every individual
> subtask.

I use git and private GitHub repos for all of our distribution engineering
work so personally this gets a big +1 from me. This is how we are operating
internally.

For example, I've been following along with Todd's HDFS-3077 feature
development locally as a patch series floating on branch-2, via cherry-pick
of the changes and periodic rebase of those picks against the branch. It
would be, personally, trivial to cherry pick from any git repo, does not
have to be git.apache.org (in this case HDFS-3077 work is on a feature
branch in Apache SVN). This has been very helpful, it has allowed us to do
some advance testing of this feature under development sufficient to
understand and validate its theory of operation.

It would be excellent to have the option to do something like that with
other works under development by those who might not have Apache SVN commit
privs. On the other hand, this takes some effort, with rebasing over an
evolving source you will have to deal with a constant stream of small merge
conflicts, and as time goes on the conflicts will become more significant.
This maintenance effort is to be taken up by the contributor? That's
currently the story of my life (but for ~10 projects, not 1) so I have the
time/patience and I'd like to think skill to deal with it. I also trust the
skill of HBase committers to do this. That does not generalize, though. To
summarize, I think this could work well if orgs like Salesforce put a small
team on a particular task, so I'm all for it, but let's leave it at that
and see how it goes.

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Hello,

Shall we move forward with this proposal?  Let's get this in place soon so
larger projects such as snapshots can make use of this.

- Dave

On Wed, Sep 5, 2012 at 11:38 AM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:

> This has been brought up in the past but we are here again.
>
> We have a few large features that are hanging out and having a hard time
> because trunk changes underneath it and in some cases because they are
> being worked by folks without a commit bit.   (ex: snapshots w/ Jesse and
> Matteo, and have some other potentially in the pipeline -- major assignment
> manager changes with Jimmy and possibly me, HBASE-4120, HBASE-2600,
> removing root)
>
> Though I wasn't around yet, it seems like this is what we did for
> coprocs/security, probably for the 0.90 master.
>
> http://search-hadoop.com/m/byzZYZMktx1/hbase+windows&subj=Re+Proposed+feature+branch+for+HBase+security
>
> Where the folks working on those features committers at the time?  What do
> we do for contributions from folks who aren't committers yet?
>
> This was proposed over on hadoop-general by Todd -- what do you all think
> about doing something like this for the major changes?  (Github seems
> easiest, svn seems "more official").
>
> Here's one proposal, making use of git as an easy way to allow
> non-committers to "commit" code while still tracking development in
> the usual places:
> - Upon anyone's request, we create a new "Version" tag in JIRA.
> - The developers create an umbrella JIRA for the project, and file the
> individual work items as subtasks (either up front, or as they are
> developed if using a more iterative model)
> - On the umbrella, they add a pointer to a git branch to be used as
> the staging area for the branch. As they develop each subtask, they
> can use the JIRA to discuss the development like they would with a
> normally committed JIRA, but when they feel it is ready to go (not
> requiring a +1 from any committer) they commit to their git branch
> instead of the SVN repo.
> - When the branch is ready to merge, they can call a merge vote, which
> requires +1 from 3 committers, same as a branch being proposed by an
> existing committer. A committer would then use git-svn to merge their
> branch commit-by-commit, or if it is less extensive, simply generate a
> single big patch to commit into SVN.
>
> My thinking is that this would provide a low-friction way for people
> to collaborate with the community and develop in the open, without
> having to work closely with any committer to review every individual
> subtask.
>
> Another alternative, if people are reluctant to use git, would be to
> add a "sandbox/" repository inside our SVN, and hand out commit bit to
> branches inside there without any PMC vote. Anyone interested in
> contributing could request a branch in the sandbox, and be granted
> access as soon as they get an apache SVN account.
>
>
>
> --
> // Jonathan Hsieh (shay)
> // Software Engineer, Cloudera
> // [EMAIL PROTECTED]
>

Yes, the consensus at the dev meetup (a poll of the room) was we
should move forward with use of git and GitHub hosting for feature
development / branches.

On Tue, Sep 11, 2012 at 11:49 PM, Dave Wang <[EMAIL PROTECTED]> wrote:
> Hello,
>
> Shall we move forward with this proposal?  Let's get this in place soon so
> larger projects such as snapshots can make use of this.
>
> - Dave

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet
Hein (via Tom White)

I hope we're not mandating using github or even git.
If a group of folks get together and decide to work together on a feature it should be up to them to decide how to organize themselves.
If they want to use github (which probably is the most obvious choice) that is fine, but if they decide to SVN, P4, CVS, Mercurial, bzr (or VisualSourceSafe :) )... That should be up to them, as long as the end result is a stream of patches that can be applied to the canonical SVN repository.

-- Lars

----- Original Message -----
From: Andrew Purtell <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Cc:
Sent: Wednesday, September 12, 2012 12:24 AM
Subject: Re: Thoughts about large feature dev branches

Yes, the consensus at the dev meetup (a poll of the room) was we
should move forward with use of git and GitHub hosting for feature
development / branches.

On Tue, Sep 11, 2012 at 11:49 PM, Dave Wang <[EMAIL PROTECTED]> wrote:
> Hello,
>
> Shall we move forward with this proposal?  Let's get this in place soon so
> larger projects such as snapshots can make use of this.
>
> - Dave

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet
Hein (via Tom White)

On Wed, Sep 12, 2012 at 1:42 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
> I hope we're not mandating using github or even git.
> If a group of folks get together and decide to work together on a feature it should be up to them to decide how to organize themselves.
> If they want to use github (which probably is the most obvious choice) that is fine, but if they decide to SVN, P4, CVS, Mercurial, bzr (or VisualSourceSafe :) )... That should be up to them, as long as the end result is a stream of patches that can be applied to the canonical SVN repository.
>

Agreed.

Text in refguide that I committed earlier is this (no explicit git
requirement or githubbery):

Feature Branches are easy to make.  You do not have to be a committer
to make one.  Just request the name of your branch be added to JIRA up
on the developer's mailing list and a committer will add it for you.
Thereafter you can file issues against your feature branch in HBase
JIRA.  Your code you keep elsewhere -- it should be public so it can
be observed -- and you can update dev mailing list on progress.   When
the feature is ready for commit, 3 +1s from committers will get your
feature merged

<footnote><para>See <link
xlink:href="http://search-hadoop.com/m/uPNz32C5FkS1&subj=Re+Thoughts+about+large+feature+dev+branches">HBase,
mail # dev - Thoughts about large feature dev
branches</link></para></footnote>

The footnote references this thread so your comment above about any
VCS will do will be picked up Lars.

St.Ack

On Wed, Sep 12, 2012 at 12:24 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
> Yes, the consensus at the dev meetup (a poll of the room) was we
> should move forward with use of git and GitHub hosting for feature
> development / branches.
>
> On Tue, Sep 11, 2012 at 11:49 PM, Dave Wang <[EMAIL PROTECTED]> wrote:
>> Hello,
>>
>> Shall we move forward with this proposal?  Let's get this in place soon so
>> larger projects such as snapshots can make use of this.
>>
>> - Dave
>

+1

(Should we make a page to keep 'project decisions' on? Or just
remember to refer to this thread when feature branches comes up in
future?)

St.Ack

On Wed, Sep 12, 2012 at 11:28 AM, Stack <[EMAIL PROTECTED]> wrote:
> On Wed, Sep 12, 2012 at 12:24 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
>> Yes, the consensus at the dev meetup (a poll of the room) was we
>> should move forward with use of git and GitHub hosting for feature
>> development / branches.
>>
> +1
>
> (Should we make a page to keep 'project decisions' on? Or just
> remember to refer to this thread when feature branches comes up in
> future?)

"How To Contribute" on the project wiki could be a place for this
stuff specifically, as well as the manual.

Generally, we could put these things into the manual, under a
'Process' section or some such?

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet
Hein (via Tom White)

On Wed, Sep 12, 2012 at 11:43 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
> Generally, we could put these things into the manual, under a
> 'Process' section or some such?
>

I added a community chapter to the book with a 'decisions' subsection
that has one entry in it currently, this mail thread.  I haven't
pushed it out yet so if better place, just say.

St.Ack