On Wed, Mar 28, 2012 at 1:03 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>
> On Mar 28, 2012, at 12:48 AM, Aaron T. Myers wrote:
>
>> Hey Arun,
>>
>> On Wed, Mar 28, 2012 at 12:28 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>>
>>> Done, all clear; I've also created jira revisions. Please let me know if
>>> you find any issues.
>>>
>>
>> Thanks a lot for making these changes. Two questions:
>>
>> - Now that we've renamed branch-0.23 to branch-2, and since there is as yet
>> no branch-0.23.3, what should be done with JIRAs marked with the 0.23.3
>> version? Perhaps all of these should be updated to 2.0.0?
>>
>
> Yep, in the process of doing it.
>
>> - We still have the JIRA version 0.24.0, which is presumably still
>> representative of trunk. Given that we will likely never release an 0.24.0,
>> should we rename this version in JIRA as well?
>
> I've created a 3.0.0 version in jira too. I'll update all jiras to point to that instead of 0.24.0.

Is trunk/24 definitely 3.0.0? Given that we don't have any major new
features targeted at it (but not targeted at 2.x) yet, I've been
thinking of it more like 2.1. I also think, given we have PB in place
in both branches, it's likely we can maintain client compatibility
between 23 and 24 in the absence of anything major coming down the
pipe.

Proposal: is it possible to call the JIRA fixVersion "trunk", and then
when we branch off trunk to make a release, rename it at that point to
"2.1" or "3.0" or whatever it gets called?

Todd
--
Todd Lipcon
Software Engineer, Cloudera

On Wed, Mar 28, 2012 at 11:53 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> Proposal: is it possible to call the JIRA fixVersion "trunk", and then
> when we branch off trunk to make a release, rename it at that point to
> "2.1" or "3.0" or whatever it gets called?
>

I like this idea. Just to be clear, I think the exact workflow would be:

1. Set the version fields to "trunk" if you're not committing the JIRA to
any current versioned branch.
2. When a new release branch is made off of trunk, rename the "trunk" JIRA
version to whatever the appropriate version number is.
3. At the same time as (2), create a new JIRA version also called "trunk".
4. Go to 1.

Is this what you were thinking, Todd?

--
Aaron T. Myers
Software Engineer, Cloudera

On Wed, Mar 28, 2012 at 11:59 AM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
> On Wed, Mar 28, 2012 at 11:53 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>
>> Proposal: is it possible to call the JIRA fixVersion "trunk", and then
>> when we branch off trunk to make a release, rename it at that point to
>> "2.1" or "3.0" or whatever it gets called?
>>
>
> I like this idea. Just to be clear, I think the exact workflow would be:
>
> 1. Set the version fields to "trunk" if you're not committing the JIRA to
> any current versioned branch.
> 2. When a new release branch is made off of trunk, rename the "trunk" JIRA
> version to whatever the appropriate version number is.
> 3. At the same time as (2), create a new JIRA version also called "trunk".
> 4. Go to 1.
>
> Is this what you were thinking, Todd?

Yep, that's right.

-Todd
--
Todd Lipcon
Software Engineer, Cloudera

I disagree. Trunk should become branch-3 once someone wants to start
stabilizing it. Arun is going to need the minor versions for when he adds
features.

X.Y.Z

Z = bug fixes
Y = minor release (compatible, adds features)
X = major release (incompatible)

So from branch-2 will come branch-2.0 with tags for 2.0.0, 2.0.1. New
features will go into branch-2, which will become branch-2.1, branch-2.2,
and so on.

-- Owen

On Wed, Mar 28, 2012 at 12:25 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote:
> I disagree. Trunk should become branch-3 once someone wants to start
> stabilizing it. Arun is going to need the minor versions for when he adds
> features.
>
> X.Y.Z
>
> Z = bug fixes
> Y = minor release (compatible, adds features)
> X = major release (incompatible)
I agree with this classification.

>
> So from branch-2 will come branch-2.0 with tags for 2.0.0, 2.0.1. New
> features will go into branch-2, which will become branch-2.1, branch-2.2,
> and so on.

But new features also go to trunk. And if none of our new features are
incompatible, why do we anticipate that trunk is 3.0?

Basically trunk is almost identical to branch-2 right now... I guess
my proposal is basically to not bother having a trunk separate from a
branch-2, and use a branch-2.0 for continued stabilization of what we
currently call branch-23. When someone has something incompatible to
propose, then we can bother splitting it off..

Todd
--
Todd Lipcon
Software Engineer, Cloudera

On Wed, Mar 28, 2012 at 12:32 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:

But new features also go to trunk. And if none of our new features are
> incompatible, why do we anticipate that trunk is 3.0?
>

Let's imagine that we already had a 2.0.0 release. Now we want to add
features like HA. The only place to put that is in 2.1.0. On the other
hand, you don't want to pull *ALL* of the changes from trunk. That is way
too much scope. So the RM of the 2 branch needs to make the call of what
should be 2.1 vs 3.0.

-- Owen

On 03/28/2012 12:39 PM, Owen O'Malley wrote:
> [ ... ] So the RM of the 2 branch needs to make the call of what
> should be 2.1 vs 3.0.

I thought these were community decisions, not RM decisions, no?

http://s.apache.org/rm

Doug

Hi Owen,

On Wed, Mar 28, 2012 at 12:39 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote:

> Let's imagine that we already had a 2.0.0 release. Now we want to add
> features like HA. The only place to put that is in 2.1.0. On the other
> hand, you don't want to pull *ALL* of the changes from trunk. That is way
> too much scope. So the RM of the 2 branch needs to make the call of what
> should be 2.1 vs 3.0.
>

I don't think anyone disagrees with this line of reasoning. It's certainly
up to the RM what gets included in branch-2 and hence what gets put up for
release votes under the "2.y.z" version numbers. I don't think Todd was
suggesting we rename the JIRA version "0.24.0" to "2.1.0".

But, the question still remains of how to refer to the branch "trunk" in
JIRA. I don't think it should be called 3.0.0, as that's not necessarily
the next release that will come off of it, and using a version number for
trunk that changes from time to time has other downsides as I described in
my response to Arun. Given this, do you object to renaming the JIRA fix
version that refers to the branch trunk to "trunk" ?

--
Aaron T. Myers
Software Engineer, Cloudera

On Mar 28, 2012, at 12:32 PM, Todd Lipcon wrote:

> But new features also go to trunk. And if none of our new features are
> incompatible, why do we anticipate that trunk is 3.0?
Essentially 'trunk' is where incompatible changes *may* be committed (in future). We should allow for that.

I'm ambivalent about the 'version string' for trunk being either trunk-SNAPSHOT or 3.0.0-SNAPSHOT.

On on hand, we've historically bumped the version number for trunk (i.e. 3.0.0-SNAPSHOT). This has the nice property that when we do cut a release branch off trunk we don't have to scramble to change fix-versions on all the jiras committed to trunk from 'trunk' to X.0.0. Since I've done my fair share of jira manipulation for releases as the RM, as recently as last night, it's nice to not force the burden on the RM for branch-3.

OTOH, having a constant trunk-SNAPSHOT version string helps devs working on trunk since they don't have to deal with version bumps on trunk (albeit, once in a while). (Credit to Scott Carey for this idea.)

Given the above I'd stick with 3.0.0 since it means lesser confusion and lesser work for the RM on future major releases.

On a related note: as I noted last night, I'd again urge committers to not set the 'fix-version' to trunk's version if it's committed to other branches (branch-1, branch-2 etc.)

thanks,
Arun

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

On Wed, Mar 28, 2012 at 1:57 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> On on hand, we've historically bumped the version number for trunk (i.e.
> 3.0.0-SNAPSHOT). This has the nice property that when we do cut a release
> branch off trunk we don't have to scramble to change fix-versions on all
> the jiras committed to trunk from 'trunk' to X.0.0. Since I've done my fair
> share of jira manipulation for releases as the RM, as recently as last
> night, it's nice to not force the burden on the RM for branch-3.
>

I don't think you'd have to change all the JIRAs. You'd just have to change
the name of the "trunk" JIRA version to whatever the right number is, and
then create a new version in JIRA also named "trunk." This would serve the
same purpose, without having to update any individual JIRAs.
> OTOH, having a constant trunk-SNAPSHOT version string helps devs working
> on trunk since they don't have to deal with version bumps on trunk (albeit,
> once in a while). (Credit to Scott Carey for this idea.)
>

This is a nice benefit of Todd's (and Scott's?) proposal. Whenever we do
change the trunk version number, I have to update a bunch of scripts,
environment files, and configuration XML files. Not the end of the world,
but annoying nonetheless.

I'd also add to this benefit that users who are new to the project will
more easily be able to determine what version to put for a JIRA they want
to get committed to trunk. I've seen plenty of users who are confused by
having to set "0.24.0" as the version indicating trunk.

There's also a nice symmetry in having the branch in svn/git (named trunk)
have the same name as the JIRA version indicator.
>
> Given the above I'd stick with 3.0.0 since it means lesser confusion and
> lesser work for the RM on future major releases.
>

I honestly believe that this scheme is more confusing for devs and users,
and almost no different for RMs given what I described above with JIRA
version renaming. But, I don't feel super strongly about it. If this makes
sense to you, then I'll stop pushing.

--
Aaron T. Myers
Software Engineer, Cloudera

On Mar 28, 2012, at 2:14 PM, Aaron T. Myers wrote:

> On Wed, Mar 28, 2012 at 1:57 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>
>> On on hand, we've historically bumped the version number for trunk (i.e.
>> 3.0.0-SNAPSHOT). This has the nice property that when we do cut a release
>> branch off trunk we don't have to scramble to change fix-versions on all
>> the jiras committed to trunk from 'trunk' to X.0.0. Since I've done my fair
>> share of jira manipulation for releases as the RM, as recently as last
>> night, it's nice to not force the burden on the RM for branch-3.
>>
>
> I don't think you'd have to change all the JIRAs. You'd just have to change
> the name of the "trunk" JIRA version to whatever the right number is, and
> then create a new version in JIRA also named "trunk." This would serve the
> same purpose, without having to update any individual JIRAs.
Ah, I didn't know that, thanks for the tip. That alleviates my concerns.

Arun

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

On 3/28/12 2:14 PM, "Aaron T. Myers" <[EMAIL PROTECTED]> wrote:

>On Wed, Mar 28, 2012 at 1:57 PM, Arun C Murthy <[EMAIL PROTECTED]>
>wrote:
>
>> On on hand, we've historically bumped the version number for trunk (i.e.
>> 3.0.0-SNAPSHOT). This has the nice property that when we do cut a
>>release
>> branch off trunk we don't have to scramble to change fix-versions on all
>> the jiras committed to trunk from 'trunk' to X.0.0. Since I've done my
>>fair
>> share of jira manipulation for releases as the RM, as recently as last
>> night, it's nice to not force the burden on the RM for branch-3.
>>
>
>I don't think you'd have to change all the JIRAs. You'd just have to
>change
>the name of the "trunk" JIRA version to whatever the right number is, and
>then create a new version in JIRA also named "trunk." This would serve the
>same purpose, without having to update any individual JIRAs.
>
>
>> OTOH, having a constant trunk-SNAPSHOT version string helps devs working
>> on trunk since they don't have to deal with version bumps on trunk
>>(albeit,
>> once in a while). (Credit to Scott Carey for this idea.)
>>
>
>This is a nice benefit of Todd's (and Scott's?) proposal. Whenever we do
>change the trunk version number, I have to update a bunch of scripts,
>environment files, and configuration XML files. Not the end of the world,
>but annoying nonetheless.
>
>I'd also add to this benefit that users who are new to the project will
>more easily be able to determine what version to put for a JIRA they want
>to get committed to trunk. I've seen plenty of users who are confused by
>having to set "0.24.0" as the version indicating trunk.
>
>There's also a nice symmetry in having the branch in svn/git (named trunk)
>have the same name as the JIRA version indicator.

My proposal was from a few months back related to the naming of versions
in maven.  
It boils down to 'laziness is a virtue' + 'the maven version should match
the branch'.  
Pick a name for a branch when you actually branch, not months before.
'trunk' in svn == 'trunk-SNAPSHOT' in maven == 'trunk' in JIRA.

The creation of a branch should avoid impacting the place branched from
(i.e. cause work for people on trunk because of a branch, or cause work
for a branch due to a tag, etc).
If the version in 'branch/2.0.x' has maven version '2.0.x-SNAPSHOT' and
JIRA tag '2.0.0' then tagging 2.0.0 has the following steps:

$> svn cp branch/2.0.x tags/2.0.0
$> cd tags/2.0.0
$> mvn versoins:set -DnewVersion=2.0.0
$> svn diff   (spot check pom changes that versions:set did)
$> mvn versions:commit
$> svn commit

The result is a new tag with the version set, and no changes to the branch
that was tagged from.
When the decision is made to have a 2.0.1, a jira tag can be made (or a
2.0.x tag renamed to 2.0.1, and a new 2.0.x tag made as Todd suggests).
Being lazy here helps because maybe instead after some development on the
2.0.x branch it is decided to branch 2.1.x from it.

When the decision to branch 2.1 occurs, a new branch 'branch/2.1.x' is
made in svn, its version becomes '2.1.x-SNAPSHOT' and a JIRA tag for
'2.1.x' is made or renamed from an old one.  The old 2.0.x branch is
unchanged (and available for more minor bugfix releases).

Once trunk or a branch is set up for build scripts or hudson, etc, it
works without modification no matter how many times a branch or tag occurs
off of it.
>
>
>>
>> Given the above I'd stick with 3.0.0 since it means lesser confusion and
>> lesser work for the RM on future major releases.
>>
>
>I honestly believe that this scheme is more confusing for devs and users,
>and almost no different for RMs given what I described above with JIRA
>version renaming. But, I don't feel super strongly about it. If this makes
>sense to you, then I'll stop pushing.
>
>--
>Aaron T. Myers
>Software Engineer, Cloudera

Thanks a lot, Scott, for bringing the discussion back to what to call
"trunk" in JIRA and elsewhere. The proposal you describe to me makes a lot
of sense.

Owen, does this (and the JIRA proposal) make sense to you?

--
Aaron T. Myers
Software Engineer, Cloudera

On Thu, Mar 29, 2012 at 12:45 PM, Scott Carey <[EMAIL PROTECTED]>wrote:

>
>
> On 3/28/12 2:14 PM, "Aaron T. Myers" <[EMAIL PROTECTED]> wrote:
>
> >On Wed, Mar 28, 2012 at 1:57 PM, Arun C Murthy <[EMAIL PROTECTED]>
> >wrote:
> >
> >> On on hand, we've historically bumped the version number for trunk (i.e.
> >> 3.0.0-SNAPSHOT). This has the nice property that when we do cut a
> >>release
> >> branch off trunk we don't have to scramble to change fix-versions on all
> >> the jiras committed to trunk from 'trunk' to X.0.0. Since I've done my
> >>fair
> >> share of jira manipulation for releases as the RM, as recently as last
> >> night, it's nice to not force the burden on the RM for branch-3.
> >>
> >
> >I don't think you'd have to change all the JIRAs. You'd just have to
> >change
> >the name of the "trunk" JIRA version to whatever the right number is, and
> >then create a new version in JIRA also named "trunk." This would serve the
> >same purpose, without having to update any individual JIRAs.
> >
> >
> >> OTOH, having a constant trunk-SNAPSHOT version string helps devs working
> >> on trunk since they don't have to deal with version bumps on trunk
> >>(albeit,
> >> once in a while). (Credit to Scott Carey for this idea.)
> >>
> >
> >This is a nice benefit of Todd's (and Scott's?) proposal. Whenever we do
> >change the trunk version number, I have to update a bunch of scripts,
> >environment files, and configuration XML files. Not the end of the world,
> >but annoying nonetheless.
> >
> >I'd also add to this benefit that users who are new to the project will
> >more easily be able to determine what version to put for a JIRA they want
> >to get committed to trunk. I've seen plenty of users who are confused by
> >having to set "0.24.0" as the version indicating trunk.
> >
> >There's also a nice symmetry in having the branch in svn/git (named trunk)
> >have the same name as the JIRA version indicator.
>
> My proposal was from a few months back related to the naming of versions
> in maven.
> It boils down to 'laziness is a virtue' + 'the maven version should match
> the branch'.
> Pick a name for a branch when you actually branch, not months before.
> 'trunk' in svn == 'trunk-SNAPSHOT' in maven == 'trunk' in JIRA.
>
> The creation of a branch should avoid impacting the place branched from
> (i.e. cause work for people on trunk because of a branch, or cause work
> for a branch due to a tag, etc).
> If the version in 'branch/2.0.x' has maven version '2.0.x-SNAPSHOT' and
> JIRA tag '2.0.0' then tagging 2.0.0 has the following steps:
>
> $> svn cp branch/2.0.x tags/2.0.0
> $> cd tags/2.0.0
> $> mvn versoins:set -DnewVersion=2.0.0
> $> svn diff   (spot check pom changes that versions:set did)
> $> mvn versions:commit
> $> svn commit
>
> The result is a new tag with the version set, and no changes to the branch
> that was tagged from.
> When the decision is made to have a 2.0.1, a jira tag can be made (or a
> 2.0.x tag renamed to 2.0.1, and a new 2.0.x tag made as Todd suggests).
> Being lazy here helps because maybe instead after some development on the
> 2.0.x branch it is decided to branch 2.1.x from it.
>
> When the decision to branch 2.1 occurs, a new branch 'branch/2.1.x' is
> made in svn, its version becomes '2.1.x-SNAPSHOT' and a JIRA tag for
> '2.1.x' is made or renamed from an old one.  The old 2.0.x branch is
> unchanged (and available for more minor bugfix releases).
>
> Once trunk or a branch is set up for build scripts or hudson, etc, it
> works without modification no matter how many times a branch or tag occurs
> off of it.
>
>
> >
> >
> >>
> >> Given the above I'd stick with 3.0.0 since it means lesser confusion and
> >> lesser work for the RM on future major releases.

Arun,

I am even more confused now than I was before:

Here you say:
> Essentially 'trunk' is where incompatible changes *may* be committed (in
>future). We should allow for that.

On another thread, responding to Avner (re: MAPREDUCE-4049?) you say,

> We do expect 'new features' to make it to trunk before we can commit to
>either branch-1 or branch-2.
Which one is it ?

Do you expect that "new features" will always remain compatible ?

- Milind

---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)

Hi Milind,

On Tue, Apr 3, 2012 at 11:27 AM, <[EMAIL PROTECTED]> wrote:

> Here you say:
>
>
> > Essentially 'trunk' is where incompatible changes *may* be committed (in
> >future). We should allow for that.
>

What I believe Arun is alluding to here is that we expect for compatibility
to be maintained for the lifetime of a major release branch.
>
> On another thread, responding to Avner (re: MAPREDUCE-4049?) you say,
>
> > We do expect 'new features' to make it to trunk before we can commit to
> >either branch-1 or branch-2.
>
>
> Which one is it ?
>

These two statements aren't mutually exclusive. We require that all new
features go to trunk first so as to ensure that future releases are
supersets of the functionality of previous releases, except in the case of
explicit deprecation. Only once it's committed to trunk may it be
back-ported to an earlier branch.
>
> Do you expect that "new features" will always remain compatible ?
>

Not necessarily, but only if a feature is compatible may it be back-ported
to major release branches.

--
Aaron T. Myers
Software Engineer, Cloudera

Thanks ATM.

I guess the "*may*" emphasis confused me.

Just to get some more clarity:

What would be guideline for a new feature, such as
https://issues.apache.org/jira/browse/MAPREDUCE-4049, which maintains
compatibility for 1.x, but is not relevant to trunk, because the codebases
have completely diverged, so cannot be committed to trunk ?

- Milind

---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)
On 4/3/12 1:58 PM, "Aaron T. Myers" <[EMAIL PROTECTED]> wrote:

>Hi Milind,
>
>On Tue, Apr 3, 2012 at 11:27 AM, <[EMAIL PROTECTED]> wrote:
>
>> Here you say:
>>
>>
>> > Essentially 'trunk' is where incompatible changes *may* be committed
>>(in
>> >future). We should allow for that.
>>
>
>What I believe Arun is alluding to here is that we expect for
>compatibility
>to be maintained for the lifetime of a major release branch.
>
>
>>
>> On another thread, responding to Avner (re: MAPREDUCE-4049?) you say,
>>
>> > We do expect 'new features' to make it to trunk before we can commit
>>to
>> >either branch-1 or branch-2.
>>
>>
>> Which one is it ?
>>
>
>These two statements aren't mutually exclusive. We require that all new
>features go to trunk first so as to ensure that future releases are
>supersets of the functionality of previous releases, except in the case of
>explicit deprecation. Only once it's committed to trunk may it be
>back-ported to an earlier branch.
>
>
>>
>> Do you expect that "new features" will always remain compatible ?
>>
>
>Not necessarily, but only if a feature is compatible may it be back-ported
>to major release branches.
>
>--
>Aaron T. Myers
>Software Engineer, Cloudera

On Tue, Apr 3, 2012 at 2:37 PM, <[EMAIL PROTECTED]> wrote:

> What would be guideline for a new feature, such as
> https://issues.apache.org/jira/browse/MAPREDUCE-4049, which maintains
> compatibility for 1.x, but is not relevant to trunk, because the codebases
> have completely diverged, so cannot be committed to trunk ?
>

Are you sure this isn't relevant to trunk? The "target versions" field of
that JIRA lists both 1.0.x and 0.24.0 (trunk.) In the latest comment on
that JIRA, the author appears to intend to do this work for both trunk and
1.0:

"I want to have the described plugin-ability (desired with same interface)
for all future versions of Hadoop (as mentioned in the Target Version/s
field). <snip> On the first phase, I am focusing on the existing 1.0 branch
as I know it. In parallel, I'll try to learn what exists in 0.23"

--
Aaron T. Myers
Software Engineer, Cloudera

To my knowledge, shuffle is already pluggable in 0.23 onwards, as long as
it is used only by mapreduce framework.

That's why Avner says : "In parallel, I'll try to *learn what exists* in
0.23". (Emphasize my own.)

That's why I was wondering about the insistence of committing to trunk
first.

- Milind

---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)

On 4/3/12 2:44 PM, "Aaron T. Myers" <[EMAIL PROTECTED]> wrote:

>On Tue, Apr 3, 2012 at 2:37 PM, <[EMAIL PROTECTED]> wrote:
>
>> What would be guideline for a new feature, such as
>> https://issues.apache.org/jira/browse/MAPREDUCE-4049, which maintains
>> compatibility for 1.x, but is not relevant to trunk, because the
>>codebases
>> have completely diverged, so cannot be committed to trunk ?
>>
>
>Are you sure this isn't relevant to trunk? The "target versions" field of
>that JIRA lists both 1.0.x and 0.24.0 (trunk.) In the latest comment on
>that JIRA, the author appears to intend to do this work for both trunk and
>1.0:
>
>"I want to have the described plugin-ability (desired with same interface)
>for all future versions of Hadoop (as mentioned in the Target Version/s
>field). <snip> On the first phase, I am focusing on the existing 1.0
>branch
>as I know it. In parallel, I'll try to learn what exists in 0.23"
>
>--
>Aaron T. Myers
>Software Engineer, Cloudera

If that's the case then there doesn't seem to be any question here. The
feature is in trunk, and an implementation could be done for an older
release branch that would be compatible with that branch. Sure, the code to
implement the feature is quite different between the two branches, but
trunk will remain a superset of the functionality of the past release, so
no issue.

--
Aaron T. Myers
Software Engineer, Cloudera

On Tue, Apr 3, 2012 at 3:14 PM, <[EMAIL PROTECTED]> wrote:

> To my knowledge, shuffle is already pluggable in 0.23 onwards, as long as
> it is used only by mapreduce framework.
>
> That's why Avner says : "In parallel, I'll try to *learn what exists* in
> 0.23". (Emphasize my own.)
>
> That's why I was wondering about the insistence of committing to trunk
> first.
>
> - Milind
>
> ---
> Milind Bhandarkar
> Greenplum Labs, EMC
> (Disclaimer: Opinions expressed in this email are those of the author, and
> do not necessarily represent the views of any organization, past or
> present, the author might be affiliated with.)
>
>
>
> On 4/3/12 2:44 PM, "Aaron T. Myers" <[EMAIL PROTECTED]> wrote:
>
> >On Tue, Apr 3, 2012 at 2:37 PM, <[EMAIL PROTECTED]> wrote:
> >
> >> What would be guideline for a new feature, such as
> >> https://issues.apache.org/jira/browse/MAPREDUCE-4049, which maintains
> >> compatibility for 1.x, but is not relevant to trunk, because the
> >>codebases
> >> have completely diverged, so cannot be committed to trunk ?
> >>
> >
> >Are you sure this isn't relevant to trunk? The "target versions" field of
> >that JIRA lists both 1.0.x and 0.24.0 (trunk.) In the latest comment on
> >that JIRA, the author appears to intend to do this work for both trunk and
> >1.0:
> >
> >"I want to have the described plugin-ability (desired with same interface)
> >for all future versions of Hadoop (as mentioned in the Target Version/s
> >field). <snip> On the first phase, I am focusing on the existing 1.0
> >branch
> >as I know it. In parallel, I'll try to learn what exists in 0.23"
> >
> >--
> >Aaron T. Myers
> >Software Engineer, Cloudera
>
>

Great !

Thanks @atm,

- milind

On 4/3/12 3:21 PM, "Aaron T. Myers" <[EMAIL PROTECTED]> wrote:

>If that's the case then there doesn't seem to be any question here. The
>feature is in trunk, and an implementation could be done for an older
>release branch that would be compatible with that branch. Sure, the code
>to
>implement the feature is quite different between the two branches, but
>trunk will remain a superset of the functionality of the past release, so
>no issue.
>
>--
>Aaron T. Myers
>Software Engineer, Cloudera
>
>
>
>On Tue, Apr 3, 2012 at 3:14 PM, <[EMAIL PROTECTED]> wrote:
>
>> To my knowledge, shuffle is already pluggable in 0.23 onwards, as long
>>as
>> it is used only by mapreduce framework.
>>
>> That's why Avner says : "In parallel, I'll try to *learn what exists* in
>> 0.23". (Emphasize my own.)
>>
>> That's why I was wondering about the insistence of committing to trunk
>> first.
>>
>> - Milind
>>
>> ---
>> Milind Bhandarkar
>> Greenplum Labs, EMC
>> (Disclaimer: Opinions expressed in this email are those of the author,
>>and
>> do not necessarily represent the views of any organization, past or
>> present, the author might be affiliated with.)
>>
>>
>>
>> On 4/3/12 2:44 PM, "Aaron T. Myers" <[EMAIL PROTECTED]> wrote:
>>
>> >On Tue, Apr 3, 2012 at 2:37 PM, <[EMAIL PROTECTED]> wrote:
>> >
>> >> What would be guideline for a new feature, such as
>> >> https://issues.apache.org/jira/browse/MAPREDUCE-4049, which maintains
>> >> compatibility for 1.x, but is not relevant to trunk, because the
>> >>codebases
>> >> have completely diverged, so cannot be committed to trunk ?
>> >>
>> >
>> >Are you sure this isn't relevant to trunk? The "target versions" field
>>of
>> >that JIRA lists both 1.0.x and 0.24.0 (trunk.) In the latest comment on
>> >that JIRA, the author appears to intend to do this work for both trunk
>>and
>> >1.0:
>> >
>> >"I want to have the described plugin-ability (desired with same
>>interface)
>> >for all future versions of Hadoop (as mentioned in the Target Version/s
>> >field). <snip> On the first phase, I am focusing on the existing 1.0
>> >branch
>> >as I know it. In parallel, I'll try to learn what exists in 0.23"
>> >
>> >--
>> >Aaron T. Myers
>> >Software Engineer, Cloudera
>>
>>

On 28 March 2012 06:31, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> >> (3) Rename branch-0.23 to branch-2, keep branch-0.22 as-is.
Quick follow up on this

-there is now a branch-2 that replaced branch-0.23,

http://svn.eu.apache.org/viewvc/hadoop/common/branches/branch-0.23/

-but there is still a 0.23.2 that is having some active work on it

http://svn.eu.apache.org/viewvc/hadoop/common/branches/branch-2/

Can I verify then that If i were to push changes into trunk and then into
the Hadoop 2.0 branch, I should check out the branch-2 branch and ignore
the older 0.23, branch?

And if that is so, should the 0.23 branch still be active at all?

On Thu, Apr 12, 2012 at 9:32 AM, Steve Loughran
<[EMAIL PROTECTED]> wrote:
> On 28 March 2012 06:31, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>
>> >> (3) Rename branch-0.23 to branch-2, keep branch-0.22 as-is.
>
>
> Quick follow up on this
>
> -there is now a branch-2 that replaced branch-0.23,
>
> http://svn.eu.apache.org/viewvc/hadoop/common/branches/branch-0.23/
>
> -but there is still a 0.23.2 that is having some active work on it
>
> http://svn.eu.apache.org/viewvc/hadoop/common/branches/branch-2/
>
> Can I verify then that If i were to push changes into trunk and then into
> the Hadoop 2.0 branch, I should check out the branch-2 branch and ignore
> the older 0.23, branch?

Yes, that's what folks are mostly doing.

>
> And if that is so, should the 0.23 branch still be active at all?

Bobby Evans took over management of branch-0.23 to work on for some
internal customers at Yahoo, as I understand it. See the thread:
"[DISCUSS] branch-0.23 is dead, long live branch-0.23" on general@ for
info.

-Todd
--
Todd Lipcon
Software Engineer, Cloudera

Steve,

Todd is correct, we are running two yarn trains here at Yahoo.  We are trying to stabilize 0.23 and get it pushed out to production, while also working on stabilizing branch-2.  Once branch-2 truly stabilizes we will switch over to it and retire branch-0.23.  We may call for a vote on a few official Apache releases off of 0.23 if there is interest in it outside of just Yahoo!

--Bobby Evans

On 4/12/12 12:27 PM, "Todd Lipcon" <[EMAIL PROTECTED]> wrote:

On Thu, Apr 12, 2012 at 9:32 AM, Steve Loughran
<[EMAIL PROTECTED]> wrote:
> On 28 March 2012 06:31, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>
>> >> (3) Rename branch-0.23 to branch-2, keep branch-0.22 as-is.
>
>
> Quick follow up on this
>
> -there is now a branch-2 that replaced branch-0.23,
>
> http://svn.eu.apache.org/viewvc/hadoop/common/branches/branch-0.23/
>
> -but there is still a 0.23.2 that is having some active work on it
>
> http://svn.eu.apache.org/viewvc/hadoop/common/branches/branch-2/
>
> Can I verify then that If i were to push changes into trunk and then into
> the Hadoop 2.0 branch, I should check out the branch-2 branch and ignore
> the older 0.23, branch?

Yes, that's what folks are mostly doing.

>
> And if that is so, should the 0.23 branch still be active at all?

Bobby Evans took over management of branch-0.23 to work on for some
internal customers at Yahoo, as I understand it. See the thread:
"[DISCUSS] branch-0.23 is dead, long live branch-0.23" on general@ for
info.

-Todd
--
Todd Lipcon
Software Engineer, Cloudera