On the heels of the planned 0.23.5 release (thanks Bobby & Thomas) I want to rollout a hadoop-2.0.3 release to reflect the growing stability of YARN.

I'm hoping we can also release the QJM along-with; hence I'd love to know an ETA - Todd? Sanjay? Suresh?

One other thing which would be nice henceforth is to better reflect release content for end-users in release-notes etc.; thus, can I ask committers to start paying closer attention to bug classification such as Blocker/Critical/Major/Minor etc.? This way, as we get closer to stable hadoop-2 releases, we can do a better job communicating content and it's criticality.

thanks,
Arun

+1 on having QJM in hadoop-2.0.3. Any rough estimate when this is targeted
for?

2012/11/15 Arun C Murthy <[EMAIL PROTECTED]>

> On the heels of the planned 0.23.5 release (thanks Bobby & Thomas) I want
> to rollout a hadoop-2.0.3 release to reflect the growing stability of YARN.
>
> I'm hoping we can also release the QJM along-with; hence I'd love to know
> an ETA - Todd? Sanjay? Suresh?
>
> One other thing which would be nice henceforth is to better reflect
> release content for end-users in release-notes etc.; thus, can I ask
> committers to start paying closer attention to bug classification such as
> Blocker/Critical/Major/Minor etc.? This way, as we get closer to stable
> hadoop-2 releases, we can do a better job communicating content and it's
> criticality.
>
> thanks,
> Arun
>
>
--
Have a Nice Day!
Lohit

+1 from me, too. I wanted to let it sit in trunk for a few weeks to see if
anyone found issues, but it's now been a bit over a month all the feedback
I've gotten so far has been good, tests have been stable, etc.

Unless anyone votes otherwise, I'll start backporting the patches into
branch-2.

Todd

On Fri, Nov 16, 2012 at 12:58 PM, lohit <[EMAIL PROTECTED]> wrote:

> +1 on having QJM in hadoop-2.0.3. Any rough estimate when this is targeted
> for?
>
> 2012/11/15 Arun C Murthy <[EMAIL PROTECTED]>
>
> > On the heels of the planned 0.23.5 release (thanks Bobby & Thomas) I want
> > to rollout a hadoop-2.0.3 release to reflect the growing stability of
> YARN.
> >
> > I'm hoping we can also release the QJM along-with; hence I'd love to know
> > an ETA - Todd? Sanjay? Suresh?
> >
> > One other thing which would be nice henceforth is to better reflect
> > release content for end-users in release-notes etc.; thus, can I ask
> > committers to start paying closer attention to bug classification such as
> > Blocker/Critical/Major/Minor etc.? This way, as we get closer to stable
> > hadoop-2 releases, we can do a better job communicating content and it's
> > criticality.
> >
> > thanks,
> > Arun
> >
> >
>
>
> --
> Have a Nice Day!
> Lohit
>

--
Todd Lipcon
Software Engineer, Cloudera

Here's a git branch with the backported changes in case anyone has time to
take a look this weekend:

https://github.com/toddlipcon/hadoop-common/tree/branch-2-QJM

There were a few conflicts due to patches committed in different orders,
and I had to pull in a couple other JIRAs along the way, but it is passing
its tests. If it looks good I'll start putting up the patches on JIRA and
committing next week.

-Todd

On Fri, Nov 16, 2012 at 1:14 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> +1 from me, too. I wanted to let it sit in trunk for a few weeks to see if
> anyone found issues, but it's now been a bit over a month all the feedback
> I've gotten so far has been good, tests have been stable, etc.
>
> Unless anyone votes otherwise, I'll start backporting the patches into
> branch-2.
>
> Todd
>
> On Fri, Nov 16, 2012 at 12:58 PM, lohit <[EMAIL PROTECTED]>wrote:
>
>> +1 on having QJM in hadoop-2.0.3. Any rough estimate when this is targeted
>> for?
>>
>> 2012/11/15 Arun C Murthy <[EMAIL PROTECTED]>
>>
>> > On the heels of the planned 0.23.5 release (thanks Bobby & Thomas) I
>> want
>> > to rollout a hadoop-2.0.3 release to reflect the growing stability of
>> YARN.
>> >
>> > I'm hoping we can also release the QJM along-with; hence I'd love to
>> know
>> > an ETA - Todd? Sanjay? Suresh?
>> >
>> > One other thing which would be nice henceforth is to better reflect
>> > release content for end-users in release-notes etc.; thus, can I ask
>> > committers to start paying closer attention to bug classification such
>> as
>> > Blocker/Critical/Major/Minor etc.? This way, as we get closer to stable
>> > hadoop-2 releases, we can do a better job communicating content and it's
>> > criticality.
>> >
>> > thanks,
>> > Arun
>> >
>> >
>>
>>
>> --
>> Have a Nice Day!
>> Lohit
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

--
Todd Lipcon
Software Engineer, Cloudera

Hi Arun,

Given that the 2.0.3 release is intended to reflect the growing stability
of YARN, and the QJM work will be included in 2.0.3 which provides a
complete HDFS HA solution, I think it's time we consider removing the
"-alpha" label from the release version. My preference would be to remove
the label entirely, but we could also perhaps call it "-beta" or something.

Thoughts?

--
Aaron T. Myers
Software Engineer, Cloudera

On Thu, Nov 15, 2012 at 10:41 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> On the heels of the planned 0.23.5 release (thanks Bobby & Thomas) I want
> to rollout a hadoop-2.0.3 release to reflect the growing stability of YARN.
>
> I'm hoping we can also release the QJM along-with; hence I'd love to know
> an ETA - Todd? Sanjay? Suresh?
>
> One other thing which would be nice henceforth is to better reflect
> release content for end-users in release-notes etc.; thus, can I ask
> committers to start paying closer attention to bug classification such as
> Blocker/Critical/Major/Minor etc.? This way, as we get closer to stable
> hadoop-2 releases, we can do a better job communicating content and it's
> criticality.
>
> thanks,
> Arun
>
>

I would recommend https://issues.apache.org/jira/browse/MAPREDUCE-2454 for
inclusion.

- milind

---
Milind Bhandarkar
Chief Scientist,
Machine Learning Platforms,
Greenplum, A Division of EMC
+1-650-523-3858 (W)
+1-408-666-8483 (C)

On 11/16/12 3:38 PM, "Aaron T. Myers" <[EMAIL PROTECTED]> wrote:

>Hi Arun,
>
>Given that the 2.0.3 release is intended to reflect the growing stability
>of YARN, and the QJM work will be included in 2.0.3 which provides a
>complete HDFS HA solution, I think it's time we consider removing the
>"-alpha" label from the release version. My preference would be to remove
>the label entirely, but we could also perhaps call it "-beta" or
>something.
>
>Thoughts?
>
>--
>Aaron T. Myers
>Software Engineer, Cloudera
>
>
>
>On Thu, Nov 15, 2012 at 10:41 PM, Arun C Murthy <[EMAIL PROTECTED]>
>wrote:
>
>> On the heels of the planned 0.23.5 release (thanks Bobby & Thomas) I
>>want
>> to rollout a hadoop-2.0.3 release to reflect the growing stability of
>>YARN.
>>
>> I'm hoping we can also release the QJM along-with; hence I'd love to
>>know
>> an ETA - Todd? Sanjay? Suresh?
>>
>> One other thing which would be nice henceforth is to better reflect
>> release content for end-users in release-notes etc.; thus, can I ask
>> committers to start paying closer attention to bug classification such
>>as
>> Blocker/Critical/Major/Minor etc.? This way, as we get closer to stable
>> hadoop-2 releases, we can do a better job communicating content and it's
>> criticality.
>>
>> thanks,
>> Arun
>>
>>

Agree with Milind, I'd also like to see MAPREDUCE-2454 in it.

Thx

On Fri, Nov 16, 2012 at 4:05 PM, Bhandarkar, Milind
<[EMAIL PROTECTED]> wrote:
> I would recommend https://issues.apache.org/jira/browse/MAPREDUCE-2454 for
> inclusion.
>
> - milind
>
> ---
> Milind Bhandarkar
> Chief Scientist,
> Machine Learning Platforms,
> Greenplum, A Division of EMC
> +1-650-523-3858 (W)
> +1-408-666-8483 (C)
>
>
>
>
>
> On 11/16/12 3:38 PM, "Aaron T. Myers" <[EMAIL PROTECTED]> wrote:
>
>>Hi Arun,
>>
>>Given that the 2.0.3 release is intended to reflect the growing stability
>>of YARN, and the QJM work will be included in 2.0.3 which provides a
>>complete HDFS HA solution, I think it's time we consider removing the
>>"-alpha" label from the release version. My preference would be to remove
>>the label entirely, but we could also perhaps call it "-beta" or
>>something.
>>
>>Thoughts?
>>
>>--
>>Aaron T. Myers
>>Software Engineer, Cloudera
>>
>>
>>
>>On Thu, Nov 15, 2012 at 10:41 PM, Arun C Murthy <[EMAIL PROTECTED]>
>>wrote:
>>
>>> On the heels of the planned 0.23.5 release (thanks Bobby & Thomas) I
>>>want
>>> to rollout a hadoop-2.0.3 release to reflect the growing stability of
>>>YARN.
>>>
>>> I'm hoping we can also release the QJM along-with; hence I'd love to
>>>know
>>> an ETA - Todd? Sanjay? Suresh?
>>>
>>> One other thing which would be nice henceforth is to better reflect
>>> release content for end-users in release-notes etc.; thus, can I ask
>>> committers to start paying closer attention to bug classification such
>>>as
>>> Blocker/Critical/Major/Minor etc.? This way, as we get closer to stable
>>> hadoop-2 releases, we can do a better job communicating content and it's
>>> criticality.
>>>
>>> thanks,
>>> Arun
>>>
>>>
>

--
Alejandro

On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
> Hi Arun,
>
> Given that the 2.0.3 release is intended to reflect the growing stability
> of YARN, and the QJM work will be included in 2.0.3 which provides a
> complete HDFS HA solution, I think it's time we consider removing the
> "-alpha" label from the release version. My preference would be to remove
> the label entirely, but we could also perhaps call it "-beta" or something.
>
> Thoughts?
>

I think it fine after two minor releases undoing the '-alpha' suffix.

If folks insist we next go to '-beta', I'd hope we'd travel all
remaining 22 letters of the greek alphabet before we 2.0.x.

St.Ack

I am OK with removing the alpha assuming that we think that the APIs are
stable enough that we are willing to truly start maintaining backwards
compatibility on them within 2.X. From what I have seen I think that they
are fairly stable and I think there is enough adoption by other projects
right now that breaking backwards compatibility would be problematic.

--Bobby Evans

On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:

>On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
>> Hi Arun,
>>
>> Given that the 2.0.3 release is intended to reflect the growing
>>stability
>> of YARN, and the QJM work will be included in 2.0.3 which provides a
>> complete HDFS HA solution, I think it's time we consider removing the
>> "-alpha" label from the release version. My preference would be to
>>remove
>> the label entirely, but we could also perhaps call it "-beta" or
>>something.
>>
>> Thoughts?
>>
>
>I think it fine after two minor releases undoing the '-alpha' suffix.
>
>If folks insist we next go to '-beta', I'd hope we'd travel all
>remaining 22 letters of the greek alphabet before we 2.0.x.
>
>St.Ack

We really need https://issues.apache.org/jira/browse/MAPREDUCE-4805 too.
The history server is completely useless without it.

--Bobby

On 11/16/12 10:27 PM, "Alejandro Abdelnur" <[EMAIL PROTECTED]> wrote:

>Agree with Milind, I'd also like to see MAPREDUCE-2454 in it.
>
>Thx
>
>On Fri, Nov 16, 2012 at 4:05 PM, Bhandarkar, Milind
><[EMAIL PROTECTED]> wrote:
>> I would recommend https://issues.apache.org/jira/browse/MAPREDUCE-2454
>>for
>> inclusion.
>>
>> - milind
>>
>> ---
>> Milind Bhandarkar
>> Chief Scientist,
>> Machine Learning Platforms,
>> Greenplum, A Division of EMC
>> +1-650-523-3858 (W)
>> +1-408-666-8483 (C)
>>
>>
>>
>>
>>
>> On 11/16/12 3:38 PM, "Aaron T. Myers" <[EMAIL PROTECTED]> wrote:
>>
>>>Hi Arun,
>>>
>>>Given that the 2.0.3 release is intended to reflect the growing
>>>stability
>>>of YARN, and the QJM work will be included in 2.0.3 which provides a
>>>complete HDFS HA solution, I think it's time we consider removing the
>>>"-alpha" label from the release version. My preference would be to
>>>remove
>>>the label entirely, but we could also perhaps call it "-beta" or
>>>something.
>>>
>>>Thoughts?
>>>
>>>--
>>>Aaron T. Myers
>>>Software Engineer, Cloudera
>>>
>>>
>>>
>>>On Thu, Nov 15, 2012 at 10:41 PM, Arun C Murthy <[EMAIL PROTECTED]>
>>>wrote:
>>>
>>>> On the heels of the planned 0.23.5 release (thanks Bobby & Thomas) I
>>>>want
>>>> to rollout a hadoop-2.0.3 release to reflect the growing stability of
>>>>YARN.
>>>>
>>>> I'm hoping we can also release the QJM along-with; hence I'd love to
>>>>know
>>>> an ETA - Todd? Sanjay? Suresh?
>>>>
>>>> One other thing which would be nice henceforth is to better reflect
>>>> release content for end-users in release-notes etc.; thus, can I ask
>>>> committers to start paying closer attention to bug classification such
>>>>as
>>>> Blocker/Critical/Major/Minor etc.? This way, as we get closer to
>>>>stable
>>>> hadoop-2 releases, we can do a better job communicating content and
>>>>it's
>>>> criticality.
>>>>
>>>> thanks,
>>>> Arun
>>>>
>>>>
>>
>
>
>
>--
>Alejandro

I also agree with Millind and Alejandro.

Thanks.

-- Asokan

Agree with Milind, I'd also like to see MAPREDUCE-2454 in it.

Thx

On Fri, Nov 16, 2012 at 4:05 PM, Bhandarkar, Milind
<[EMAIL PROTECTED]> wrote:
> I would recommend https://issues.apache.org/jira/browse/MAPREDUCE-2454 for
> inclusion.
>
> - milind
>
> ---
> Milind Bhandarkar
> Chief Scientist,
> Machine Learning Platforms,
> Greenplum, A Division of EMC
> +1-650-523-3858 (W)
> +1-408-666-8483 (C)
>
>
>
>
>
> On 11/16/12 3:38 PM, "Aaron T. Myers" <[EMAIL PROTECTED]> wrote:
>
>>Hi Arun,
>>
>>Given that the 2.0.3 release is intended to reflect the growing stability
>>of YARN, and the QJM work will be included in 2.0.3 which provides a
>>complete HDFS HA solution, I think it's time we consider removing the
>>"-alpha" label from the release version. My preference would be to remove
>>the label entirely, but we could also perhaps call it "-beta" or
>>something.
>>
>>Thoughts?
>>
>>--
>>Aaron T. Myers
>>Software Engineer, Cloudera
>>
>>
>>
>>On Thu, Nov 15, 2012 at 10:41 PM, Arun C Murthy <[EMAIL PROTECTED]>
>>wrote:
>>
>>> On the heels of the planned 0.23.5 release (thanks Bobby & Thomas) I
>>>want
>>> to rollout a hadoop-2.0.3 release to reflect the growing stability of
>>>YARN.
>>>
>>> I'm hoping we can also release the QJM along-with; hence I'd love to
>>>know
>>> an ETA - Todd? Sanjay? Suresh?
>>>
>>> One other thing which would be nice henceforth is to better reflect
>>> release content for end-users in release-notes etc.; thus, can I ask
>>> committers to start paying closer attention to bug classification such
>>>as
>>> Blocker/Critical/Major/Minor etc.? This way, as we get closer to stable
>>> hadoop-2 releases, we can do a better job communicating content and it's
>>> criticality.
>>>
>>> thanks,
>>> Arun
>>>
>>>
>

--
Alejandro

+1 for MAPREDUCE-2424
Best wishes
----- Mensaje original -----
De: Mariappan Asokan <[EMAIL PROTECTED]>
Para: [EMAIL PROTECTED]
Enviado: Mon, 19 Nov 2012 12:18:34 -0500 (EST)
Asunto: Re: Heads Up - hadoop-2.0.3 release

I also agree with Millind and Alejandro.

Thanks.

-- Asokan

Agree with Milind, I'd also like to see MAPREDUCE-2454 in it.

Thx

On Fri, Nov 16, 2012 at 4:05 PM, Bhandarkar, Milind
<[EMAIL PROTECTED]> wrote:
> I would recommend https://issues.apache.org/jira/browse/MAPREDUCE-2454 for
> inclusion.
>
> - milind
>
> ---
> Milind Bhandarkar
> Chief Scientist,
> Machine Learning Platforms,
> Greenplum, A Division of EMC
> +1-650-523-3858 (W)
> +1-408-666-8483 (C)
>
>
>
>
>
> On 11/16/12 3:38 PM, "Aaron T. Myers" <[EMAIL PROTECTED]> wrote:
>
>>Hi Arun,
>>
>>Given that the 2.0.3 release is intended to reflect the growing stability
>>of YARN, and the QJM work will be included in 2.0.3 which provides a
>>complete HDFS HA solution, I think it's time we consider removing the
>>"-alpha" label from the release version. My preference would be to remove
>>the label entirely, but we could also perhaps call it "-beta" or
>>something.
>>
>>Thoughts?
>>
>>--
>>Aaron T. Myers
>>Software Engineer, Cloudera
>>
>>
>>
>>On Thu, Nov 15, 2012 at 10:41 PM, Arun C Murthy <[EMAIL PROTECTED]>
>>wrote:
>>
>>> On the heels of the planned 0.23.5 release (thanks Bobby & Thomas) I
>>>want
>>> to rollout a hadoop-2.0.3 release to reflect the growing stability of
>>>YARN.
>>>
>>> I'm hoping we can also release the QJM along-with; hence I'd love to
>>>know
>>> an ETA - Todd? Sanjay? Suresh?
>>>
>>> One other thing which would be nice henceforth is to better reflect
>>> release content for end-users in release-notes etc.; thus, can I ask
>>> committers to start paying closer attention to bug classification such
>>>as
>>> Blocker/Critical/Major/Minor etc.? This way, as we get closer to stable
>>> hadoop-2 releases, we can do a better job communicating content and it's
>>> criticality.
>>>
>>> thanks,
>>> Arun
>>>
>>>
>

--
Alejandro
10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci

10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci

YARN-142/MAPREDUCE-4067 should ideally be fixed before we commit to API
backward compatibility. Also, from the recent YARN meetup - there seemed to
be a requirement to change the AM-RM protocol for container requests. In
this case, I believe it's OK to not have all functionality implemented, as
long as the protocol itself can represent the requirements. However, as
Bobby pointed out, given the current adoption by other projects -
incompatible changes at this point can be problematic and needs to be
figured out.

Thanks
- Sid
On Mon, Nov 19, 2012 at 8:22 AM, Robert Evans <[EMAIL PROTECTED]> wrote:

> I am OK with removing the alpha assuming that we think that the APIs are
> stable enough that we are willing to truly start maintaining backwards
> compatibility on them within 2.X. From what I have seen I think that they
> are fairly stable and I think there is enough adoption by other projects
> right now that breaking backwards compatibility would be problematic.
>
> --Bobby Evans
>
> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>
> >On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
> >> Hi Arun,
> >>
> >> Given that the 2.0.3 release is intended to reflect the growing
> >>stability
> >> of YARN, and the QJM work will be included in 2.0.3 which provides a
> >> complete HDFS HA solution, I think it's time we consider removing the
> >> "-alpha" label from the release version. My preference would be to
> >>remove
> >> the label entirely, but we could also perhaps call it "-beta" or
> >>something.
> >>
> >> Thoughts?
> >>
> >
> >I think it fine after two minor releases undoing the '-alpha' suffix.
> >
> >If folks insist we next go to '-beta', I'd hope we'd travel all
> >remaining 22 letters of the greek alphabet before we 2.0.x.
> >
> >St.Ack
>
>

Agreed, I was about to point this out.

What else is on HDFS of this nature?

Arun

On Nov 19, 2012, at 10:09 AM, Siddharth Seth wrote:

> YARN-142/MAPREDUCE-4067 should ideally be fixed before we commit to API
> backward compatibility. Also, from the recent YARN meetup - there seemed to
> be a requirement to change the AM-RM protocol for container requests. In
> this case, I believe it's OK to not have all functionality implemented, as
> long as the protocol itself can represent the requirements. However, as
> Bobby pointed out, given the current adoption by other projects -
> incompatible changes at this point can be problematic and needs to be
> figured out.
>
> Thanks
> - Sid
>
>
> On Mon, Nov 19, 2012 at 8:22 AM, Robert Evans <[EMAIL PROTECTED]> wrote:
>
>> I am OK with removing the alpha assuming that we think that the APIs are
>> stable enough that we are willing to truly start maintaining backwards
>> compatibility on them within 2.X. From what I have seen I think that they
>> are fairly stable and I think there is enough adoption by other projects
>> right now that breaking backwards compatibility would be problematic.
>>
>> --Bobby Evans
>>
>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>>
>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
>>>> Hi Arun,
>>>>
>>>> Given that the 2.0.3 release is intended to reflect the growing
>>>> stability
>>>> of YARN, and the QJM work will be included in 2.0.3 which provides a
>>>> complete HDFS HA solution, I think it's time we consider removing the
>>>> "-alpha" label from the release version. My preference would be to
>>>> remove
>>>> the label entirely, but we could also perhaps call it "-beta" or
>>>> something.
>>>>
>>>> Thoughts?
>>>>
>>>
>>> I think it fine after two minor releases undoing the '-alpha' suffix.
>>>
>>> If folks insist we next go to '-beta', I'd hope we'd travel all
>>> remaining 22 letters of the greek alphabet before we 2.0.x.
>>>
>>> St.Ack
>>
>>

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

I want to explicitly point out that the backward compatibility that is
being brought up here is related to APIs alone. I am fine with that as long
as it is for the APIs exposed to the users/applications. Incompatible API
changes, if necessary, among the daemons in Hadoop should be allowed. Also
if the wire protocols need an incompatible change, we should be able to
make those changes post beta. As long this is possible, I am okay with
dropping alpha from the name tag.

As regards some comments about dropping alpha is justified because we have
had two minor releases, I want to point out that the current release is
made of parts that are baked for long time and parts that are relatively
new. Some examples in HDFS, functionality such as Quorum journal Manager
and libwebhdfs etc. It would have been good to have a way to mark features
like that with alpha tag. How do we want to handle such content?
On Mon, Nov 19, 2012 at 10:09 AM, Siddharth Seth
<[EMAIL PROTECTED]>wrote:

> YARN-142/MAPREDUCE-4067 should ideally be fixed before we commit to API
> backward compatibility. Also, from the recent YARN meetup - there seemed to
> be a requirement to change the AM-RM protocol for container requests. In
> this case, I believe it's OK to not have all functionality implemented, as
> long as the protocol itself can represent the requirements. However, as
> Bobby pointed out, given the current adoption by other projects -
> incompatible changes at this point can be problematic and needs to be
> figured out.
>
> Thanks
> - Sid
>
>
> On Mon, Nov 19, 2012 at 8:22 AM, Robert Evans <[EMAIL PROTECTED]> wrote:
>
> > I am OK with removing the alpha assuming that we think that the APIs are
> > stable enough that we are willing to truly start maintaining backwards
> > compatibility on them within 2.X. From what I have seen I think that they
> > are fairly stable and I think there is enough adoption by other projects
> > right now that breaking backwards compatibility would be problematic.
> >
> > --Bobby Evans
> >
> > On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
> >
> > >On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]>
> wrote:
> > >> Hi Arun,
> > >>
> > >> Given that the 2.0.3 release is intended to reflect the growing
> > >>stability
> > >> of YARN, and the QJM work will be included in 2.0.3 which provides a
> > >> complete HDFS HA solution, I think it's time we consider removing the
> > >> "-alpha" label from the release version. My preference would be to
> > >>remove
> > >> the label entirely, but we could also perhaps call it "-beta" or
> > >>something.
> > >>
> > >> Thoughts?
> > >>
> > >
> > >I think it fine after two minor releases undoing the '-alpha' suffix.
> > >
> > >If folks insist we next go to '-beta', I'd hope we'd travel all
> > >remaining 22 letters of the greek alphabet before we 2.0.x.
> > >
> > >St.Ack
> >
> >
>

--
http://hortonworks.com/download/

I want to make some changes to the lifecycle of a yarn service (in a
backwards compatible way).

https://issues.apache.org/jira/browse/YARN-117
   1. formal state machine model with stop state idempotent and entry-able
   from any state
   2. waiting/blocked state a service can enter when waiting for something
   else
   3. an alternate base class that does the state model checks before
   executing any state change functions -currently its done at
   end-of-operation in the super() calls.
   4. gradual move of services to the stricter base class.

With a new base class nothing will break (as the move can be done
case-by-case, leaving the heavily subclassed ones alone); the state model
extensions & formalisation would be visible but not used.

I don't want to hold anything up, because I need more testing of things
before this is ready for review. I just want to get the fixes in before it
ships

On 19 November 2012 16:22, Robert Evans <[EMAIL PROTECTED]> wrote:

> I am OK with removing the alpha assuming that we think that the APIs are
> stable enough that we are willing to truly start maintaining backwards
> compatibility on them within 2.X. From what I have seen I think that they
> are fairly stable and I think there is enough adoption by other projects
> right now that breaking backwards compatibility would be problematic.
>
> --Bobby Evans
>
> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>
> >On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
> >> Hi Arun,
> >>
> >> Given that the 2.0.3 release is intended to reflect the growing
> >>stability
> >> of YARN, and the QJM work will be included in 2.0.3 which provides a
> >> complete HDFS HA solution, I think it's time we consider removing the
> >> "-alpha" label from the release version. My preference would be to
> >>remove
> >> the label entirely, but we could also perhaps call it "-beta" or
> >>something.
> >>
> >> Thoughts?
> >>
> >
> >I think it fine after two minor releases undoing the '-alpha' suffix.
> >
> >If folks insist we next go to '-beta', I'd hope we'd travel all
> >remaining 22 letters of the greek alphabet before we 2.0.x.
> >
> >St.Ack
>
>

+1 for the API changes. We need to be careful about back-compat but at the
same time the API's are the core of YARN and there seem to be some issues
around it.

Bikas

On 11/19/12 10:09 AM, "Siddharth Seth" <[EMAIL PROTECTED]> wrote:

>YARN-142/MAPREDUCE-4067 should ideally be fixed before we commit to API
>backward compatibility. Also, from the recent YARN meetup - there seemed
>to
>be a requirement to change the AM-RM protocol for container requests. In
>this case, I believe it's OK to not have all functionality implemented, as
>long as the protocol itself can represent the requirements. However, as
>Bobby pointed out, given the current adoption by other projects -
>incompatible changes at this point can be problematic and needs to be
>figured out.
>
>Thanks
>- Sid
>
>
>On Mon, Nov 19, 2012 at 8:22 AM, Robert Evans <[EMAIL PROTECTED]> wrote:
>
>> I am OK with removing the alpha assuming that we think that the APIs are
>> stable enough that we are willing to truly start maintaining backwards
>> compatibility on them within 2.X. From what I have seen I think that
>>they
>> are fairly stable and I think there is enough adoption by other projects
>> right now that breaking backwards compatibility would be problematic.
>>
>> --Bobby Evans
>>
>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>>
>> >On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]>
>>wrote:
>> >> Hi Arun,
>> >>
>> >> Given that the 2.0.3 release is intended to reflect the growing
>> >>stability
>> >> of YARN, and the QJM work will be included in 2.0.3 which provides a
>> >> complete HDFS HA solution, I think it's time we consider removing the
>> >> "-alpha" label from the release version. My preference would be to
>> >>remove
>> >> the label entirely, but we could also perhaps call it "-beta" or
>> >>something.
>> >>
>> >> Thoughts?
>> >>
>> >
>> >I think it fine after two minor releases undoing the '-alpha' suffix.
>> >
>> >If folks insist we next go to '-beta', I'd hope we'd travel all
>> >remaining 22 letters of the greek alphabet before we 2.0.x.
>> >
>> >St.Ack
>>
>>

On Mon, Nov 19, 2012 at 6:09 PM, Siddharth Seth
<[EMAIL PROTECTED]> wrote:
> YARN-142/MAPREDUCE-4067 should ideally be fixed before we commit to API
> backward compatibility. Also, from the recent YARN meetup - there seemed to
> be a requirement to change the AM-RM protocol for container requests. In
> this case, I believe it's OK to not have all functionality implemented, as
> long as the protocol itself can represent the requirements.

I agree. Do you think we can make these changes before removing the
'alpha' label, i.e. in 2.0.3? If that's not possible for the container
requests change, then we could mark AMRMProtocol (or related classes)
as @Evolving. Another alternative would be to introduce a new
interface.

> However, as
> Bobby pointed out, given the current adoption by other projects -
> incompatible changes at this point can be problematic and needs to be
> figured out.

We have a mechanism for this already. If something is marked as
@Evolving it can change incompatibly between minor versions - e.g.
2.0.x to 2.1.0. If it is @Stable then it can only change on major
versions, e.g. 2.x.y to 3.0.0. Let's make sure we are happy with the
annotations - and willing to support them at the indicated level -
before we remove the 'alpha' label. Of course, we strive not to change
APIs without a very good reason, but if we do we should do so within
the guidelines so that users know what to expect.

Cheers,
Tom

>
> Thanks
> - Sid
>
>
> On Mon, Nov 19, 2012 at 8:22 AM, Robert Evans <[EMAIL PROTECTED]> wrote:
>
>> I am OK with removing the alpha assuming that we think that the APIs are
>> stable enough that we are willing to truly start maintaining backwards
>> compatibility on them within 2.X. From what I have seen I think that they
>> are fairly stable and I think there is enough adoption by other projects
>> right now that breaking backwards compatibility would be problematic.
>>
>> --Bobby Evans
>>
>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>>
>> >On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
>> >> Hi Arun,
>> >>
>> >> Given that the 2.0.3 release is intended to reflect the growing
>> >>stability
>> >> of YARN, and the QJM work will be included in 2.0.3 which provides a
>> >> complete HDFS HA solution, I think it's time we consider removing the
>> >> "-alpha" label from the release version. My preference would be to
>> >>remove
>> >> the label entirely, but we could also perhaps call it "-beta" or
>> >>something.
>> >>
>> >> Thoughts?
>> >>
>> >
>> >I think it fine after two minor releases undoing the '-alpha' suffix.
>> >
>> >If folks insist we next go to '-beta', I'd hope we'd travel all
>> >remaining 22 letters of the greek alphabet before we 2.0.x.
>> >
>> >St.Ack
>>
>>

Hello Hadoop Release managers,
Any update on this?

Thanks,
Lohit

2012/11/20 Tom White <[EMAIL PROTECTED]>

> On Mon, Nov 19, 2012 at 6:09 PM, Siddharth Seth
> <[EMAIL PROTECTED]> wrote:
> > YARN-142/MAPREDUCE-4067 should ideally be fixed before we commit to API
> > backward compatibility. Also, from the recent YARN meetup - there seemed
> to
> > be a requirement to change the AM-RM protocol for container requests. In
> > this case, I believe it's OK to not have all functionality implemented,
> as
> > long as the protocol itself can represent the requirements.
>
> I agree. Do you think we can make these changes before removing the
> 'alpha' label, i.e. in 2.0.3? If that's not possible for the container
> requests change, then we could mark AMRMProtocol (or related classes)
> as @Evolving. Another alternative would be to introduce a new
> interface.
>
> > However, as
> > Bobby pointed out, given the current adoption by other projects -
> > incompatible changes at this point can be problematic and needs to be
> > figured out.
>
> We have a mechanism for this already. If something is marked as
> @Evolving it can change incompatibly between minor versions - e.g.
> 2.0.x to 2.1.0. If it is @Stable then it can only change on major
> versions, e.g. 2.x.y to 3.0.0. Let's make sure we are happy with the
> annotations - and willing to support them at the indicated level -
> before we remove the 'alpha' label. Of course, we strive not to change
> APIs without a very good reason, but if we do we should do so within
> the guidelines so that users know what to expect.
>
> Cheers,
> Tom
>
> >
> > Thanks
> > - Sid
> >
> >
> > On Mon, Nov 19, 2012 at 8:22 AM, Robert Evans <[EMAIL PROTECTED]>
> wrote:
> >
> >> I am OK with removing the alpha assuming that we think that the APIs are
> >> stable enough that we are willing to truly start maintaining backwards
> >> compatibility on them within 2.X. From what I have seen I think that
> they
> >> are fairly stable and I think there is enough adoption by other projects
> >> right now that breaking backwards compatibility would be problematic.
> >>
> >> --Bobby Evans
> >>
> >> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
> >>
> >> >On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]>
> wrote:
> >> >> Hi Arun,
> >> >>
> >> >> Given that the 2.0.3 release is intended to reflect the growing
> >> >>stability
> >> >> of YARN, and the QJM work will be included in 2.0.3 which provides a
> >> >> complete HDFS HA solution, I think it's time we consider removing the
> >> >> "-alpha" label from the release version. My preference would be to
> >> >>remove
> >> >> the label entirely, but we could also perhaps call it "-beta" or
> >> >>something.
> >> >>
> >> >> Thoughts?
> >> >>
> >> >
> >> >I think it fine after two minor releases undoing the '-alpha' suffix.
> >> >
> >> >If folks insist we next go to '-beta', I'd hope we'd travel all
> >> >remaining 22 letters of the greek alphabet before we 2.0.x.
> >> >
> >> >St.Ack
> >>
> >>
>

--
Have a Nice Day!
Lohit

Lohit,

 There are some outstanding blockers and I'm still awaiting the QJM merge.
 
 Feel free to watch the blocker list:
 http://s.apache.org/e1J

Arun

On Dec 3, 2012, at 10:02 AM, lohit wrote:

> Hello Hadoop Release managers,
> Any update on this?
>
> Thanks,
> Lohit
>
> 2012/11/20 Tom White <[EMAIL PROTECTED]>
>
>> On Mon, Nov 19, 2012 at 6:09 PM, Siddharth Seth
>> <[EMAIL PROTECTED]> wrote:
>>> YARN-142/MAPREDUCE-4067 should ideally be fixed before we commit to API
>>> backward compatibility. Also, from the recent YARN meetup - there seemed
>> to
>>> be a requirement to change the AM-RM protocol for container requests. In
>>> this case, I believe it's OK to not have all functionality implemented,
>> as
>>> long as the protocol itself can represent the requirements.
>>
>> I agree. Do you think we can make these changes before removing the
>> 'alpha' label, i.e. in 2.0.3? If that's not possible for the container
>> requests change, then we could mark AMRMProtocol (or related classes)
>> as @Evolving. Another alternative would be to introduce a new
>> interface.
>>
>>> However, as
>>> Bobby pointed out, given the current adoption by other projects -
>>> incompatible changes at this point can be problematic and needs to be
>>> figured out.
>>
>> We have a mechanism for this already. If something is marked as
>> @Evolving it can change incompatibly between minor versions - e.g.
>> 2.0.x to 2.1.0. If it is @Stable then it can only change on major
>> versions, e.g. 2.x.y to 3.0.0. Let's make sure we are happy with the
>> annotations - and willing to support them at the indicated level -
>> before we remove the 'alpha' label. Of course, we strive not to change
>> APIs without a very good reason, but if we do we should do so within
>> the guidelines so that users know what to expect.
>>
>> Cheers,
>> Tom
>>
>>>
>>> Thanks
>>> - Sid
>>>
>>>
>>> On Mon, Nov 19, 2012 at 8:22 AM, Robert Evans <[EMAIL PROTECTED]>
>> wrote:
>>>
>>>> I am OK with removing the alpha assuming that we think that the APIs are
>>>> stable enough that we are willing to truly start maintaining backwards
>>>> compatibility on them within 2.X. From what I have seen I think that
>> they
>>>> are fairly stable and I think there is enough adoption by other projects
>>>> right now that breaking backwards compatibility would be problematic.
>>>>
>>>> --Bobby Evans
>>>>
>>>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]>
>> wrote:
>>>>>> Hi Arun,
>>>>>>
>>>>>> Given that the 2.0.3 release is intended to reflect the growing
>>>>>> stability
>>>>>> of YARN, and the QJM work will be included in 2.0.3 which provides a
>>>>>> complete HDFS HA solution, I think it's time we consider removing the
>>>>>> "-alpha" label from the release version. My preference would be to
>>>>>> remove
>>>>>> the label entirely, but we could also perhaps call it "-beta" or
>>>>>> something.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>
>>>>> I think it fine after two minor releases undoing the '-alpha' suffix.
>>>>>
>>>>> If folks insist we next go to '-beta', I'd hope we'd travel all
>>>>> remaining 22 letters of the greek alphabet before we 2.0.x.
>>>>>
>>>>> St.Ack
>>>>
>>>>
>>
>
>
>
> --
> Have a Nice Day!
> Lohit

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

Hey Arun,

I put up patches for the QJM backport merge yesterday. Aaron said he'd
take a look at reviewing them, so I anticipate that to be finished
"real soon now". Sorry for the delay.

-Todd

On Tue, Dec 4, 2012 at 6:09 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> Lohit,
>
>  There are some outstanding blockers and I'm still awaiting the QJM merge.
>
>  Feel free to watch the blocker list:
>  http://s.apache.org/e1J
>
> Arun
>
> On Dec 3, 2012, at 10:02 AM, lohit wrote:
>
>> Hello Hadoop Release managers,
>> Any update on this?
>>
>> Thanks,
>> Lohit
>>
>> 2012/11/20 Tom White <[EMAIL PROTECTED]>
>>
>>> On Mon, Nov 19, 2012 at 6:09 PM, Siddharth Seth
>>> <[EMAIL PROTECTED]> wrote:
>>>> YARN-142/MAPREDUCE-4067 should ideally be fixed before we commit to API
>>>> backward compatibility. Also, from the recent YARN meetup - there seemed
>>> to
>>>> be a requirement to change the AM-RM protocol for container requests. In
>>>> this case, I believe it's OK to not have all functionality implemented,
>>> as
>>>> long as the protocol itself can represent the requirements.
>>>
>>> I agree. Do you think we can make these changes before removing the
>>> 'alpha' label, i.e. in 2.0.3? If that's not possible for the container
>>> requests change, then we could mark AMRMProtocol (or related classes)
>>> as @Evolving. Another alternative would be to introduce a new
>>> interface.
>>>
>>>> However, as
>>>> Bobby pointed out, given the current adoption by other projects -
>>>> incompatible changes at this point can be problematic and needs to be
>>>> figured out.
>>>
>>> We have a mechanism for this already. If something is marked as
>>> @Evolving it can change incompatibly between minor versions - e.g.
>>> 2.0.x to 2.1.0. If it is @Stable then it can only change on major
>>> versions, e.g. 2.x.y to 3.0.0. Let's make sure we are happy with the
>>> annotations - and willing to support them at the indicated level -
>>> before we remove the 'alpha' label. Of course, we strive not to change
>>> APIs without a very good reason, but if we do we should do so within
>>> the guidelines so that users know what to expect.
>>>
>>> Cheers,
>>> Tom
>>>
>>>>
>>>> Thanks
>>>> - Sid
>>>>
>>>>
>>>> On Mon, Nov 19, 2012 at 8:22 AM, Robert Evans <[EMAIL PROTECTED]>
>>> wrote:
>>>>
>>>>> I am OK with removing the alpha assuming that we think that the APIs are
>>>>> stable enough that we are willing to truly start maintaining backwards
>>>>> compatibility on them within 2.X. From what I have seen I think that
>>> they
>>>>> are fairly stable and I think there is enough adoption by other projects
>>>>> right now that breaking backwards compatibility would be problematic.
>>>>>
>>>>> --Bobby Evans
>>>>>
>>>>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]>
>>> wrote:
>>>>>>> Hi Arun,
>>>>>>>
>>>>>>> Given that the 2.0.3 release is intended to reflect the growing
>>>>>>> stability
>>>>>>> of YARN, and the QJM work will be included in 2.0.3 which provides a
>>>>>>> complete HDFS HA solution, I think it's time we consider removing the
>>>>>>> "-alpha" label from the release version. My preference would be to
>>>>>>> remove
>>>>>>> the label entirely, but we could also perhaps call it "-beta" or
>>>>>>> something.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>
>>>>>> I think it fine after two minor releases undoing the '-alpha' suffix.
>>>>>>
>>>>>> If folks insist we next go to '-beta', I'd hope we'd travel all
>>>>>> remaining 22 letters of the greek alphabet before we 2.0.x.
>>>>>>
>>>>>> St.Ack
>>>>>
>>>>>
>>>
>>
>>
>>
>> --
>> Have a Nice Day!
>> Lohit
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>

--
Todd Lipcon
Software Engineer, Cloudera

Thanks Todd!
On Dec 4, 2012, at 6:04 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> Hey Arun,
>
> I put up patches for the QJM backport merge yesterday. Aaron said he'd
> take a look at reviewing them, so I anticipate that to be finished
> "real soon now". Sorry for the delay.
>
> -Todd
>
> On Tue, Dec 4, 2012 at 6:09 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>> Lohit,
>>
>> There are some outstanding blockers and I'm still awaiting the QJM merge.
>>
>> Feel free to watch the blocker list:
>> http://s.apache.org/e1J
>>
>> Arun
>>
>> On Dec 3, 2012, at 10:02 AM, lohit wrote:
>>
>>> Hello Hadoop Release managers,
>>> Any update on this?
>>>
>>> Thanks,
>>> Lohit
>>>
>>> 2012/11/20 Tom White <[EMAIL PROTECTED]>
>>>
>>>> On Mon, Nov 19, 2012 at 6:09 PM, Siddharth Seth
>>>> <[EMAIL PROTECTED]> wrote:
>>>>> YARN-142/MAPREDUCE-4067 should ideally be fixed before we commit to API
>>>>> backward compatibility. Also, from the recent YARN meetup - there seemed
>>>> to
>>>>> be a requirement to change the AM-RM protocol for container requests. In
>>>>> this case, I believe it's OK to not have all functionality implemented,
>>>> as
>>>>> long as the protocol itself can represent the requirements.
>>>>
>>>> I agree. Do you think we can make these changes before removing the
>>>> 'alpha' label, i.e. in 2.0.3? If that's not possible for the container
>>>> requests change, then we could mark AMRMProtocol (or related classes)
>>>> as @Evolving. Another alternative would be to introduce a new
>>>> interface.
>>>>
>>>>> However, as
>>>>> Bobby pointed out, given the current adoption by other projects -
>>>>> incompatible changes at this point can be problematic and needs to be
>>>>> figured out.
>>>>
>>>> We have a mechanism for this already. If something is marked as
>>>> @Evolving it can change incompatibly between minor versions - e.g.
>>>> 2.0.x to 2.1.0. If it is @Stable then it can only change on major
>>>> versions, e.g. 2.x.y to 3.0.0. Let's make sure we are happy with the
>>>> annotations - and willing to support them at the indicated level -
>>>> before we remove the 'alpha' label. Of course, we strive not to change
>>>> APIs without a very good reason, but if we do we should do so within
>>>> the guidelines so that users know what to expect.
>>>>
>>>> Cheers,
>>>> Tom
>>>>
>>>>>
>>>>> Thanks
>>>>> - Sid
>>>>>
>>>>>
>>>>> On Mon, Nov 19, 2012 at 8:22 AM, Robert Evans <[EMAIL PROTECTED]>
>>>> wrote:
>>>>>
>>>>>> I am OK with removing the alpha assuming that we think that the APIs are
>>>>>> stable enough that we are willing to truly start maintaining backwards
>>>>>> compatibility on them within 2.X. From what I have seen I think that
>>>> they
>>>>>> are fairly stable and I think there is enough adoption by other projects
>>>>>> right now that breaking backwards compatibility would be problematic.
>>>>>>
>>>>>> --Bobby Evans
>>>>>>
>>>>>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>>>>>>
>>>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]>
>>>> wrote:
>>>>>>>> Hi Arun,
>>>>>>>>
>>>>>>>> Given that the 2.0.3 release is intended to reflect the growing
>>>>>>>> stability
>>>>>>>> of YARN, and the QJM work will be included in 2.0.3 which provides a
>>>>>>>> complete HDFS HA solution, I think it's time we consider removing the
>>>>>>>> "-alpha" label from the release version. My preference would be to
>>>>>>>> remove
>>>>>>>> the label entirely, but we could also perhaps call it "-beta" or
>>>>>>>> something.
>>>>>>>>
>>>>>>>> Thoughts?
>>>>>>>>
>>>>>>>
>>>>>>> I think it fine after two minor releases undoing the '-alpha' suffix.
>>>>>>>
>>>>>>> If folks insist we next go to '-beta', I'd hope we'd travel all
>>>>>>> remaining 22 letters of the greek alphabet before we 2.0.x.
>>>>>>>
>>>>>>> St.Ack
>>>>>>
>>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Have a Nice Day!
>>> Lohit
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera

OK, QJM is now in branch-2. I also merged all the follow-up patches I
could find so that branch-2 and trunk should be equivalent with regard
to QJM functionality at this point. If anyone sees anything I missed,
feel free to give me a holler.

Thanks
Todd

On Tue, Dec 4, 2012 at 6:56 PM, Arun Murthy <[EMAIL PROTECTED]> wrote:
> Thanks Todd!
>
>
> On Dec 4, 2012, at 6:04 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>
>> Hey Arun,
>>
>> I put up patches for the QJM backport merge yesterday. Aaron said he'd
>> take a look at reviewing them, so I anticipate that to be finished
>> "real soon now". Sorry for the delay.
>>
>> -Todd
>>
>> On Tue, Dec 4, 2012 at 6:09 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>>> Lohit,
>>>
>>> There are some outstanding blockers and I'm still awaiting the QJM merge.
>>>
>>> Feel free to watch the blocker list:
>>> http://s.apache.org/e1J
>>>
>>> Arun
>>>
>>> On Dec 3, 2012, at 10:02 AM, lohit wrote:
>>>
>>>> Hello Hadoop Release managers,
>>>> Any update on this?
>>>>
>>>> Thanks,
>>>> Lohit
>>>>
>>>> 2012/11/20 Tom White <[EMAIL PROTECTED]>
>>>>
>>>>> On Mon, Nov 19, 2012 at 6:09 PM, Siddharth Seth
>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>> YARN-142/MAPREDUCE-4067 should ideally be fixed before we commit to API
>>>>>> backward compatibility. Also, from the recent YARN meetup - there seemed
>>>>> to
>>>>>> be a requirement to change the AM-RM protocol for container requests. In
>>>>>> this case, I believe it's OK to not have all functionality implemented,
>>>>> as
>>>>>> long as the protocol itself can represent the requirements.
>>>>>
>>>>> I agree. Do you think we can make these changes before removing the
>>>>> 'alpha' label, i.e. in 2.0.3? If that's not possible for the container
>>>>> requests change, then we could mark AMRMProtocol (or related classes)
>>>>> as @Evolving. Another alternative would be to introduce a new
>>>>> interface.
>>>>>
>>>>>> However, as
>>>>>> Bobby pointed out, given the current adoption by other projects -
>>>>>> incompatible changes at this point can be problematic and needs to be
>>>>>> figured out.
>>>>>
>>>>> We have a mechanism for this already. If something is marked as
>>>>> @Evolving it can change incompatibly between minor versions - e.g.
>>>>> 2.0.x to 2.1.0. If it is @Stable then it can only change on major
>>>>> versions, e.g. 2.x.y to 3.0.0. Let's make sure we are happy with the
>>>>> annotations - and willing to support them at the indicated level -
>>>>> before we remove the 'alpha' label. Of course, we strive not to change
>>>>> APIs without a very good reason, but if we do we should do so within
>>>>> the guidelines so that users know what to expect.
>>>>>
>>>>> Cheers,
>>>>> Tom
>>>>>
>>>>>>
>>>>>> Thanks
>>>>>> - Sid
>>>>>>
>>>>>>
>>>>>> On Mon, Nov 19, 2012 at 8:22 AM, Robert Evans <[EMAIL PROTECTED]>
>>>>> wrote:
>>>>>>
>>>>>>> I am OK with removing the alpha assuming that we think that the APIs are
>>>>>>> stable enough that we are willing to truly start maintaining backwards
>>>>>>> compatibility on them within 2.X. From what I have seen I think that
>>>>> they
>>>>>>> are fairly stable and I think there is enough adoption by other projects
>>>>>>> right now that breaking backwards compatibility would be problematic.
>>>>>>>
>>>>>>> --Bobby Evans
>>>>>>>
>>>>>>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>>>>>>>
>>>>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]>
>>>>> wrote:
>>>>>>>>> Hi Arun,
>>>>>>>>>
>>>>>>>>> Given that the 2.0.3 release is intended to reflect the growing
>>>>>>>>> stability
>>>>>>>>> of YARN, and the QJM work will be included in 2.0.3 which provides a
>>>>>>>>> complete HDFS HA solution, I think it's time we consider removing the
>>>>>>>>> "-alpha" label from the release version. My preference would be to
>>>>>>>>> remove
>>>>>>>>> the label entirely, but we could also perhaps call it "-beta" or
>>>>>>>>> something.
>>>>>>>>>
>>>>>>>>> Thoughts?
>>>>>>>>>
>>>>>>>>
>>>>>>>> I think it fine after two minor releases undoing the '-alpha' suffix.

Todd Lipcon
Software Engineer, Cloudera

Hey Arun,

On Tue, Dec 4, 2012 at 6:09 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

>  Feel free to watch the blocker list:
>  http://s.apache.org/e1J
>

I notice that query doesn't include "2.0.3-alpha"  in the "affected
version" or "target version" fields. You should probably add it to get the
complete list of blockers for 2.0.3-alpha. For example, that query didn't
include HADOOP-9070 which I just stumbled across and is marked as a blocker
for 2.0.3-alpha.

--
Aaron T. Myers
Software Engineer, Cloudera

Any news on how this is progressing? Some folks in this thread below
inquired about getting this release out around the New Year timeframe,
but it looks like YARN-117 subtasks have gone pretty quiet. We all
know how long lifecycle changes can take to get pushed through ;-)

-Todd

On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
<[EMAIL PROTECTED]> wrote:
> I want to make some changes to the lifecycle of a yarn service (in a
> backwards compatible way).
>
> https://issues.apache.org/jira/browse/YARN-117
>
>
>    1. formal state machine model with stop state idempotent and entry-able
>    from any state
>    2. waiting/blocked state a service can enter when waiting for something
>    else
>    3. an alternate base class that does the state model checks before
>    executing any state change functions -currently its done at
>    end-of-operation in the super() calls.
>    4. gradual move of services to the stricter base class.
>
> With a new base class nothing will break (as the move can be done
> case-by-case, leaving the heavily subclassed ones alone); the state model
> extensions & formalisation would be visible but not used.
>
> I don't want to hold anything up, because I need more testing of things
> before this is ready for review. I just want to get the fixes in before it
> ships
>
> On 19 November 2012 16:22, Robert Evans <[EMAIL PROTECTED]> wrote:
>
>> I am OK with removing the alpha assuming that we think that the APIs are
>> stable enough that we are willing to truly start maintaining backwards
>> compatibility on them within 2.X. From what I have seen I think that they
>> are fairly stable and I think there is enough adoption by other projects
>> right now that breaking backwards compatibility would be problematic.
>>
>> --Bobby Evans
>>
>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>>
>> >On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
>> >> Hi Arun,
>> >>
>> >> Given that the 2.0.3 release is intended to reflect the growing
>> >>stability
>> >> of YARN, and the QJM work will be included in 2.0.3 which provides a
>> >> complete HDFS HA solution, I think it's time we consider removing the
>> >> "-alpha" label from the release version. My preference would be to
>> >>remove
>> >> the label entirely, but we could also perhaps call it "-beta" or
>> >>something.
>> >>
>> >> Thoughts?
>> >>
>> >
>> >I think it fine after two minor releases undoing the '-alpha' suffix.
>> >
>> >If folks insist we next go to '-beta', I'd hope we'd travel all
>> >remaining 22 letters of the greek alphabet before we 2.0.x.
>> >
>> >St.Ack
>>
>>

--
Todd Lipcon
Software Engineer, Cloudera

Nearly there:
http://s.apache.org/hadoop-2-blockers

YARN-217 should be easy, I'd also like to get in YARN-253.

Arun

On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:

> Any news on how this is progressing? Some folks in this thread below
> inquired about getting this release out around the New Year timeframe,
> but it looks like YARN-117 subtasks have gone pretty quiet. We all
> know how long lifecycle changes can take to get pushed through ;-)
>
> -Todd
>
> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
> <[EMAIL PROTECTED]> wrote:
>> I want to make some changes to the lifecycle of a yarn service (in a
>> backwards compatible way).
>>
>> https://issues.apache.org/jira/browse/YARN-117
>>
>>
>>   1. formal state machine model with stop state idempotent and entry-able
>>   from any state
>>   2. waiting/blocked state a service can enter when waiting for something
>>   else
>>   3. an alternate base class that does the state model checks before
>>   executing any state change functions -currently its done at
>>   end-of-operation in the super() calls.
>>   4. gradual move of services to the stricter base class.
>>
>> With a new base class nothing will break (as the move can be done
>> case-by-case, leaving the heavily subclassed ones alone); the state model
>> extensions & formalisation would be visible but not used.
>>
>> I don't want to hold anything up, because I need more testing of things
>> before this is ready for review. I just want to get the fixes in before it
>> ships
>>
>> On 19 November 2012 16:22, Robert Evans <[EMAIL PROTECTED]> wrote:
>>
>>> I am OK with removing the alpha assuming that we think that the APIs are
>>> stable enough that we are willing to truly start maintaining backwards
>>> compatibility on them within 2.X. From what I have seen I think that they
>>> are fairly stable and I think there is enough adoption by other projects
>>> right now that breaking backwards compatibility would be problematic.
>>>
>>> --Bobby Evans
>>>
>>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>>>
>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
>>>>> Hi Arun,
>>>>>
>>>>> Given that the 2.0.3 release is intended to reflect the growing
>>>>> stability
>>>>> of YARN, and the QJM work will be included in 2.0.3 which provides a
>>>>> complete HDFS HA solution, I think it's time we consider removing the
>>>>> "-alpha" label from the release version. My preference would be to
>>>>> remove
>>>>> the label entirely, but we could also perhaps call it "-beta" or
>>>>> something.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>
>>>> I think it fine after two minor releases undoing the '-alpha' suffix.
>>>>
>>>> If folks insist we next go to '-beta', I'd hope we'd travel all
>>>> remaining 22 letters of the greek alphabet before we 2.0.x.
>>>>
>>>> St.Ack
>>>
>>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

Hey Arun,

Awesome to see we're almost down to zero blockers. What are your thoughts
on removing the "alpha" label from the upcoming release? It seems to me
from the earlier discussion that most folks feel that we're at the point
where the interfaces are sufficiently stable to warrant it.

Aaron
On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> Nearly there:
> http://s.apache.org/hadoop-2-blockers
>
> YARN-217 should be easy, I'd also like to get in YARN-253.
>
> Arun
>
> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:
>
> > Any news on how this is progressing? Some folks in this thread below
> > inquired about getting this release out around the New Year timeframe,
> > but it looks like YARN-117 subtasks have gone pretty quiet. We all
> > know how long lifecycle changes can take to get pushed through ;-)
> >
> > -Todd
> >
> > On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
> > <[EMAIL PROTECTED]> wrote:
> >> I want to make some changes to the lifecycle of a yarn service (in a
> >> backwards compatible way).
> >>
> >> https://issues.apache.org/jira/browse/YARN-117
> >>
> >>
> >>   1. formal state machine model with stop state idempotent and
> entry-able
> >>   from any state
> >>   2. waiting/blocked state a service can enter when waiting for
> something
> >>   else
> >>   3. an alternate base class that does the state model checks before
> >>   executing any state change functions -currently its done at
> >>   end-of-operation in the super() calls.
> >>   4. gradual move of services to the stricter base class.
> >>
> >> With a new base class nothing will break (as the move can be done
> >> case-by-case, leaving the heavily subclassed ones alone); the state
> model
> >> extensions & formalisation would be visible but not used.
> >>
> >> I don't want to hold anything up, because I need more testing of things
> >> before this is ready for review. I just want to get the fixes in before
> it
> >> ships
> >>
> >> On 19 November 2012 16:22, Robert Evans <[EMAIL PROTECTED]> wrote:
> >>
> >>> I am OK with removing the alpha assuming that we think that the APIs
> are
> >>> stable enough that we are willing to truly start maintaining backwards
> >>> compatibility on them within 2.X. From what I have seen I think that
> they
> >>> are fairly stable and I think there is enough adoption by other
> projects
> >>> right now that breaking backwards compatibility would be problematic.
> >>>
> >>> --Bobby Evans
> >>>
> >>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
> >>>
> >>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]>
> wrote:
> >>>>> Hi Arun,
> >>>>>
> >>>>> Given that the 2.0.3 release is intended to reflect the growing
> >>>>> stability
> >>>>> of YARN, and the QJM work will be included in 2.0.3 which provides a
> >>>>> complete HDFS HA solution, I think it's time we consider removing the
> >>>>> "-alpha" label from the release version. My preference would be to
> >>>>> remove
> >>>>> the label entirely, but we could also perhaps call it "-beta" or
> >>>>> something.
> >>>>>
> >>>>> Thoughts?
> >>>>>
> >>>>
> >>>> I think it fine after two minor releases undoing the '-alpha' suffix.
> >>>>
> >>>> If folks insist we next go to '-beta', I'd hope we'd travel all
> >>>> remaining 22 letters of the greek alphabet before we 2.0.x.
> >>>>
> >>>> St.Ack
> >>>
> >>>
> >
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>

As Sid responded I think we can move off alpha once we fix YARN-142/MAPREDUCE-4067. There are other apis we should clean up, but none as egregious as those two.

Someone on my team is starting on it as we speak and I believe we can get it done sometime in Jan... thus targetting 2.0.4 (as a beta?). By then we'll also have wider rollouts of YARN and would have fixed some more issues we've seen at very high scale deployments at Y!. Sounds like the right time to do a beta release to me.

Arun

On Dec 19, 2012, at 10:13 AM, Aaron T. Myers wrote:

> Hey Arun,
>
> Awesome to see we're almost down to zero blockers. What are your thoughts
> on removing the "alpha" label from the upcoming release? It seems to me
> from the earlier discussion that most folks feel that we're at the point
> where the interfaces are sufficiently stable to warrant it.
>
> Aaron
>
>
> On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>
>> Nearly there:
>> http://s.apache.org/hadoop-2-blockers
>>
>> YARN-217 should be easy, I'd also like to get in YARN-253.
>>
>> Arun
>>
>> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:
>>
>>> Any news on how this is progressing? Some folks in this thread below
>>> inquired about getting this release out around the New Year timeframe,
>>> but it looks like YARN-117 subtasks have gone pretty quiet. We all
>>> know how long lifecycle changes can take to get pushed through ;-)
>>>
>>> -Todd
>>>
>>> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
>>> <[EMAIL PROTECTED]> wrote:
>>>> I want to make some changes to the lifecycle of a yarn service (in a
>>>> backwards compatible way).
>>>>
>>>> https://issues.apache.org/jira/browse/YARN-117
>>>>
>>>>
>>>>  1. formal state machine model with stop state idempotent and
>> entry-able
>>>>  from any state
>>>>  2. waiting/blocked state a service can enter when waiting for
>> something
>>>>  else
>>>>  3. an alternate base class that does the state model checks before
>>>>  executing any state change functions -currently its done at
>>>>  end-of-operation in the super() calls.
>>>>  4. gradual move of services to the stricter base class.
>>>>
>>>> With a new base class nothing will break (as the move can be done
>>>> case-by-case, leaving the heavily subclassed ones alone); the state
>> model
>>>> extensions & formalisation would be visible but not used.
>>>>
>>>> I don't want to hold anything up, because I need more testing of things
>>>> before this is ready for review. I just want to get the fixes in before
>> it
>>>> ships
>>>>
>>>> On 19 November 2012 16:22, Robert Evans <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> I am OK with removing the alpha assuming that we think that the APIs
>> are
>>>>> stable enough that we are willing to truly start maintaining backwards
>>>>> compatibility on them within 2.X. From what I have seen I think that
>> they
>>>>> are fairly stable and I think there is enough adoption by other
>> projects
>>>>> right now that breaking backwards compatibility would be problematic.
>>>>>
>>>>> --Bobby Evans
>>>>>
>>>>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]>
>> wrote:
>>>>>>> Hi Arun,
>>>>>>>
>>>>>>> Given that the 2.0.3 release is intended to reflect the growing
>>>>>>> stability
>>>>>>> of YARN, and the QJM work will be included in 2.0.3 which provides a
>>>>>>> complete HDFS HA solution, I think it's time we consider removing the
>>>>>>> "-alpha" label from the release version. My preference would be to
>>>>>>> remove
>>>>>>> the label entirely, but we could also perhaps call it "-beta" or
>>>>>>> something.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>
>>>>>> I think it fine after two minor releases undoing the '-alpha' suffix.
>>>>>>
>>>>>> If folks insist we next go to '-beta', I'd hope we'd travel all
>>>>>> remaining 22 letters of the greek alphabet before we 2.0.x.
>>>>>>
>>>>>> St.Ack
>>>>>
>>>>>
>>>
>>>
>>
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

Great, that makes sense to me as well.

Thanks a lot, and have a happy holidays.

Aaron
On Tue, Dec 18, 2012 at 9:00 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> As Sid responded I think we can move off alpha once we fix
> YARN-142/MAPREDUCE-4067. There are other apis we should clean up, but none
> as egregious as those two.
>
> Someone on my team is starting on it as we speak and I believe we can get
> it done sometime in Jan... thus targetting 2.0.4 (as a beta?). By then
> we'll also have wider rollouts of YARN and would have fixed some more
> issues we've seen at very high scale deployments at Y!. Sounds like the
> right time to do a beta release to me.
>
> Arun
>
> On Dec 19, 2012, at 10:13 AM, Aaron T. Myers wrote:
>
> > Hey Arun,
> >
> > Awesome to see we're almost down to zero blockers. What are your thoughts
> > on removing the "alpha" label from the upcoming release? It seems to me
> > from the earlier discussion that most folks feel that we're at the point
> > where the interfaces are sufficiently stable to warrant it.
> >
> > Aaron
> >
> >
> > On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <[EMAIL PROTECTED]>
> wrote:
> >
> >> Nearly there:
> >> http://s.apache.org/hadoop-2-blockers
> >>
> >> YARN-217 should be easy, I'd also like to get in YARN-253.
> >>
> >> Arun
> >>
> >> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:
> >>
> >>> Any news on how this is progressing? Some folks in this thread below
> >>> inquired about getting this release out around the New Year timeframe,
> >>> but it looks like YARN-117 subtasks have gone pretty quiet. We all
> >>> know how long lifecycle changes can take to get pushed through ;-)
> >>>
> >>> -Todd
> >>>
> >>> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
> >>> <[EMAIL PROTECTED]> wrote:
> >>>> I want to make some changes to the lifecycle of a yarn service (in a
> >>>> backwards compatible way).
> >>>>
> >>>> https://issues.apache.org/jira/browse/YARN-117
> >>>>
> >>>>
> >>>>  1. formal state machine model with stop state idempotent and
> >> entry-able
> >>>>  from any state
> >>>>  2. waiting/blocked state a service can enter when waiting for
> >> something
> >>>>  else
> >>>>  3. an alternate base class that does the state model checks before
> >>>>  executing any state change functions -currently its done at
> >>>>  end-of-operation in the super() calls.
> >>>>  4. gradual move of services to the stricter base class.
> >>>>
> >>>> With a new base class nothing will break (as the move can be done
> >>>> case-by-case, leaving the heavily subclassed ones alone); the state
> >> model
> >>>> extensions & formalisation would be visible but not used.
> >>>>
> >>>> I don't want to hold anything up, because I need more testing of
> things
> >>>> before this is ready for review. I just want to get the fixes in
> before
> >> it
> >>>> ships
> >>>>
> >>>> On 19 November 2012 16:22, Robert Evans <[EMAIL PROTECTED]> wrote:
> >>>>
> >>>>> I am OK with removing the alpha assuming that we think that the APIs
> >> are
> >>>>> stable enough that we are willing to truly start maintaining
> backwards
> >>>>> compatibility on them within 2.X. From what I have seen I think that
> >> they
> >>>>> are fairly stable and I think there is enough adoption by other
> >> projects
> >>>>> right now that breaking backwards compatibility would be problematic.
> >>>>>
> >>>>> --Bobby Evans
> >>>>>
> >>>>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
> >>>>>
> >>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]>
> >> wrote:
> >>>>>>> Hi Arun,
> >>>>>>>
> >>>>>>> Given that the 2.0.3 release is intended to reflect the growing
> >>>>>>> stability
> >>>>>>> of YARN, and the QJM work will be included in 2.0.3 which provides
> a
> >>>>>>> complete HDFS HA solution, I think it's time we consider removing
> the
> >>>>>>> "-alpha" label from the release version. My preference would be to
> >>>>>>> remove
> >>>>>>> the label entirely, but we could also perhaps call it "-beta" or

Hi
How to insert and update datin hadoop? Could you please explain me...
 
Thanks
Prabakara Krishnan
 
 
________________________________
From: Aaron T. Myers <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Wednesday, 19 December 2012 10:37 AM
Subject: Re: Heads Up - hadoop-2.0.3 release

Great, that makes sense to me as well.

Thanks a lot, and have a happy holidays.

Aaron
On Tue, Dec 18, 2012 at 9:00 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> As Sid responded I think we can move off alpha once we fix
> YARN-142/MAPREDUCE-4067. There are other apis we should clean up, but none
> as egregious as those two.
>
> Someone on my team is starting on it as we speak and I believe we can get
> it done sometime in Jan... thus targetting 2.0.4 (as a beta?). By then
> we'll also have wider rollouts of YARN and would have fixed some more
> issues we've seen at very high scale deployments at Y!. Sounds like the
> right time to do a beta release to me.
>
> Arun
>
> On Dec 19, 2012, at 10:13 AM, Aaron T. Myers wrote:
>
> > Hey Arun,
> >
> > Awesome to see we're almost down to zero blockers. What are your thoughts
> > on removing the "alpha" label from the upcoming release? It seems to me
> > from the earlier discussion that most folks feel that we're at the point
> > where the interfaces are sufficiently stable to warrant it.
> >
> > Aaron
> >
> >
> > On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <[EMAIL PROTECTED]>
> wrote:
> >
> >> Nearly there:
> >> http://s.apache.org/hadoop-2-blockers
> >>
> >> YARN-217 should be easy, I'd also like to get in YARN-253.
> >>
> >> Arun
> >>
> >> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:
> >>
> >>> Any news on how this is progressing? Some folks in this thread below
> >>> inquired about getting this release out around the New Year timeframe,
> >>> but it looks like YARN-117 subtasks have gone pretty quiet. We all
> >>> know how long lifecycle changes can take to get pushed through ;-)
> >>>
> >>> -Todd
> >>>
> >>> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
> >>> <[EMAIL PROTECTED]> wrote:
> >>>> I want to make some changes to the lifecycle of a yarn service (in a
> >>>> backwards compatible way).
> >>>>
> >>>> https://issues.apache.org/jira/browse/YARN-117
> >>>>
> >>>>
> >>>>  1. formal state machine model with stop state idempotent and
> >> entry-able
> >>>>  from any state
> >>>>  2. waiting/blocked state a service can enter when waiting for
> >> something
> >>>>  else
> >>>>  3. an alternate base class that does the state model checks before
> >>>>  executing any state change functions -currently its done at
> >>>>  end-of-operation in the super() calls.
> >>>>  4. gradual move of services to the stricter base class.
> >>>>
> >>>> With a new base class nothing will break (as the move can be done
> >>>> case-by-case, leaving the heavily subclassed ones alone); the state
> >> model
> >>>> extensions & formalisation would be visible but not used.
> >>>>
> >>>> I don't want to hold anything up, because I need more testing of
> things
> >>>> before this is ready for review. I just want to get the fixes in
> before
> >> it
> >>>> ships
> >>>>
> >>>> On 19 November 2012 16:22, Robert Evans <[EMAIL PROTECTED]> wrote:
> >>>>
> >>>>> I am OK with removing the alpha assuming that we think that the APIs
> >> are
> >>>>> stable enough that we are willing to truly start maintaining
> backwards
> >>>>> compatibility on them within 2.X. From what I have seen I think that
> >> they
> >>>>> are fairly stable and I think there is enough adoption by other
> >> projects
> >>>>> right now that breaking backwards compatibility would be problematic.
> >>>>>
> >>>>> --Bobby Evans
> >>>>>
> >>>>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
> >>>>>
> >>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]>
> >> wrote:
> >>>>>>> Hi Arun,
> >>>>>>>
> >>>>>>> Given that the 2.0.3 release is intended to reflect the growing
> >>>>>>> stability
> >>>>>>> of YARN, and the QJM work will be included in 2.0.3 which provides

On 19 December 2012 11:26, Prabakaran Krishnan <[EMAIL PROTECTED]>wrote:

> Hi
> How to insert and update datin hadoop? Could you please explain me...
>
> Thanks
> Prabakara Krishnan
>
>
1. please don't steal ongoing threads for asking questions
2. this is the kind of question to ask on the user@hadoop list. Please
subscribe to that and ask your question there.
3. Hadoop HDFS is a fileystem without seek, only append. Database layers on
top, e.g. HBase, do offer such services.

I have not a clue how I got on this thread...It is all very interesting, but please take me off.

Heather Bales ☺
CRM Manager - Digital Marketing
NCSOFT
(W)206-588-7268  (C)206-790-2665
[EMAIL PROTECTED] | http://us.ncsoft.com
1501 Fourth Ave. 20th Floor Seattle, WA 98101
-----Original Message-----
From: Aaron T. Myers [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, December 18, 2012 9:07 PM
To: [EMAIL PROTECTED]
Subject: Re: Heads Up - hadoop-2.0.3 release

Great, that makes sense to me as well.

Thanks a lot, and have a happy holidays.

Aaron
On Tue, Dec 18, 2012 at 9:00 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> As Sid responded I think we can move off alpha once we fix
> YARN-142/MAPREDUCE-4067. There are other apis we should clean up, but
> none as egregious as those two.
>
> Someone on my team is starting on it as we speak and I believe we can
> get it done sometime in Jan... thus targetting 2.0.4 (as a beta?). By
> then we'll also have wider rollouts of YARN and would have fixed some
> more issues we've seen at very high scale deployments at Y!. Sounds
> like the right time to do a beta release to me.
>
> Arun
>
> On Dec 19, 2012, at 10:13 AM, Aaron T. Myers wrote:
>
> > Hey Arun,
> >
> > Awesome to see we're almost down to zero blockers. What are your
> > thoughts on removing the "alpha" label from the upcoming release? It
> > seems to me from the earlier discussion that most folks feel that
> > we're at the point where the interfaces are sufficiently stable to warrant it.
> >
> > Aaron
> >
> >
> > On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <[EMAIL PROTECTED]>
> wrote:
> >
> >> Nearly there:
> >> http://s.apache.org/hadoop-2-blockers
> >>
> >> YARN-217 should be easy, I'd also like to get in YARN-253.
> >>
> >> Arun
> >>
> >> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:
> >>
> >>> Any news on how this is progressing? Some folks in this thread
> >>> below inquired about getting this release out around the New Year
> >>> timeframe, but it looks like YARN-117 subtasks have gone pretty
> >>> quiet. We all know how long lifecycle changes can take to get
> >>> pushed through ;-)
> >>>
> >>> -Todd
> >>>
> >>> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
> >>> <[EMAIL PROTECTED]> wrote:
> >>>> I want to make some changes to the lifecycle of a yarn service
> >>>> (in a backwards compatible way).
> >>>>
> >>>> https://issues.apache.org/jira/browse/YARN-117
> >>>>
> >>>>
> >>>>  1. formal state machine model with stop state idempotent and
> >> entry-able
> >>>>  from any state
> >>>>  2. waiting/blocked state a service can enter when waiting for
> >> something
> >>>>  else
> >>>>  3. an alternate base class that does the state model checks
> >>>> before  executing any state change functions -currently its done
> >>>> at  end-of-operation in the super() calls.
> >>>>  4. gradual move of services to the stricter base class.
> >>>>
> >>>> With a new base class nothing will break (as the move can be done
> >>>> case-by-case, leaving the heavily subclassed ones alone); the
> >>>> state
> >> model
> >>>> extensions & formalisation would be visible but not used.
> >>>>
> >>>> I don't want to hold anything up, because I need more testing of
> things
> >>>> before this is ready for review. I just want to get the fixes in
> before
> >> it
> >>>> ships
> >>>>
> >>>> On 19 November 2012 16:22, Robert Evans <[EMAIL PROTECTED]> wrote:
> >>>>
> >>>>> I am OK with removing the alpha assuming that we think that the
> >>>>> APIs
> >> are
> >>>>> stable enough that we are willing to truly start maintaining
> backwards
> >>>>> compatibility on them within 2.X. From what I have seen I think
> >>>>> that
> >> they
> >>>>> are fairly stable and I think there is enough adoption by other
> >> projects
> >>>>> right now that breaking backwards compatibility would be problematic.
> >>>>>
> >>>>> --Bobby Evans
> >>>>>
> >>>>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
> >>>>>
> >>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers

Happy new year folks!

I'm glad to see that we are down to the last couple of blockers I hope we can resolve in the next 24-hrs or so.

Once done, I'll create a branch-2.0.3-alpha to unblock branch-2 for commits targeted towards the next release, create the new fix-versions in jira and spin the RC.
Committers - after the branch is created, please use only the new fix-version (2.0.4) and check with me before you commit to branch-2.0.3-alpha. Thanks.

As always, I'd appreciate help to resolve any unexpected surprises and also to help verify the RC.

Hopefully we can start the new year with a great release, there are lots of goodies in 2.0.3-alpha.

thanks,
Arun

On Dec 18, 2012, at 9:00 PM, Arun C Murthy wrote:

> As Sid responded I think we can move off alpha once we fix YARN-142/MAPREDUCE-4067. There are other apis we should clean up, but none as egregious as those two.
>
> Someone on my team is starting on it as we speak and I believe we can get it done sometime in Jan... thus targetting 2.0.4 (as a beta?). By then we'll also have wider rollouts of YARN and would have fixed some more issues we've seen at very high scale deployments at Y!. Sounds like the right time to do a beta release to me.
>
> Arun
>
> On Dec 19, 2012, at 10:13 AM, Aaron T. Myers wrote:
>
>> Hey Arun,
>>
>> Awesome to see we're almost down to zero blockers. What are your thoughts
>> on removing the "alpha" label from the upcoming release? It seems to me
>> from the earlier discussion that most folks feel that we're at the point
>> where the interfaces are sufficiently stable to warrant it.
>>
>> Aaron
>>
>>
>> On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>>
>>> Nearly there:
>>> http://s.apache.org/hadoop-2-blockers
>>>
>>> YARN-217 should be easy, I'd also like to get in YARN-253.
>>>
>>> Arun
>>>
>>> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:
>>>
>>>> Any news on how this is progressing? Some folks in this thread below
>>>> inquired about getting this release out around the New Year timeframe,
>>>> but it looks like YARN-117 subtasks have gone pretty quiet. We all
>>>> know how long lifecycle changes can take to get pushed through ;-)
>>>>
>>>> -Todd
>>>>
>>>> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
>>>> <[EMAIL PROTECTED]> wrote:
>>>>> I want to make some changes to the lifecycle of a yarn service (in a
>>>>> backwards compatible way).
>>>>>
>>>>> https://issues.apache.org/jira/browse/YARN-117
>>>>>
>>>>>
>>>>>  1. formal state machine model with stop state idempotent and
>>> entry-able
>>>>>  from any state
>>>>>  2. waiting/blocked state a service can enter when waiting for
>>> something
>>>>>  else
>>>>>  3. an alternate base class that does the state model checks before
>>>>>  executing any state change functions -currently its done at
>>>>>  end-of-operation in the super() calls.
>>>>>  4. gradual move of services to the stricter base class.
>>>>>
>>>>> With a new base class nothing will break (as the move can be done
>>>>> case-by-case, leaving the heavily subclassed ones alone); the state
>>> model
>>>>> extensions & formalisation would be visible but not used.
>>>>>
>>>>> I don't want to hold anything up, because I need more testing of things
>>>>> before this is ready for review. I just want to get the fixes in before
>>> it
>>>>> ships
>>>>>
>>>>> On 19 November 2012 16:22, Robert Evans <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> I am OK with removing the alpha assuming that we think that the APIs
>>> are
>>>>>> stable enough that we are willing to truly start maintaining backwards
>>>>>> compatibility on them within 2.X. From what I have seen I think that
>>> they
>>>>>> are fairly stable and I think there is enough adoption by other
>>> projects
>>>>>> right now that breaking backwards compatibility would be problematic.
>>>>>>
>>>>>> --Bobby Evans
>>>>>>
>>>>>> On 11/16/12 11:34 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>>>>>>
>>>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <[EMAIL PROTECTED]>

Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

Arun,

HDFS-4404 is not marked as a blocker. It is a serious bug and I would like
to make it a blocker. Let me know if you are okay.

If folks do not want to hold the release for that fix, lets at least try to
capture that in release notes and try get that fix into 2.0.4.

Regards,
Suresh
On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> Happy new year folks!
>
> I'm glad to see that we are down to the last couple of blockers I hope we
> can resolve in the next 24-hrs or so.
>
> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2 for
> commits targeted towards the next release, create the new fix-versions in
> jira and spin the RC.
> Committers - after the branch is created, please use only the new
> fix-version (2.0.4) and check with me before you commit to
> branch-2.0.3-alpha. Thanks.
>
> As always, I'd appreciate help to resolve any unexpected surprises and
> also to help verify the RC.
>
> Hopefully we can start the new year with a great release, there are lots
> of goodies in 2.0.3-alpha.
>
> thanks,
> Arun
>
> On Dec 18, 2012, at 9:00 PM, Arun C Murthy wrote:
>
> > As Sid responded I think we can move off alpha once we fix
> YARN-142/MAPREDUCE-4067. There are other apis we should clean up, but none
> as egregious as those two.
> >
> > Someone on my team is starting on it as we speak and I believe we can
> get it done sometime in Jan... thus targetting 2.0.4 (as a beta?). By then
> we'll also have wider rollouts of YARN and would have fixed some more
> issues we've seen at very high scale deployments at Y!. Sounds like the
> right time to do a beta release to me.
> >
> > Arun
> >
> > On Dec 19, 2012, at 10:13 AM, Aaron T. Myers wrote:
> >
> >> Hey Arun,
> >>
> >> Awesome to see we're almost down to zero blockers. What are your
> thoughts
> >> on removing the "alpha" label from the upcoming release? It seems to me
> >> from the earlier discussion that most folks feel that we're at the point
> >> where the interfaces are sufficiently stable to warrant it.
> >>
> >> Aaron
> >>
> >>
> >> On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <[EMAIL PROTECTED]>
> wrote:
> >>
> >>> Nearly there:
> >>> http://s.apache.org/hadoop-2-blockers
> >>>
> >>> YARN-217 should be easy, I'd also like to get in YARN-253.
> >>>
> >>> Arun
> >>>
> >>> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:
> >>>
> >>>> Any news on how this is progressing? Some folks in this thread below
> >>>> inquired about getting this release out around the New Year timeframe,
> >>>> but it looks like YARN-117 subtasks have gone pretty quiet. We all
> >>>> know how long lifecycle changes can take to get pushed through ;-)
> >>>>
> >>>> -Todd
> >>>>
> >>>> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
> >>>> <[EMAIL PROTECTED]> wrote:
> >>>>> I want to make some changes to the lifecycle of a yarn service (in a
> >>>>> backwards compatible way).
> >>>>>
> >>>>> https://issues.apache.org/jira/browse/YARN-117
> >>>>>
> >>>>>
> >>>>>  1. formal state machine model with stop state idempotent and
> >>> entry-able
> >>>>>  from any state
> >>>>>  2. waiting/blocked state a service can enter when waiting for
> >>> something
> >>>>>  else
> >>>>>  3. an alternate base class that does the state model checks before
> >>>>>  executing any state change functions -currently its done at
> >>>>>  end-of-operation in the super() calls.
> >>>>>  4. gradual move of services to the stricter base class.
> >>>>>
> >>>>> With a new base class nothing will break (as the move can be done
> >>>>> case-by-case, leaving the heavily subclassed ones alone); the state
> >>> model
> >>>>> extensions & formalisation would be visible but not used.
> >>>>>
> >>>>> I don't want to hold anything up, because I need more testing of
> things
> >>>>> before this is ready for review. I just want to get the fixes in
> before
> >>> it
> >>>>> ships
> >>>>>
> >>>>> On 19 November 2012 16:22, Robert Evans <[EMAIL PROTECTED]> wrote:
> >>>>>
> >>>>>> I am OK with removing the alpha assuming that we think that the APIs
http://hortonworks.com/download/

Seems reasonable since it's a small patch and has had one round of review.

I spoke to Todd yesterday and he seemed to think it can be pushed in fairly soon too. So that's two of you.

Anyway, since I'm still awaiting HADOOP-9215 too, I'd appreciate if we can get HDFS-4404 committed asap.

thanks,
Arun

On Jan 16, 2013, at 10:30 AM, Suresh Srinivas wrote:

> Arun,
>
> HDFS-4404 is not marked as a blocker. It is a serious bug and I would like
> to make it a blocker. Let me know if you are okay.
>
> If folks do not want to hold the release for that fix, lets at least try to
> capture that in release notes and try get that fix into 2.0.4.
>
> Regards,
> Suresh
>
>
> On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>
>> Happy new year folks!
>>
>> I'm glad to see that we are down to the last couple of blockers I hope we
>> can resolve in the next 24-hrs or so.
>>
>> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2 for
>> commits targeted towards the next release, create the new fix-versions in
>> jira and spin the RC.
>> Committers - after the branch is created, please use only the new
>> fix-version (2.0.4) and check with me before you commit to
>> branch-2.0.3-alpha. Thanks.
>>
>> As always, I'd appreciate help to resolve any unexpected surprises and
>> also to help verify the RC.
>>
>> Hopefully we can start the new year with a great release, there are lots
>> of goodies in 2.0.3-alpha.
>>
>> thanks,
>> Arun
>>
>> On Dec 18, 2012, at 9:00 PM, Arun C Murthy wrote:
>>
>>> As Sid responded I think we can move off alpha once we fix
>> YARN-142/MAPREDUCE-4067. There are other apis we should clean up, but none
>> as egregious as those two.
>>>
>>> Someone on my team is starting on it as we speak and I believe we can
>> get it done sometime in Jan... thus targetting 2.0.4 (as a beta?). By then
>> we'll also have wider rollouts of YARN and would have fixed some more
>> issues we've seen at very high scale deployments at Y!. Sounds like the
>> right time to do a beta release to me.
>>>
>>> Arun
>>>
>>> On Dec 19, 2012, at 10:13 AM, Aaron T. Myers wrote:
>>>
>>>> Hey Arun,
>>>>
>>>> Awesome to see we're almost down to zero blockers. What are your
>> thoughts
>>>> on removing the "alpha" label from the upcoming release? It seems to me
>>>> from the earlier discussion that most folks feel that we're at the point
>>>> where the interfaces are sufficiently stable to warrant it.
>>>>
>>>> Aaron
>>>>
>>>>
>>>> On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <[EMAIL PROTECTED]>
>> wrote:
>>>>
>>>>> Nearly there:
>>>>> http://s.apache.org/hadoop-2-blockers
>>>>>
>>>>> YARN-217 should be easy, I'd also like to get in YARN-253.
>>>>>
>>>>> Arun
>>>>>
>>>>> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:
>>>>>
>>>>>> Any news on how this is progressing? Some folks in this thread below
>>>>>> inquired about getting this release out around the New Year timeframe,
>>>>>> but it looks like YARN-117 subtasks have gone pretty quiet. We all
>>>>>> know how long lifecycle changes can take to get pushed through ;-)
>>>>>>
>>>>>> -Todd
>>>>>>
>>>>>> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
>>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>>> I want to make some changes to the lifecycle of a yarn service (in a
>>>>>>> backwards compatible way).
>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/YARN-117
>>>>>>>
>>>>>>>
>>>>>>> 1. formal state machine model with stop state idempotent and
>>>>> entry-able
>>>>>>> from any state
>>>>>>> 2. waiting/blocked state a service can enter when waiting for
>>>>> something
>>>>>>> else
>>>>>>> 3. an alternate base class that does the state model checks before
>>>>>>> executing any state change functions -currently its done at
>>>>>>> end-of-operation in the super() calls.
>>>>>>> 4. gradual move of services to the stricter base class.
>>>>>>>
>>>>>>> With a new base class nothing will break (as the move can be done
>>>>>>> case-by-case, leaving the heavily subclassed ones alone); the state

Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

I'd like to get in 2.0.3 the JIRAs that enable pluggable shuffle and
pluggable sort: MAPREDUCE-4049, MAPREDUCE-4809, MAPREDUCE-4807 &
MAPREDUCE-4808

Unless I hear otherwise, I'd be merging those commits in branch-2 tomorrow
mid afternoon PST.

Thx
On Wed, Jan 16, 2013 at 10:38 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> Seems reasonable since it's a small patch and has had one round of review.
>
> I spoke to Todd yesterday and he seemed to think it can be pushed in
> fairly soon too. So that's two of you.
>
> Anyway, since I'm still awaiting HADOOP-9215 too, I'd appreciate if we can
> get HDFS-4404 committed asap.
>
> thanks,
> Arun
>
> On Jan 16, 2013, at 10:30 AM, Suresh Srinivas wrote:
>
> > Arun,
> >
> > HDFS-4404 is not marked as a blocker. It is a serious bug and I would
> like
> > to make it a blocker. Let me know if you are okay.
> >
> > If folks do not want to hold the release for that fix, lets at least try
> to
> > capture that in release notes and try get that fix into 2.0.4.
> >
> > Regards,
> > Suresh
> >
> >
> > On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <[EMAIL PROTECTED]>
> wrote:
> >
> >> Happy new year folks!
> >>
> >> I'm glad to see that we are down to the last couple of blockers I hope
> we
> >> can resolve in the next 24-hrs or so.
> >>
> >> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2 for
> >> commits targeted towards the next release, create the new fix-versions
> in
> >> jira and spin the RC.
> >> Committers - after the branch is created, please use only the new
> >> fix-version (2.0.4) and check with me before you commit to
> >> branch-2.0.3-alpha. Thanks.
> >>
> >> As always, I'd appreciate help to resolve any unexpected surprises and
> >> also to help verify the RC.
> >>
> >> Hopefully we can start the new year with a great release, there are lots
> >> of goodies in 2.0.3-alpha.
> >>
> >> thanks,
> >> Arun
> >>
> >> On Dec 18, 2012, at 9:00 PM, Arun C Murthy wrote:
> >>
> >>> As Sid responded I think we can move off alpha once we fix
> >> YARN-142/MAPREDUCE-4067. There are other apis we should clean up, but
> none
> >> as egregious as those two.
> >>>
> >>> Someone on my team is starting on it as we speak and I believe we can
> >> get it done sometime in Jan... thus targetting 2.0.4 (as a beta?). By
> then
> >> we'll also have wider rollouts of YARN and would have fixed some more
> >> issues we've seen at very high scale deployments at Y!. Sounds like the
> >> right time to do a beta release to me.
> >>>
> >>> Arun
> >>>
> >>> On Dec 19, 2012, at 10:13 AM, Aaron T. Myers wrote:
> >>>
> >>>> Hey Arun,
> >>>>
> >>>> Awesome to see we're almost down to zero blockers. What are your
> >> thoughts
> >>>> on removing the "alpha" label from the upcoming release? It seems to
> me
> >>>> from the earlier discussion that most folks feel that we're at the
> point
> >>>> where the interfaces are sufficiently stable to warrant it.
> >>>>
> >>>> Aaron
> >>>>
> >>>>
> >>>> On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <[EMAIL PROTECTED]>
> >> wrote:
> >>>>
> >>>>> Nearly there:
> >>>>> http://s.apache.org/hadoop-2-blockers
> >>>>>
> >>>>> YARN-217 should be easy, I'd also like to get in YARN-253.
> >>>>>
> >>>>> Arun
> >>>>>
> >>>>> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:
> >>>>>
> >>>>>> Any news on how this is progressing? Some folks in this thread below
> >>>>>> inquired about getting this release out around the New Year
> timeframe,
> >>>>>> but it looks like YARN-117 subtasks have gone pretty quiet. We all
> >>>>>> know how long lifecycle changes can take to get pushed through ;-)
> >>>>>>
> >>>>>> -Todd
> >>>>>>
> >>>>>> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
> >>>>>> <[EMAIL PROTECTED]> wrote:
> >>>>>>> I want to make some changes to the lifecycle of a yarn service (in
> a
> >>>>>>> backwards compatible way).
> >>>>>>>
> >>>>>>> https://issues.apache.org/jira/browse/YARN-117
> >>>>>>>
> >>>>>>>
> >>>>>>> 1. formal state machine model with stop state idempotent and
Alejandro

Hi Arun,

On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> Happy new year folks!
>
> I'm glad to see that we are down to the last couple of blockers I hope we can resolve in the next 24-hrs or so.
>
> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2
> for commits targeted towards the next release, create the new fix-versions in jira and spin the RC.

Quick question -- when do you think you can have the
branch-2.0.3-alpha in place?

We'd *really* like to start testing it in Bigtop ASAP (Bigtop 0.6.0 is
supposed to be
based on Hadoop 2.0.3) so we can avoid integration issues like we had between
2.0.2 and Oozie.

Any help in making it available to us sooner would be greatly appreciated!

Thanks,
Roman.

On Mon, Jan 28, 2013 at 3:11 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> Hi Arun,
>
> On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>> Happy new year folks!
>>
>> I'm glad to see that we are down to the last couple of blockers I hope we can resolve in the next 24-hrs or so.
>>
>> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2
>> for commits targeted towards the next release, create the new fix-versions in jira and spin the RC.
>
> Quick question -- when do you think you can have the
> branch-2.0.3-alpha in place?
>
> We'd *really* like to start testing it in Bigtop ASAP (Bigtop 0.6.0 is
> supposed to be
> based on Hadoop 2.0.3) so we can avoid integration issues like we had between
> 2.0.2 and Oozie.
>
> Any help in making it available to us sooner would be greatly appreciated!

On top of that I'd like to nominate:
   https://issues.apache.org/jira/browse/MAPREDUCE-4820
as a blocker. For as long as it is not fixed Oozie is unusable with
Hadoop 2.0.[23].

Thanks,
Roman.

+1 - that'd be critical.

On Tue, Jan 29, 2013 at 08:51AM, Roman Shaposhnik wrote:
> On Mon, Jan 28, 2013 at 3:11 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> > Hi Arun,
> >
> > On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> >> Happy new year folks!
> >>
> >> I'm glad to see that we are down to the last couple of blockers I hope we can resolve in the next 24-hrs or so.
> >>
> >> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2
> >> for commits targeted towards the next release, create the new fix-versions in jira and spin the RC.
> >
> > Quick question -- when do you think you can have the
> > branch-2.0.3-alpha in place?
> >
> > We'd *really* like to start testing it in Bigtop ASAP (Bigtop 0.6.0 is
> > supposed to be
> > based on Hadoop 2.0.3) so we can avoid integration issues like we had between
> > 2.0.2 and Oozie.
> >
> > Any help in making it available to us sooner would be greatly appreciated!
>
> On top of that I'd like to nominate:
>    https://issues.apache.org/jira/browse/MAPREDUCE-4820
> as a blocker. For as long as it is not fixed Oozie is unusable with
> Hadoop 2.0.[23].
>
> Thanks,
> Roman.

I believe we can disregard MAPREDUCE-4820 since MAPREDUCE-4549 is already in branch-2. FYI.

On Jan 29, 2013, at 8:51 AM, Roman Shaposhnik wrote:

> On Mon, Jan 28, 2013 at 3:11 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
>> Hi Arun,
>>
>> On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>>> Happy new year folks!
>>>
>>> I'm glad to see that we are down to the last couple of blockers I hope we can resolve in the next 24-hrs or so.
>>>
>>> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2
>>> for commits targeted towards the next release, create the new fix-versions in jira and spin the RC.
>>
>> Quick question -- when do you think you can have the
>> branch-2.0.3-alpha in place?
>>
>> We'd *really* like to start testing it in Bigtop ASAP (Bigtop 0.6.0 is
>> supposed to be
>> based on Hadoop 2.0.3) so we can avoid integration issues like we had between
>> 2.0.2 and Oozie.
>>
>> Any help in making it available to us sooner would be greatly appreciated!
>
> On top of that I'd like to nominate:
>   https://issues.apache.org/jira/browse/MAPREDUCE-4820
> as a blocker. For as long as it is not fixed Oozie is unusable with
> Hadoop 2.0.[23].
>
> Thanks,
> Roman.

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

On Tue, Jan 29, 2013 at 9:51 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> I believe we can disregard MAPREDUCE-4820 since MAPREDUCE-4549 is already in branch-2. FYI.

Arun, that very well may be the case -- let me take a closer look and
report back (on the JIRA).

Also, any chance you could comment on when the branch can be cut?
It would make verification of things MAPREDUCE-4820 so much easier.

Thanks,
Roman.

On Mon, Jan 28, 2013 at 03:11PM, Roman Shaposhnik wrote:
> Hi Arun,
>
> On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> > Happy new year folks!
> >
> > I'm glad to see that we are down to the last couple of blockers I hope we can resolve in the next 24-hrs or so.
> >
> > Once done, I'll create a branch-2.0.3-alpha to unblock branch-2
> > for commits targeted towards the next release, create the new fix-versions in jira and spin the RC.
>
> Quick question -- when do you think you can have the
> branch-2.0.3-alpha in place?
>
> We'd *really* like to start testing it in Bigtop ASAP (Bigtop 0.6.0 is
> supposed to be
> based on Hadoop 2.0.3) so we can avoid integration issues like we had between
> 2.0.2 and Oozie.

Oozie is hardcoded to be build against 2.0.2-alpha, so there would be
integration issues nonetheless (see BIGTOP-837)

>
> Any help in making it available to us sooner would be greatly appreciated!
>
> Thanks,
> Roman.