Quick update on Hadoop 0.22.  Candidate build of 0.22 is still blocked by a couple key issues:
* https://issues.apache.org/jira/browse/MAPREDUCE-2178
* MapReduce build problem in our CI environment
* HDFS & MapReduce test failures in our CI environment

A couple people have expressed to me their interest in helping to stabilize 0.22.  Any status updates on the above issues?

Alternatively, we could drop support for security in 0.22 and not fix MAPREDUCE-2178.  Thoughts?

Thanks,
Nige

Having talked to various folks in the community about their willingness to
stabilize and use 0.22 and make it production-quality, here is what I
propose:

1. Cut a release 0.22.0 without mapreduce-2178 patch, with
hadoop.security.authentication set to simple (I.e. No authentication).
Make sure that MR-2178 is highlighted as known-issue in the top-level dir
(create a KNOWN-ISSUES.txt ?), and carry out a vote.

2. Later, when MR-2178 patch is committed, cut a release for 0.22.1. (I
have heard of folks working on it, but no date commitments yet.)

The reason a 0.22.0 release should happen, IMHO, even without MR-2178, is
that the other components of Hadoop stack (e.g. Pig, hbase, hive etc) will
have a release to test against, and fix any issues for incompatibility.
Also, for folks who do not use kerberos authentication (>90% of the hadoop
users, to my knowledge), it will be a release closest to trunk, with HDFS
appends. Plus, any issues found in the HDFS part of the release would be
directly applicable to the trunk and other future releases. (Not so about
map-reduce, which will undergo major surgery in trunk because of MR-279.)

Thoughts ?

- milind

---
Milind Bhandarkar
Chief Architect, Greenplum Labs, Data Computing Division, EMC
+1-650-523-3858 (W)
+1-408-666-8483 (M)
On 8/1/11 11:33 PM, "Nigel Daley" <[EMAIL PROTECTED]> wrote:

>Quick update on Hadoop 0.22.  Candidate build of 0.22 is still blocked by
>a couple key issues:
>* https://issues.apache.org/jira/browse/MAPREDUCE-2178
>* MapReduce build problem in our CI environment
>* HDFS & MapReduce test failures in our CI environment
>
>A couple people have expressed to me their interest in helping to
>stabilize 0.22.  Any status updates on the above issues?
>
>Alternatively, we could drop support for security in 0.22 and not fix
>MAPREDUCE-2178.  Thoughts?
>
>Thanks,
>Nige
>

On Tue, Aug 2, 2011 at 12:19 PM,  <[EMAIL PROTECTED]> wrote:
> Having talked to various folks in the community about their willingness to
> stabilize and use 0.22 and make it production-quality, here is what I
> propose:
>
> 1. Cut a release 0.22.0 without mapreduce-2178 patch, with
> hadoop.security.authentication set to simple (I.e. No authentication).
> Make sure that MR-2178 is highlighted as known-issue in the top-level dir
> (create a KNOWN-ISSUES.txt ?), and carry out a vote.
>
> 2. Later, when MR-2178 patch is committed, cut a release for 0.22.1. (I
> have heard of folks working on it, but no date commitments yet.)
>
> The reason a 0.22.0 release should happen, IMHO, even without MR-2178, is
> that the other components of Hadoop stack (e.g. Pig, hbase, hive etc) will
> have a release to test against, and fix any issues for incompatibility.
> Also, for folks who do not use kerberos authentication (>90% of the hadoop
> users, to my knowledge), it will be a release closest to trunk, with HDFS
> appends. Plus, any issues found in the HDFS part of the release would be
> directly applicable to the trunk and other future releases. (Not so about
> map-reduce, which will undergo major surgery in trunk because of MR-279.)
>
> Thoughts ?
>

Sounds reasonable to me.  #1 is what we did for the 0.21 release.

Thanks,
Eli

On Tue, Aug 2, 2011 at 12:25 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
> On Tue, Aug 2, 2011 at 12:19 PM,  <[EMAIL PROTECTED]> wrote:
>> Having talked to various folks in the community about their willingness to
>> stabilize and use 0.22 and make it production-quality, here is what I
>> propose:
>>
>> 1. Cut a release 0.22.0 without mapreduce-2178 patch, with
>> hadoop.security.authentication set to simple (I.e. No authentication).
>> Make sure that MR-2178 is highlighted as known-issue in the top-level dir
>> (create a KNOWN-ISSUES.txt ?), and carry out a vote.
>>
>> 2. Later, when MR-2178 patch is committed, cut a release for 0.22.1. (I
>> have heard of folks working on it, but no date commitments yet.)
>>
>> The reason a 0.22.0 release should happen, IMHO, even without MR-2178, is
>> that the other components of Hadoop stack (e.g. Pig, hbase, hive etc) will
>> have a release to test against, and fix any issues for incompatibility.
>> Also, for folks who do not use kerberos authentication (>90% of the hadoop
>> users, to my knowledge), it will be a release closest to trunk, with HDFS
>> appends. Plus, any issues found in the HDFS part of the release would be
>> directly applicable to the trunk and other future releases. (Not so about
>> map-reduce, which will undergo major surgery in trunk because of MR-279.)
>>
>> Thoughts ?
>>
>
> Sounds reasonable to me.  #1 is what we did for the 0.21 release.

Actually, I didn't read close enough. For 0.21 we actually removed the
task tracker, think it makes sense to do that here as well.

Thanks,
Eli

+1 to removing the TaskController code for the 0.22 release.

In the MR-279 branch, the security issues identified in MAPREDUCE-2178 have been taken care of (to my knowledge).

On Aug 2, 2011, at 12:58 PM, Eli Collins wrote:

> On Tue, Aug 2, 2011 at 12:25 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
>> On Tue, Aug 2, 2011 at 12:19 PM,  <[EMAIL PROTECTED]> wrote:
>>> Having talked to various folks in the community about their willingness to
>>> stabilize and use 0.22 and make it production-quality, here is what I
>>> propose:
>>>
>>> 1. Cut a release 0.22.0 without mapreduce-2178 patch, with
>>> hadoop.security.authentication set to simple (I.e. No authentication).
>>> Make sure that MR-2178 is highlighted as known-issue in the top-level dir
>>> (create a KNOWN-ISSUES.txt ?), and carry out a vote.
>>>
>>> 2. Later, when MR-2178 patch is committed, cut a release for 0.22.1. (I
>>> have heard of folks working on it, but no date commitments yet.)
>>>
>>> The reason a 0.22.0 release should happen, IMHO, even without MR-2178, is
>>> that the other components of Hadoop stack (e.g. Pig, hbase, hive etc) will
>>> have a release to test against, and fix any issues for incompatibility.
>>> Also, for folks who do not use kerberos authentication (>90% of the hadoop
>>> users, to my knowledge), it will be a release closest to trunk, with HDFS
>>> appends. Plus, any issues found in the HDFS part of the release would be
>>> directly applicable to the trunk and other future releases. (Not so about
>>> map-reduce, which will undergo major surgery in trunk because of MR-279.)
>>>
>>> Thoughts ?
>>>
>>
>> Sounds reasonable to me.  #1 is what we did for the 0.21 release.
>
> Actually, I didn't read close enough. For 0.21 we actually removed the
> task tracker, think it makes sense to do that here as well.
>
> Thanks,
> Eli

Eli,

I hope we don't have to remove the TaskTracker *grin*.

Devaraj/Allen,

You mean LinuxTaskController only, right ?

The default is DefaultTaskController, which is not relevant to these
discussions.

+1. This is the easiest way to get 0.22.0 out.

- Milind
---
Milind Bhandarkar
Greenplum Labs, EMC
On 8/2/11 1:07 PM, "Devaraj Das" <[EMAIL PROTECTED]> wrote:

>+1 to removing the TaskController code for the 0.22 release.
>
>In the MR-279 branch, the security issues identified in MAPREDUCE-2178
>have been taken care of (to my knowledge).
>
>On Aug 2, 2011, at 12:58 PM, Eli Collins wrote:
>
>> On Tue, Aug 2, 2011 at 12:25 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
>>> On Tue, Aug 2, 2011 at 12:19 PM,  <[EMAIL PROTECTED]> wrote:
>>>> Having talked to various folks in the community about their
>>>>willingness to
>>>> stabilize and use 0.22 and make it production-quality, here is what I
>>>> propose:
>>>>
>>>> 1. Cut a release 0.22.0 without mapreduce-2178 patch, with
>>>> hadoop.security.authentication set to simple (I.e. No authentication).
>>>> Make sure that MR-2178 is highlighted as known-issue in the top-level
>>>>dir
>>>> (create a KNOWN-ISSUES.txt ?), and carry out a vote.
>>>>
>>>> 2. Later, when MR-2178 patch is committed, cut a release for 0.22.1.
>>>>(I
>>>> have heard of folks working on it, but no date commitments yet.)
>>>>
>>>> The reason a 0.22.0 release should happen, IMHO, even without
>>>>MR-2178, is
>>>> that the other components of Hadoop stack (e.g. Pig, hbase, hive etc)
>>>>will
>>>> have a release to test against, and fix any issues for
>>>>incompatibility.
>>>> Also, for folks who do not use kerberos authentication (>90% of the
>>>>hadoop
>>>> users, to my knowledge), it will be a release closest to trunk, with
>>>>HDFS
>>>> appends. Plus, any issues found in the HDFS part of the release would
>>>>be
>>>> directly applicable to the trunk and other future releases. (Not so
>>>>about
>>>> map-reduce, which will undergo major surgery in trunk because of
>>>>MR-279.)
>>>>
>>>> Thoughts ?
>>>>
>>>
>>> Sounds reasonable to me.  #1 is what we did for the 0.21 release.
>>
>> Actually, I didn't read close enough. For 0.21 we actually removed the
>> task tracker, think it makes sense to do that here as well.
>>
>> Thanks,
>> Eli
>
>

Yes...  I meant the LTC.

On Aug 2, 2011, at 1:23 PM, <[EMAIL PROTECTED]> wrote:

> Eli,
>
> I hope we don't have to remove the TaskTracker *grin*.
>
> Devaraj/Allen,
>
> You mean LinuxTaskController only, right ?
>
> The default is DefaultTaskController, which is not relevant to these
> discussions.
>
> +1. This is the easiest way to get 0.22.0 out.
>
> - Milind
> ---
> Milind Bhandarkar
> Greenplum Labs, EMC
>
>
>
>
> On 8/2/11 1:07 PM, "Devaraj Das" <[EMAIL PROTECTED]> wrote:
>
>> +1 to removing the TaskController code for the 0.22 release.
>>
>> In the MR-279 branch, the security issues identified in MAPREDUCE-2178
>> have been taken care of (to my knowledge).
>>
>> On Aug 2, 2011, at 12:58 PM, Eli Collins wrote:
>>
>>> On Tue, Aug 2, 2011 at 12:25 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
>>>> On Tue, Aug 2, 2011 at 12:19 PM,  <[EMAIL PROTECTED]> wrote:
>>>>> Having talked to various folks in the community about their
>>>>> willingness to
>>>>> stabilize and use 0.22 and make it production-quality, here is what I
>>>>> propose:
>>>>>
>>>>> 1. Cut a release 0.22.0 without mapreduce-2178 patch, with
>>>>> hadoop.security.authentication set to simple (I.e. No authentication).
>>>>> Make sure that MR-2178 is highlighted as known-issue in the top-level
>>>>> dir
>>>>> (create a KNOWN-ISSUES.txt ?), and carry out a vote.
>>>>>
>>>>> 2. Later, when MR-2178 patch is committed, cut a release for 0.22.1.
>>>>> (I
>>>>> have heard of folks working on it, but no date commitments yet.)
>>>>>
>>>>> The reason a 0.22.0 release should happen, IMHO, even without
>>>>> MR-2178, is
>>>>> that the other components of Hadoop stack (e.g. Pig, hbase, hive etc)
>>>>> will
>>>>> have a release to test against, and fix any issues for
>>>>> incompatibility.
>>>>> Also, for folks who do not use kerberos authentication (>90% of the
>>>>> hadoop
>>>>> users, to my knowledge), it will be a release closest to trunk, with
>>>>> HDFS
>>>>> appends. Plus, any issues found in the HDFS part of the release would
>>>>> be
>>>>> directly applicable to the trunk and other future releases. (Not so
>>>>> about
>>>>> map-reduce, which will undergo major surgery in trunk because of
>>>>> MR-279.)
>>>>>
>>>>> Thoughts ?
>>>>>
>>>>
>>>> Sounds reasonable to me.  #1 is what we did for the 0.21 release.
>>>
>>> Actually, I didn't read close enough. For 0.21 we actually removed the
>>> task tracker, think it makes sense to do that here as well.
>>>
>>> Thanks,
>>> Eli
>>
>>
>

+1 (non-binding).  Sounds good to me.
--Matt
On Tue, Aug 2, 2011 at 1:27 PM, Devaraj Das <[EMAIL PROTECTED]> wrote:

> Yes...  I meant the LTC.
>
> On Aug 2, 2011, at 1:23 PM, <[EMAIL PROTECTED]> wrote:
>
> > Eli,
> >
> > I hope we don't have to remove the TaskTracker *grin*.
> >
> > Devaraj/Allen,
> >
> > You mean LinuxTaskController only, right ?
> >
> > The default is DefaultTaskController, which is not relevant to these
> > discussions.
> >
> > +1. This is the easiest way to get 0.22.0 out.
> >
> > - Milind
> > ---
> > Milind Bhandarkar
> > Greenplum Labs, EMC
> >
> >
> >
> >
> > On 8/2/11 1:07 PM, "Devaraj Das" <[EMAIL PROTECTED]> wrote:
> >
> >> +1 to removing the TaskController code for the 0.22 release.
> >>
> >> In the MR-279 branch, the security issues identified in MAPREDUCE-2178
> >> have been taken care of (to my knowledge).
> >>
> >> On Aug 2, 2011, at 12:58 PM, Eli Collins wrote:
> >>
> >>> On Tue, Aug 2, 2011 at 12:25 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
> >>>> On Tue, Aug 2, 2011 at 12:19 PM,  <[EMAIL PROTECTED]> wrote:
> >>>>> Having talked to various folks in the community about their
> >>>>> willingness to
> >>>>> stabilize and use 0.22 and make it production-quality, here is what I
> >>>>> propose:
> >>>>>
> >>>>> 1. Cut a release 0.22.0 without mapreduce-2178 patch, with
> >>>>> hadoop.security.authentication set to simple (I.e. No
> authentication).
> >>>>> Make sure that MR-2178 is highlighted as known-issue in the top-level
> >>>>> dir
> >>>>> (create a KNOWN-ISSUES.txt ?), and carry out a vote.
> >>>>>
> >>>>> 2. Later, when MR-2178 patch is committed, cut a release for 0.22.1.
> >>>>> (I
> >>>>> have heard of folks working on it, but no date commitments yet.)
> >>>>>
> >>>>> The reason a 0.22.0 release should happen, IMHO, even without
> >>>>> MR-2178, is
> >>>>> that the other components of Hadoop stack (e.g. Pig, hbase, hive etc)
> >>>>> will
> >>>>> have a release to test against, and fix any issues for
> >>>>> incompatibility.
> >>>>> Also, for folks who do not use kerberos authentication (>90% of the
> >>>>> hadoop
> >>>>> users, to my knowledge), it will be a release closest to trunk, with
> >>>>> HDFS
> >>>>> appends. Plus, any issues found in the HDFS part of the release would
> >>>>> be
> >>>>> directly applicable to the trunk and other future releases. (Not so
> >>>>> about
> >>>>> map-reduce, which will undergo major surgery in trunk because of
> >>>>> MR-279.)
> >>>>>
> >>>>> Thoughts ?
> >>>>>
> >>>>
> >>>> Sounds reasonable to me.  #1 is what we did for the 0.21 release.
> >>>
> >>> Actually, I didn't read close enough. For 0.21 we actually removed the
> >>> task tracker, think it makes sense to do that here as well.
> >>>
> >>> Thanks,
> >>> Eli
> >>
> >>
> >
>
>

+1.

n.

On Aug 2, 2011, at 1:35 PM, Matt Foley wrote:

> +1 (non-binding).  Sounds good to me.
> --Matt
>
>
> On Tue, Aug 2, 2011 at 1:27 PM, Devaraj Das <[EMAIL PROTECTED]> wrote:
>
>> Yes...  I meant the LTC.
>>
>> On Aug 2, 2011, at 1:23 PM, <[EMAIL PROTECTED]> wrote:
>>
>>> Eli,
>>>
>>> I hope we don't have to remove the TaskTracker *grin*.
>>>
>>> Devaraj/Allen,
>>>
>>> You mean LinuxTaskController only, right ?
>>>
>>> The default is DefaultTaskController, which is not relevant to these
>>> discussions.
>>>
>>> +1. This is the easiest way to get 0.22.0 out.
>>>
>>> - Milind
>>> ---
>>> Milind Bhandarkar
>>> Greenplum Labs, EMC
>>>
>>>
>>>
>>>
>>> On 8/2/11 1:07 PM, "Devaraj Das" <[EMAIL PROTECTED]> wrote:
>>>
>>>> +1 to removing the TaskController code for the 0.22 release.
>>>>
>>>> In the MR-279 branch, the security issues identified in MAPREDUCE-2178
>>>> have been taken care of (to my knowledge).
>>>>
>>>> On Aug 2, 2011, at 12:58 PM, Eli Collins wrote:
>>>>
>>>>> On Tue, Aug 2, 2011 at 12:25 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
>>>>>> On Tue, Aug 2, 2011 at 12:19 PM,  <[EMAIL PROTECTED]> wrote:
>>>>>>> Having talked to various folks in the community about their
>>>>>>> willingness to
>>>>>>> stabilize and use 0.22 and make it production-quality, here is what I
>>>>>>> propose:
>>>>>>>
>>>>>>> 1. Cut a release 0.22.0 without mapreduce-2178 patch, with
>>>>>>> hadoop.security.authentication set to simple (I.e. No
>> authentication).
>>>>>>> Make sure that MR-2178 is highlighted as known-issue in the top-level
>>>>>>> dir
>>>>>>> (create a KNOWN-ISSUES.txt ?), and carry out a vote.
>>>>>>>
>>>>>>> 2. Later, when MR-2178 patch is committed, cut a release for 0.22.1.
>>>>>>> (I
>>>>>>> have heard of folks working on it, but no date commitments yet.)
>>>>>>>
>>>>>>> The reason a 0.22.0 release should happen, IMHO, even without
>>>>>>> MR-2178, is
>>>>>>> that the other components of Hadoop stack (e.g. Pig, hbase, hive etc)
>>>>>>> will
>>>>>>> have a release to test against, and fix any issues for
>>>>>>> incompatibility.
>>>>>>> Also, for folks who do not use kerberos authentication (>90% of the
>>>>>>> hadoop
>>>>>>> users, to my knowledge), it will be a release closest to trunk, with
>>>>>>> HDFS
>>>>>>> appends. Plus, any issues found in the HDFS part of the release would
>>>>>>> be
>>>>>>> directly applicable to the trunk and other future releases. (Not so
>>>>>>> about
>>>>>>> map-reduce, which will undergo major surgery in trunk because of
>>>>>>> MR-279.)
>>>>>>>
>>>>>>> Thoughts ?
>>>>>>>
>>>>>>
>>>>>> Sounds reasonable to me.  #1 is what we did for the 0.21 release.
>>>>>
>>>>> Actually, I didn't read close enough. For 0.21 we actually removed the
>>>>> task tracker, think it makes sense to do that here as well.
>>>>>
>>>>> Thanks,
>>>>> Eli
>>>>
>>>>
>>>
>>
>>

Nigel,

I have created https://issues.apache.org/jira/browse/MAPREDUCE-2767, and
uploaded the patch.

Please review. Thanks.

- milind

---
Milind Bhandarkar
Greenplum Labs, EMC
((Disclaimer: Opinions expressed in this email are those of the author,
and do
not necessarily represent the views of any organization, past or present,
the author might be affiliated with.)

On 8/2/11 11:44 PM, "Nigel Daley" <[EMAIL PROTECTED]> wrote:

>+1.
>
>n.
>
>On Aug 2, 2011, at 1:35 PM, Matt Foley wrote:
>
>> +1 (non-binding).  Sounds good to me.
>> --Matt
>>
>>
>> On Tue, Aug 2, 2011 at 1:27 PM, Devaraj Das <[EMAIL PROTECTED]>
>>wrote:
>>
>>> Yes...  I meant the LTC.
>>>
>>> On Aug 2, 2011, at 1:23 PM, <[EMAIL PROTECTED]> wrote:
>>>
>>>> Eli,
>>>>
>>>> I hope we don't have to remove the TaskTracker *grin*.
>>>>
>>>> Devaraj/Allen,
>>>>
>>>> You mean LinuxTaskController only, right ?
>>>>
>>>> The default is DefaultTaskController, which is not relevant to these
>>>> discussions.
>>>>
>>>> +1. This is the easiest way to get 0.22.0 out.
>>>>
>>>> - Milind
>>>> ---
>>>> Milind Bhandarkar
>>>> Greenplum Labs, EMC
>>>>
>>>>
>>>>
>>>>
>>>> On 8/2/11 1:07 PM, "Devaraj Das" <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> +1 to removing the TaskController code for the 0.22 release.
>>>>>
>>>>> In the MR-279 branch, the security issues identified in
>>>>>MAPREDUCE-2178
>>>>> have been taken care of (to my knowledge).
>>>>>
>>>>> On Aug 2, 2011, at 12:58 PM, Eli Collins wrote:
>>>>>
>>>>>> On Tue, Aug 2, 2011 at 12:25 PM, Eli Collins <[EMAIL PROTECTED]>
>>>>>>wrote:
>>>>>>> On Tue, Aug 2, 2011 at 12:19 PM,  <[EMAIL PROTECTED]>
>>>>>>>wrote:
>>>>>>>> Having talked to various folks in the community about their
>>>>>>>> willingness to
>>>>>>>> stabilize and use 0.22 and make it production-quality, here is
>>>>>>>>what I
>>>>>>>> propose:
>>>>>>>>
>>>>>>>> 1. Cut a release 0.22.0 without mapreduce-2178 patch, with
>>>>>>>> hadoop.security.authentication set to simple (I.e. No
>>> authentication).
>>>>>>>> Make sure that MR-2178 is highlighted as known-issue in the
>>>>>>>>top-level
>>>>>>>> dir
>>>>>>>> (create a KNOWN-ISSUES.txt ?), and carry out a vote.
>>>>>>>>
>>>>>>>> 2. Later, when MR-2178 patch is committed, cut a release for
>>>>>>>>0.22.1.
>>>>>>>> (I
>>>>>>>> have heard of folks working on it, but no date commitments yet.)
>>>>>>>>
>>>>>>>> The reason a 0.22.0 release should happen, IMHO, even without
>>>>>>>> MR-2178, is
>>>>>>>> that the other components of Hadoop stack (e.g. Pig, hbase, hive
>>>>>>>>etc)
>>>>>>>> will
>>>>>>>> have a release to test against, and fix any issues for
>>>>>>>> incompatibility.
>>>>>>>> Also, for folks who do not use kerberos authentication (>90% of
>>>>>>>>the
>>>>>>>> hadoop
>>>>>>>> users, to my knowledge), it will be a release closest to trunk,
>>>>>>>>with
>>>>>>>> HDFS
>>>>>>>> appends. Plus, any issues found in the HDFS part of the release
>>>>>>>>would
>>>>>>>> be
>>>>>>>> directly applicable to the trunk and other future releases. (Not
>>>>>>>>so
>>>>>>>> about
>>>>>>>> map-reduce, which will undergo major surgery in trunk because of
>>>>>>>> MR-279.)
>>>>>>>>
>>>>>>>> Thoughts ?
>>>>>>>>
>>>>>>>
>>>>>>> Sounds reasonable to me.  #1 is what we did for the 0.21 release.
>>>>>>
>>>>>> Actually, I didn't read close enough. For 0.21 we actually removed
>>>>>>the
>>>>>> task tracker, think it makes sense to do that here as well.
>>>>>>
>>>>>> Thanks,
>>>>>> Eli
>>>>>
>>>>>
>>>>
>>>
>>>
>
>

On Tue, Aug 2, 2011 at 1:23 PM,  <[EMAIL PROTECTED]> wrote:
> Eli,
>
> I hope we don't have to remove the TaskTracker *grin*.
>
> Devaraj/Allen,
>
> You mean LinuxTaskController only, right ?
>

Yup, meant the LTC.

Thanks,
Eli

0.22 seems to be loaded enough to add value even with
a couple of the bugs like mentioned. This is a significant step towards
field-testing of 0.23 and so on and will give to applications on top of Hadoop
a chance to with latest Apache release of what Hadoop might look like in the
near future.

+1 on the proposed changes.

Cos

On Tue, Aug 02, 2011 at 03:19PM, [EMAIL PROTECTED] wrote:
> Having talked to various folks in the community about their willingness to
> stabilize and use 0.22 and make it production-quality, here is what I
> propose:
>
> 1. Cut a release 0.22.0 without mapreduce-2178 patch, with
> hadoop.security.authentication set to simple (I.e. No authentication).
> Make sure that MR-2178 is highlighted as known-issue in the top-level dir
> (create a KNOWN-ISSUES.txt ?), and carry out a vote.
>
> 2. Later, when MR-2178 patch is committed, cut a release for 0.22.1. (I
> have heard of folks working on it, but no date commitments yet.)
>
> The reason a 0.22.0 release should happen, IMHO, even without MR-2178, is
> that the other components of Hadoop stack (e.g. Pig, hbase, hive etc) will
> have a release to test against, and fix any issues for incompatibility.
> Also, for folks who do not use kerberos authentication (>90% of the hadoop
> users, to my knowledge), it will be a release closest to trunk, with HDFS
> appends. Plus, any issues found in the HDFS part of the release would be
> directly applicable to the trunk and other future releases. (Not so about
> map-reduce, which will undergo major surgery in trunk because of MR-279.)
>
> Thoughts ?
>
> - milind
>
> ---
> Milind Bhandarkar
> Chief Architect, Greenplum Labs, Data Computing Division, EMC
> +1-650-523-3858 (W)
> +1-408-666-8483 (M)
>
>
>
>
> On 8/1/11 11:33 PM, "Nigel Daley" <[EMAIL PROTECTED]> wrote:
>
> >Quick update on Hadoop 0.22.  Candidate build of 0.22 is still blocked by
> >a couple key issues:
> >* https://issues.apache.org/jira/browse/MAPREDUCE-2178
> >* MapReduce build problem in our CI environment
> >* HDFS & MapReduce test failures in our CI environment
> >
> >A couple people have expressed to me their interest in helping to
> >stabilize 0.22.  Any status updates on the above issues?
> >
> >Alternatively, we could drop support for security in 0.22 and not fix
> >MAPREDUCE-2178.  Thoughts?
> >
> >Thanks,
> >Nige
> >
>

On Aug 2, 2011, at 12:19 PM, <[EMAIL PROTECTED]>
 <[EMAIL PROTECTED]> wrote:
> 1. Cut a release 0.22.0 without mapreduce-2178 patch, with
> hadoop.security.authentication set to simple (I.e. No authentication).
> Make sure that MR-2178 is highlighted as known-issue in the top-level dir
> (create a KNOWN-ISSUES.txt ?), and carry out a vote.

This is a great way to end up on CERT, "known issue" or not.  That's not a place where you want to be.

> The reason a 0.22.0 release should happen, IMHO, even without MR-2178, is
> that the other components of Hadoop stack (e.g. Pig, hbase, hive etc) will
> have a release to test against, and fix any issues for incompatibility.
> Also, for folks who do not use kerberos authentication (>90% of the hadoop
> users, to my knowledge),

I'm 99% certain that the task controller code is independent of Kerberos usage since it has its own conflg var...
> it will be a release closest to trunk, with HDFS
> appends. Plus, any issues found in the HDFS part of the release would be
> directly applicable to the trunk and other future releases. (Not so about
> map-reduce, which will undergo major surgery in trunk because of MR-279.)
>
> Thoughts ?

I'd rather have the task-controller code ripped out than shipped broken.

나의 iPad에서 보냄

2011. 8. 2. 오후 3:33 Nigel Daley <[EMAIL PROTECTED]> 작���:

> Quick update on Hadoop 0.22.  Candidate build of 0.22 is still blocked by a couple key issues:
> * https://issues.apache.org/jira/browse/MAPREDUCE-2178
> * MapReduce build problem in our CI environment
> * HDFS & MapReduce test failures in our CI environment
>
> A couple people have expressed to me their interest in helping to stabilize 0.22.  Any status updates on the above issues?
>
> Alternatively, we could drop support for security in 0.22 and not fix MAPREDUCE-2178.  Thoughts?
>
> Thanks,
> Nige
>