Hi Matt.

I am picking up the hat of BigTop's maintainer for Hadoop 1.x line. And I
wanted to sync up with about the Hadoop 1.1 release outlook, progress, what help
you might need, etc.

I see a few jiras left open in the release
    http://is.gd/OyuaNQ
Is this the correct representation of the current status?
How I can help from BigTop side (I haven't yet finalized the stack's
versions), etc. Looking forward for your input. Thanks.

  Cos

On Fri, May 25, 2012 at 02:49PM, Matt Foley wrote:
> Greetings.  With the approval of a public vote on common-dev@, I have
> branched Hadoop branch-1 to create branch-1.1.  From this, I will create a
> release candidate RC-0 for Hadoop-1.1.0, hopefully to be available shortly
> after this weekend.
>
> There are over 80 patches in branch-1, over and above the contents of
> hadoop-1.0.3.  So I anticipate that some stabilization will be needed,
> before the RC can be approved as a 1.1.0 release.  Your participation in
> assuring a stable RC is very important.  When it becomes available, please
> download it and work with it to determine whether it is stable enough to
> release, and report issues found.  My colleagues and I will do likewise, of
> course, but no one company can adequately exercise a new release with this
> many new contributions.
>
> There are two outstanding issue that are not yet committed, but I know the
> contributors hope to see in 1.1.0:
>     MAPREDUCE-4049 <https://issues.apache.org/jira/browse/MAPREDUCE-4049>
>     HADOOP-4012 <https://issues.apache.org/jira/browse/HADOOP-4012>
> Assuming there is an RC-1, and that these two patches can be committed
> during stabilization of RC-0, I will plan to incorporate these additional
> items in RC-1.
>
> Best regards,
> --Matt
> Release Manager

Hi Cos,
the query string didn't come thru on the link you sent, but the jira query
I use is:
    project in (HADOOP,HDFS,MAPREDUCE) and (("Target Version/s" = '1.1.0'
and (fixVersion != '1.1.0' or fixVersion is EMPTY)) or (fixVersion '1.1.0' and "Target Version/s" is EMPTY)) and (status != Closed and status
!= Resolved) ORDER BY KEY

You're correct that there are quite a few, currently 107, open jiras
originally targeted for 1.1.0 that do not have committed fixes.  Many of
these are just the inherited backlog of previously identified work.  I need
to move them to "Target Version/s" = 1.1.1.

Folks have requested that the following currently open jiras be included in
1.1.0:

HADOOP-8417 - HADOOP-6963 didn't update hadoop-core-pom-template.xml
HADOOP-8445 - Token should not print the password in toString
HDFS-96 - HDFS does not support blocks greater than 2GB
HDFS-3596 - Improve FSEditLog pre-allocation in branch-1
MAPREDUCE-2903 - Map Tasks graph is throwing XML Parse error when Job is
executed with 0 maps
MAPREDUCE-2129 - Job may hang if
mapreduce.job.committer.setup.cleanup.needed=false and
mapreduce.map/reduce.failures.maxpercent>0
---
MAPREDUCE-4049 - plugin for generic shuffle service
HADOOP-7823 - port HADOOP-4012 to branch-1 (splitting support for bzip2)

The first six are simple patches that I am comfortable including.
The last two are complex patches that have not yet been committed.
I am planning to defer those two to 1.1.1.

Beyond that, I'm going to cut 1.1.0-rc0 from the current state of
branch-1.1.
I'm planning to do that this weekend.  This is obviously delayed from the
previous plan, for which I apologize.

Comments welcome.
--Matt
On Tue, Jul 3, 2012 at 8:32 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:

> Hi Matt.
>
> I am picking up the hat of BigTop's maintainer for Hadoop 1.x line. And I
> wanted to sync up with about the Hadoop 1.1 release outlook, progress,
> what help
> you might need, etc.
>
> I see a few jiras left open in the release
>     http://is.gd/OyuaNQ
> Is this the correct representation of the current status?
> How I can help from BigTop side (I haven't yet finalized the stack's
> versions), etc. Looking forward for your input. Thanks.
>
>   Cos
>
> On Fri, May 25, 2012 at 02:49PM, Matt Foley wrote:
> > Greetings.  With the approval of a public vote on common-dev@, I have
> > branched Hadoop branch-1 to create branch-1.1.  From this, I will create
> a
> > release candidate RC-0 for Hadoop-1.1.0, hopefully to be available
> shortly
> > after this weekend.
> >
> > There are over 80 patches in branch-1, over and above the contents of
> > hadoop-1.0.3.  So I anticipate that some stabilization will be needed,
> > before the RC can be approved as a 1.1.0 release.  Your participation in
> > assuring a stable RC is very important.  When it becomes available,
> please
> > download it and work with it to determine whether it is stable enough to
> > release, and report issues found.  My colleagues and I will do likewise,
> of
> > course, but no one company can adequately exercise a new release with
> this
> > many new contributions.
> >
> > There are two outstanding issue that are not yet committed, but I know
> the
> > contributors hope to see in 1.1.0:
> >     MAPREDUCE-4049 <https://issues.apache.org/jira/browse/MAPREDUCE-4049
> >
> >     HADOOP-4012 <https://issues.apache.org/jira/browse/HADOOP-4012>
> > Assuming there is an RC-1, and that these two patches can be committed
> > during stabilization of RC-0, I will plan to incorporate these additional
> > items in RC-1.
> >
> > Best regards,
> > --Matt
> > Release Manager
>

Matt,

Thanks for the update.

HADOOP-8399 would be beneficial for BigTop release and it is marked for 1.1.0
release. The patch is available for a while now and if someone can review I'd
go ahead and commit it today.

I am working on the content of 0.3.1 BigTop release and will shortly post the
vote for it. Once Hadoop 1.1 rc is cut we'll start testing it with the rest of
the stack.

Cos

On Fri, Jul 06, 2012 at 02:24PM, Matt Foley wrote:
> Hi Cos,
> the query string didn't come thru on the link you sent, but the jira query
> I use is:
>     project in (HADOOP,HDFS,MAPREDUCE) and (("Target Version/s" = '1.1.0'
> and (fixVersion != '1.1.0' or fixVersion is EMPTY)) or (fixVersion > '1.1.0' and "Target Version/s" is EMPTY)) and (status != Closed and status
> != Resolved) ORDER BY KEY
>
> You're correct that there are quite a few, currently 107, open jiras
> originally targeted for 1.1.0 that do not have committed fixes.  Many of
> these are just the inherited backlog of previously identified work.  I need
> to move them to "Target Version/s" = 1.1.1.
>
> Folks have requested that the following currently open jiras be included in
> 1.1.0:
>
> HADOOP-8417 - HADOOP-6963 didn't update hadoop-core-pom-template.xml
> HADOOP-8445 - Token should not print the password in toString
> HDFS-96 - HDFS does not support blocks greater than 2GB
> HDFS-3596 - Improve FSEditLog pre-allocation in branch-1
> MAPREDUCE-2903 - Map Tasks graph is throwing XML Parse error when Job is
> executed with 0 maps
> MAPREDUCE-2129 - Job may hang if
> mapreduce.job.committer.setup.cleanup.needed=false and
> mapreduce.map/reduce.failures.maxpercent>0
> ---
> MAPREDUCE-4049 - plugin for generic shuffle service
> HADOOP-7823 - port HADOOP-4012 to branch-1 (splitting support for bzip2)
>
> The first six are simple patches that I am comfortable including.
> The last two are complex patches that have not yet been committed.
> I am planning to defer those two to 1.1.1.
>
> Beyond that, I'm going to cut 1.1.0-rc0 from the current state of
> branch-1.1.
> I'm planning to do that this weekend.  This is obviously delayed from the
> previous plan, for which I apologize.
>
> Comments welcome.
> --Matt
>
>
> On Tue, Jul 3, 2012 at 8:32 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:
>
> > Hi Matt.
> >
> > I am picking up the hat of BigTop's maintainer for Hadoop 1.x line. And I
> > wanted to sync up with about the Hadoop 1.1 release outlook, progress,
> > what help
> > you might need, etc.
> >
> > I see a few jiras left open in the release
> >     http://is.gd/OyuaNQ
> > Is this the correct representation of the current status?
> > How I can help from BigTop side (I haven't yet finalized the stack's
> > versions), etc. Looking forward for your input. Thanks.
> >
> >   Cos
> >
> > On Fri, May 25, 2012 at 02:49PM, Matt Foley wrote:
> > > Greetings.  With the approval of a public vote on common-dev@, I have
> > > branched Hadoop branch-1 to create branch-1.1.  From this, I will create
> > a
> > > release candidate RC-0 for Hadoop-1.1.0, hopefully to be available
> > shortly
> > > after this weekend.
> > >
> > > There are over 80 patches in branch-1, over and above the contents of
> > > hadoop-1.0.3.  So I anticipate that some stabilization will be needed,
> > > before the RC can be approved as a 1.1.0 release.  Your participation in
> > > assuring a stable RC is very important.  When it becomes available,
> > please
> > > download it and work with it to determine whether it is stable enough to
> > > release, and report issues found.  My colleagues and I will do likewise,
> > of
> > > course, but no one company can adequately exercise a new release with
> > this
> > > many new contributions.
> > >
> > > There are two outstanding issue that are not yet committed, but I know
> > the
> > > contributors hope to see in 1.1.0:
> > >     MAPREDUCE-4049 <https://issues.apache.org/jira/browse/MAPREDUCE-4049
> > >
> > >     HADOOP-4012 <https://issues.apache.org/jira/browse/HADOOP-4012>

Also, I have updated HADOOP-8417 against 1.1.0 and we need to include it.
Otherwise, 1.1 will have the same issues for the downstream projects as 1.0.3
had.

Cos

On Sat, Jul 07, 2012 at 05:52PM, Konstantin Boudnik wrote:
> Matt,
>
> Thanks for the update.
>
> HADOOP-8399 would be beneficial for BigTop release and it is marked for 1.1.0
> release. The patch is available for a while now and if someone can review I'd
> go ahead and commit it today.
>
> I am working on the content of 0.3.1 BigTop release and will shortly post the
> vote for it. Once Hadoop 1.1 rc is cut we'll start testing it with the rest of
> the stack.
>
> Cos
>
> On Fri, Jul 06, 2012 at 02:24PM, Matt Foley wrote:
> > Hi Cos,
> > the query string didn't come thru on the link you sent, but the jira query
> > I use is:
> >     project in (HADOOP,HDFS,MAPREDUCE) and (("Target Version/s" = '1.1.0'
> > and (fixVersion != '1.1.0' or fixVersion is EMPTY)) or (fixVersion > > '1.1.0' and "Target Version/s" is EMPTY)) and (status != Closed and status
> > != Resolved) ORDER BY KEY
> >
> > You're correct that there are quite a few, currently 107, open jiras
> > originally targeted for 1.1.0 that do not have committed fixes.  Many of
> > these are just the inherited backlog of previously identified work.  I need
> > to move them to "Target Version/s" = 1.1.1.
> >
> > Folks have requested that the following currently open jiras be included in
> > 1.1.0:
> >
> > HADOOP-8417 - HADOOP-6963 didn't update hadoop-core-pom-template.xml
> > HADOOP-8445 - Token should not print the password in toString
> > HDFS-96 - HDFS does not support blocks greater than 2GB
> > HDFS-3596 - Improve FSEditLog pre-allocation in branch-1
> > MAPREDUCE-2903 - Map Tasks graph is throwing XML Parse error when Job is
> > executed with 0 maps
> > MAPREDUCE-2129 - Job may hang if
> > mapreduce.job.committer.setup.cleanup.needed=false and
> > mapreduce.map/reduce.failures.maxpercent>0
> > ---
> > MAPREDUCE-4049 - plugin for generic shuffle service
> > HADOOP-7823 - port HADOOP-4012 to branch-1 (splitting support for bzip2)
> >
> > The first six are simple patches that I am comfortable including.
> > The last two are complex patches that have not yet been committed.
> > I am planning to defer those two to 1.1.1.
> >
> > Beyond that, I'm going to cut 1.1.0-rc0 from the current state of
> > branch-1.1.
> > I'm planning to do that this weekend.  This is obviously delayed from the
> > previous plan, for which I apologize.
> >
> > Comments welcome.
> > --Matt
> >
> >
> > On Tue, Jul 3, 2012 at 8:32 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:
> >
> > > Hi Matt.
> > >
> > > I am picking up the hat of BigTop's maintainer for Hadoop 1.x line. And I
> > > wanted to sync up with about the Hadoop 1.1 release outlook, progress,
> > > what help
> > > you might need, etc.
> > >
> > > I see a few jiras left open in the release
> > >     http://is.gd/OyuaNQ
> > > Is this the correct representation of the current status?
> > > How I can help from BigTop side (I haven't yet finalized the stack's
> > > versions), etc. Looking forward for your input. Thanks.
> > >
> > >   Cos
> > >
> > > On Fri, May 25, 2012 at 02:49PM, Matt Foley wrote:
> > > > Greetings.  With the approval of a public vote on common-dev@, I have
> > > > branched Hadoop branch-1 to create branch-1.1.  From this, I will create
> > > a
> > > > release candidate RC-0 for Hadoop-1.1.0, hopefully to be available
> > > shortly
> > > > after this weekend.
> > > >
> > > > There are over 80 patches in branch-1, over and above the contents of
> > > > hadoop-1.0.3.  So I anticipate that some stabilization will be needed,
> > > > before the RC can be approved as a 1.1.0 release.  Your participation in
> > > > assuring a stable RC is very important.  When it becomes available,
> > > please
> > > > download it and work with it to determine whether it is stable enough to
> > > > release, and report issues found.  My colleagues and I will do likewise,
> > > of