Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - update re: Hadoop-1.1.0-rc0


Copy link to this message
-
update re: Hadoop-1.1.0-rc0
Matt Foley 2012-07-12, 21:26
The following additional patches are committed to branch-1.1 and will be in
1.1.0:
- HADOOP-8365 - Add flag to disable durable sync
- HADOOP-8399 - Remove JDK5 dependency from Hadoop 1.0+ line
- HADOOP-8417 - HADOOP-6963 didn't update hadoop-core-pom-template.xml
- HADOOP-8445 - Token should not print the password in toString
- MAPREDUCE-2903 - Map Tasks graph is throwing XML Parse error when Job is
executed with 0 maps
- MAPREDUCE-2129 - Job may hang if
mapreduce.job.committer.setup.cleanup.needed=false and
mapreduce.map/reduce.failures.maxpercent>0

I've been trying to build and test the RC0, but am having build problems
with the 32-bit version.  Should have it figured out soon.

The following patch is committed to branch-1 and I'll see if I can apply it
to branch-1.1 before my next build:
- HDFS-3617 - Port HDFS-96 to branch-1 (support blocks greater than 2GB)

The following patch is committed to branch-1 but had conflicts with
branch-1.1 and will not be in RC0:
- HDFS-3596 - Improve FSEditLog pre-allocation in branch-1

The following patches are deferred, as they do not have currently committed
patches:
- HADOOP-7823 - port HADOOP-4012 to branch-1 (splitting support for bzip2)
- MAPREDUCE-4049 - plugin for generic shuffle service

Thanks,
--Matt

On Sat, Jul 7, 2012 at 6:32 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:

> Also, I have updated HADOOP-8417 against 1.1.0 and we need to include it.
> Otherwise, 1.1 will have the same issues for the downstream projects as
> 1.0.3
> had.
>
> Cos
>
> On Sat, Jul 07, 2012 at 05:52PM, Konstantin Boudnik wrote:
> > Matt,
> >
> > Thanks for the update.
> >
> > HADOOP-8399 would be beneficial for BigTop release and it is marked for
> 1.1.0
> > release. The patch is available for a while now and if someone can
> review I'd
> > go ahead and commit it today.
> >
> > I am working on the content of 0.3.1 BigTop release and will shortly
> post the
> > vote for it. Once Hadoop 1.1 rc is cut we'll start testing it with the
> rest of
> > the stack.
> >
> > Cos
> >
> > On Fri, Jul 06, 2012 at 02:24PM, Matt Foley wrote:
> > > Hi Cos,
> > > the query string didn't come thru on the link you sent, but the jira
> query
> > > I use is:
> > >     project in (HADOOP,HDFS,MAPREDUCE) and (("Target Version/s" > '1.1.0'
> > > and (fixVersion != '1.1.0' or fixVersion is EMPTY)) or (fixVersion > > > '1.1.0' and "Target Version/s" is EMPTY)) and (status != Closed and
> status
> > > != Resolved) ORDER BY KEY
> > >
> > > You're correct that there are quite a few, currently 107, open jiras
> > > originally targeted for 1.1.0 that do not have committed fixes.  Many
> of
> > > these are just the inherited backlog of previously identified work.  I
> need
> > > to move them to "Target Version/s" = 1.1.1.
> > >
> > > Folks have requested that the following currently open jiras be
> included in
> > > 1.1.0:
> > >
> > > HADOOP-8417 - HADOOP-6963 didn't update hadoop-core-pom-template.xml
> > > HADOOP-8445 - Token should not print the password in toString
> > > HDFS-96 - HDFS does not support blocks greater than 2GB
> > > HDFS-3596 - Improve FSEditLog pre-allocation in branch-1
> > > MAPREDUCE-2903 - Map Tasks graph is throwing XML Parse error when Job
> is
> > > executed with 0 maps
> > > MAPREDUCE-2129 - Job may hang if
> > > mapreduce.job.committer.setup.cleanup.needed=false and
> > > mapreduce.map/reduce.failures.maxpercent>0
> > > ---
> > > MAPREDUCE-4049 - plugin for generic shuffle service
> > > HADOOP-7823 - port HADOOP-4012 to branch-1 (splitting support for
> bzip2)
> > >
> > > The first six are simple patches that I am comfortable including.
> > > The last two are complex patches that have not yet been committed.
> > > I am planning to defer those two to 1.1.1.
> > >
> > > Beyond that, I'm going to cut 1.1.0-rc0 from the current state of
> > > branch-1.1.
> > > I'm planning to do that this weekend.  This is obviously delayed from
> the
> > > previous plan, for which I apologize.
> > >
> > > Comments welcome.