Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> update re: Hadoop-1.1.0-rc0


+
Matt Foley 2012-07-12, 21:26
Copy link to this message
-
Re: update re: Hadoop-1.1.0-rc0
Matt,

Meanwhile, I am planning to promote the following changes:
HDFS-2465 - Add HDFS support for fadvise readahead and
drop-behind<https://issues.apache.org/jira/browse/HDFS-2465>
HADOOP-Support fadvise and sync_data_range in NativeIO, add ReadaheadPool
class <https://issues.apache.org/jira/browse/HADOOP-7753>

These changes should be low risk - These i/o improvements are turned off by
default in configuration. Brandon Li has done quite a bit of testing
(TestDFSIO, teragen, terasort, teravalidate etc.) on a small cluster (3
nodes) in addition to unit tests.

Regards,
Suresh

On Thu, Jul 12, 2012 at 2:26 PM, Matt Foley <[EMAIL PROTECTED]> wrote:

> The following additional patches are committed to branch-1.1 and will be in
> 1.1.0:
> - HADOOP-8365 - Add flag to disable durable sync
> - HADOOP-8399 - Remove JDK5 dependency from Hadoop 1.0+ line
> - HADOOP-8417 - HADOOP-6963 didn't update hadoop-core-pom-template.xml
> - HADOOP-8445 - Token should not print the password in toString
> - MAPREDUCE-2903 - Map Tasks graph is throwing XML Parse error when Job is
> executed with 0 maps
> - MAPREDUCE-2129 - Job may hang if
> mapreduce.job.committer.setup.cleanup.needed=false and
> mapreduce.map/reduce.failures.maxpercent>0
>
> I've been trying to build and test the RC0, but am having build problems
> with the 32-bit version.  Should have it figured out soon.
>
> The following patch is committed to branch-1 and I'll see if I can apply it
> to branch-1.1 before my next build:
> - HDFS-3617 - Port HDFS-96 to branch-1 (support blocks greater than 2GB)
>
> The following patch is committed to branch-1 but had conflicts with
> branch-1.1 and will not be in RC0:
> - HDFS-3596 - Improve FSEditLog pre-allocation in branch-1
>
> The following patches are deferred, as they do not have currently committed
> patches:
> - HADOOP-7823 - port HADOOP-4012 to branch-1 (splitting support for bzip2)
> - MAPREDUCE-4049 - plugin for generic shuffle service
>
> Thanks,
> --Matt
>
> On Sat, Jul 7, 2012 at 6:32 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:
>
> > Also, I have updated HADOOP-8417 against 1.1.0 and we need to include it.
> > Otherwise, 1.1 will have the same issues for the downstream projects as
> > 1.0.3
> > had.
> >
> > Cos
> >
> > On Sat, Jul 07, 2012 at 05:52PM, Konstantin Boudnik wrote:
> > > Matt,
> > >
> > > Thanks for the update.
> > >
> > > HADOOP-8399 would be beneficial for BigTop release and it is marked for
> > 1.1.0
> > > release. The patch is available for a while now and if someone can
> > review I'd
> > > go ahead and commit it today.
> > >
> > > I am working on the content of 0.3.1 BigTop release and will shortly
> > post the
> > > vote for it. Once Hadoop 1.1 rc is cut we'll start testing it with the
> > rest of
> > > the stack.
> > >
> > > Cos
> > >
> > > On Fri, Jul 06, 2012 at 02:24PM, Matt Foley wrote:
> > > > Hi Cos,
> > > > the query string didn't come thru on the link you sent, but the jira
> > query
> > > > I use is:
> > > >     project in (HADOOP,HDFS,MAPREDUCE) and (("Target Version/s" > > '1.1.0'
> > > > and (fixVersion != '1.1.0' or fixVersion is EMPTY)) or (fixVersion > > > > '1.1.0' and "Target Version/s" is EMPTY)) and (status != Closed and
> > status
> > > > != Resolved) ORDER BY KEY
> > > >
> > > > You're correct that there are quite a few, currently 107, open jiras
> > > > originally targeted for 1.1.0 that do not have committed fixes.  Many
> > of
> > > > these are just the inherited backlog of previously identified work.
>  I
> > need
> > > > to move them to "Target Version/s" = 1.1.1.
> > > >
> > > > Folks have requested that the following currently open jiras be
> > included in
> > > > 1.1.0:
> > > >
> > > > HADOOP-8417 - HADOOP-6963 didn't update hadoop-core-pom-template.xml
> > > > HADOOP-8445 - Token should not print the password in toString
> > > > HDFS-96 - HDFS does not support blocks greater than 2GB
> > > > HDFS-3596 - Improve FSEditLog pre-allocation in branch-1
> > > > MAPREDUCE-2903 - Map Tasks graph is throwing XML Parse error when Job

http://hortonworks.com/download/
+
Bhandarkar, Milind 2012-09-14, 18:44