Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> RE: [Vote] Merge branch-trunk-win to trunk


Copy link to this message
-
Re: [Vote] Merge branch-trunk-win to trunk
+1 non-binding
Nice to see that this work is going to trunk.

Raja  Aluri
On Tue, Feb 26, 2013 at 2:55 PM, Suresh Srinivas <[EMAIL PROTECTED]>wrote:

> I had posted heads up about merging branch-trunk-win to trunk on Feb 8th. I
> am happy to announce that we are ready for the merge.
>
> Here is a brief recap on the highlights of the work done:
> - Command-line scripts for the Hadoop surface area
> - Mapping the HDFS permissions model to Windows
> - Abstracted and reconciled mismatches around differences in Path semantics
> in Java and Windows
> - Native Task Controller for Windows
> - Implementation of a Block Placement Policy to support cloud environments,
> more specifically Azure.
> - Implementation of Hadoop native libraries for Windows (compression
> codecs, native I/O)
> - Several reliability issues, including race-conditions, intermittent test
> failures, resource leaks.
> - Several new unit test cases written for the above changes
>
> Please find the details of the work in CHANGES.branch-trunk-win.txt -
> Common changes<http://bit.ly/Xe7Ynv>, HDFS changes<http://bit.ly/13QOSo9>,
> and YARN and MapReduce changes <http://bit.ly/128zzMt>. This is the work
> ported from branch-1-win to a branch based on trunk.
>
> For details of the testing done, please see the thread -
> http://bit.ly/WpavJ4. Merge patch for this is available on HADOOP-8562<
> https://issues.apache.org/jira/browse/HADOOP-8562>.
>
> This was a large undertaking that involved developing code, testing the
> entire Hadoop stack, including scale tests. This is made possible only with
> the contribution from many many folks in the community. Following people
> contributed to this work: Ivan Mitic, Chuan Liu, Ramya Sunil, Bikas Saha,
> Kanna Karanam, John Gordon, Brandon Li, Chris Nauroth, David Lao, Sumadhur
> Reddy Bolli, Arpit Agarwal, Ahmed El Baz, Mike Liddell, Jing Zhao, Thejas
> Nair, Steve Maine, Ganeshan Iyer, Raja Aluri, Giridharan Kesavan, Ramya
> Bharathi Nimmagadda, Daryn Sharp, Arun Murthy, Tsz-Wo Nicholas Sze, Suresh
> Srinivas and Sanjay Radia. There are many others who contributed as well
> providing feedback and comments on numerous jiras.
>
> The vote will run for seven days and will end on March 5, 6:00PM PST.
>
> Regards,
> Suresh
>
>
>
>
> On Thu, Feb 7, 2013 at 6:41 PM, Mahadevan Venkatraman
> <[EMAIL PROTECTED]>wrote:
>
> > It is super exciting to look at the prospect of these changes being
> merged
> > to trunk. Having Windows as one of the supported Hadoop platforms is a
> > fantastic opportunity both for the Hadoop project and Microsoft
> customers.
> >
> > This work began around a year back when a few of us started with a basic
> > port of Hadoop on Windows. Ever since, the Hadoop team in Microsoft have
> > made significant progress in the following areas:
> > (PS: Some of these items are already included in Suresh's email, but
> > including again for completeness)
> >
> > - Command-line scripts for the Hadoop surface area
> > - Mapping the HDFS permissions model to Windows
> > - Abstracted and reconciled mismatches around differences in Path
> > semantics in Java and Windows
> > - Native Task Controller for Windows
> > - Implementation of a Block Placement Policy to support cloud
> > environments, more specifically Azure.
> > - Implementation of Hadoop native libraries for Windows (compression
> > codecs, native I/O) - Several reliability issues, including
> > race-conditions, intermittent test failures, resource leaks.
> > - Several new unit test cases written for the above changes
> >
> > In the process, we have closely engaged with the Apache open source
> > community and have got great support and assistance from the community in
> > terms of contributing fixes, code review comments and commits.
> >
> > In addition, the Hadoop team at Microsoft has also made good progress in
> > other projects including Hive, Pig, Sqoop, Oozie, HCat and HBase. Many of
> > these changes have already been committed to the respective trunks with