Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # dev - Re: [Vote] Merge branch-trunk-win to trunk


Copy link to this message
-
RE: [Vote] Merge branch-trunk-win to trunk
Uma Maheswara Rao G 2013-03-01, 05:20
+1 (non-binding)

 Thanks a lot for the work done by Suresh and team of community!
I don't think there will be much problems because of platform dependancy as our development is with Java. If we have some native code portings, then dev members has to take care of them.
One question regarding to it:
Ex: if one contributor is giving the patch for some native code porting for some performance improvements and he is interested only in Linux. Then I hope some other contributors will help in getting the windows patch if possible.
If others busy to get that done with in time lines, then I think we can commit Linux support patch and leave one JIRA open for Windows support? [ make sure that porting options configurable and platform check and give release note about platform support..etc?]

Regards,
Uma

________________________________________
From: Suresh Srinivas [[EMAIL PROTECTED]]
Sent: Wednesday, February 27, 2013 4:25 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: [Vote] Merge branch-trunk-win to trunk

I had posted heads up about merging branch-trunk-win to trunk on Feb 8th. I
am happy to announce that we are ready for the merge.

Here is a brief recap on the highlights of the work done:
- Command-line scripts for the Hadoop surface area
- Mapping the HDFS permissions model to Windows
- Abstracted and reconciled mismatches around differences in Path semantics
in Java and Windows
- Native Task Controller for Windows
- Implementation of a Block Placement Policy to support cloud environments,
more specifically Azure.
- Implementation of Hadoop native libraries for Windows (compression
codecs, native I/O)
- Several reliability issues, including race-conditions, intermittent test
failures, resource leaks.
- Several new unit test cases written for the above changes

Please find the details of the work in CHANGES.branch-trunk-win.txt -
Common changes<http://bit.ly/Xe7Ynv>, HDFS changes<http://bit.ly/13QOSo9>,
and YARN and MapReduce changes <http://bit.ly/128zzMt>. This is the work
ported from branch-1-win to a branch based on trunk.

For details of the testing done, please see the thread -
http://bit.ly/WpavJ4. Merge patch for this is available on HADOOP-8562<
https://issues.apache.org/jira/browse/HADOOP-8562>.

This was a large undertaking that involved developing code, testing the
entire Hadoop stack, including scale tests. This is made possible only with
the contribution from many many folks in the community. Following people
contributed to this work: Ivan Mitic, Chuan Liu, Ramya Sunil, Bikas Saha,
Kanna Karanam, John Gordon, Brandon Li, Chris Nauroth, David Lao, Sumadhur
Reddy Bolli, Arpit Agarwal, Ahmed El Baz, Mike Liddell, Jing Zhao, Thejas
Nair, Steve Maine, Ganeshan Iyer, Raja Aluri, Giridharan Kesavan, Ramya
Bharathi Nimmagadda, Daryn Sharp, Arun Murthy, Tsz-Wo Nicholas Sze, Suresh
Srinivas and Sanjay Radia. There are many others who contributed as well
providing feedback and comments on numerous jiras.

The vote will run for seven days and will end on March 5, 6:00PM PST.

Regards,
Suresh
On Thu, Feb 7, 2013 at 6:41 PM, Mahadevan Venkatraman
<[EMAIL PROTECTED]>wrote:

> It is super exciting to look at the prospect of these changes being merged
> to trunk. Having Windows as one of the supported Hadoop platforms is a
> fantastic opportunity both for the Hadoop project and Microsoft customers.
>
> This work began around a year back when a few of us started with a basic
> port of Hadoop on Windows. Ever since, the Hadoop team in Microsoft have
> made significant progress in the following areas:
> (PS: Some of these items are already included in Suresh's email, but
> including again for completeness)
>
> - Command-line scripts for the Hadoop surface area
> - Mapping the HDFS permissions model to Windows
> - Abstracted and reconciled mismatches around differences in Path
> semantics in Java and Windows
> - Native Task Controller for Windows
http://hortonworks.com/download/