Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # dev - Heads up - merge branch-trunk-win to trunk


Copy link to this message
-
Heads up - merge branch-trunk-win to trunk
Suresh Srinivas 2013-02-08, 01:42
The support for Hadoop on Windows was proposed in
HADOOP-8079<https://issues.apache.org/jira/browse/HADOOP-8079> almost
a year ago. The goal was to make Hadoop natively integrated, full-featured,
and performance and scalability tuned on Windows Server or Windows Azure.
We are happy to announce that a lot of progress has been made in this
regard.

Initial work started in a feature branch, branch-1-win, based on branch-1.
The details related to the work done in the branch can be seen in
CHANGES.txt<http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/CHANGES.branch-1-win.txt?view=markup>.
This work has been ported to a branch, branch-trunk-win, based on trunk.
Merge patch for this is available on
HADOOP-8562<https://issues.apache.org/jira/browse/HADOOP-8562>
.

Highlights of the work done so far:
1. Necessary changes in Hadoop to run natively on Windows. These changes
handle differences in platforms related to path names, process/task
management etc.
2. Addition of winutils tools for managing file permissions and ownership,
user group mapping, hardlinks, symbolic links, chmod, disk utilization, and
process/task management.
3. Added cmd scripts equivalent to existing shell scripts hadoop-daemon.sh,
start and stop scripts.
4. Addition of block placement policy implemnation to support cloud
enviroment, more specifically Azure.

We are very close to wrapping up the work in branch-trunk-win and getting
ready for a merge. Currently the merge patch is passing close to 100% of
unit tests on Linux. Soon I will call for a vote to merge this branch into
trunk.

Next steps:
1. Call for vote to merge branch-trunk-win to trunk, when the work
completes and precommit build is clean.
2. Start a discussion on adding Jenkins precommit builds on windows and how
to integrate that with the existing commit process.

Let me know if you have any questions.

Regards,
Suresh
+
Eli Collins 2013-02-08, 02:46
+
Suresh Srinivas 2013-02-08, 17:12
+
Mahadevan Venkatraman 2013-02-08, 02:41