Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> RE: [Vote] Merge branch-trunk-win to trunk

Bikas Saha 2013-02-27, 00:30
Chris Nauroth 2013-02-27, 06:05
Raja Aluri 2013-02-28, 19:17
Eric Baldeschwieler 2013-03-01, 04:47
Copy link to this message
RE: [Vote] Merge branch-trunk-win to trunk
+1 (non-binding)

As someone also contributed to porting Hadoop to Windows, I think Java already provided a very good platform independent platform.
For features that are not available in Java, we will try to provide our platform independent APIs that abstract OS tasks away.
Most features should have no difficulty running on Windows and Linux by using Java and those platform independent APIs.

For concerns raise on new features that may fail on Windows, I think we don't need to require passing on Windows a mandate at the moment. We can simply mark it unavailable to Windows and port it later if the feature is important.


-----Original Message-----
From: Chris Nauroth [mailto:[EMAIL PROTECTED]]
Sent: Thursday, February 28, 2013 11:51 AM
Subject: Re: [Vote] Merge branch-trunk-win to trunk

> Is there a jira for resolving the outstanding TODOs in the code base
> (similar to HDFS-2148)?  Looks like this merge doesn't introduce many
> which is great (just did a quick diff and grep).

I found 2 remaining TODOs introduced in the current merge patch.  One is in ContainerLaunch.java.  The container launch script was trying to set a CLASSPATH that exceeded the Windows maximum command line length.  The fix was to wrap the long classpath into an intermediate jar containing only a manifest file with a Class-Path entry.  (See YARN-316.)  Just to be conservative, we wrapped this logic in an if (Shell.WINDOWS) guard and marked a TODO to remove it later and use that approach on all platforms after additional testing.  I've tested this code path successfully on Mac too, but several people wanted additional testing and performance checks before removing the if (Shell.WINDOWS) guard.  That work is tracked in an existing jira: YARN-358.

The other TODO is for winutils to print more usage information and examples.  At this point, I think winutils is printing sufficient information, and we can just remove the TODO.  I just submitted a new jira to start that conversation: HADOOP-9348.

Thank you,
On Thu, Feb 28, 2013 at 11:29 AM, Robert Evans <[EMAIL PROTECTED]> wrote:

> My initial question was mostly intended to understand the desired new
> classification of Windows after the merge, and how we plan to maintain
> Windows support.  I am happy to hear that hardware for Jenkins will be
> provided.  I am also fine, at least initially, with us trying to treat
> Windows as a first class supported platform.  But I realize that there
> are a lot of people that do not have easy access to Windows for
> development/debugging, myself included. I also don't want to slow down
> the pace of development too much because of this.  It will cause some
> organizations that do not use or support Windows to be more likely to
> run software that has diverged from an official release.  It also has
> the potential to make the patch submission process even more
> difficult, which increases the likelihood of submitters abandoning
> patches.  However, the great thing about being in a community is we can change if we need to.
> I am +0 for the merge.  I am not a Windows expert so I don't feel
> comfortable giving it a true +1.
> --Bobby
> On 2/28/13 10:45 AM, "Chris Nauroth" <[EMAIL PROTECTED]> wrote:
> >I'd like to share a few anecdotes about developing cross-platform,
> >hopefully to address some of the concerns about adding overhead to
> >the development process.  By reviewing past cases of cross-platform Linux vs.
> >Windows bugs, we can get a sense for how the development process
> >could look in the future.
> >
> >HADOOP-9131: TestLocalFileSystem#testListStatusWithColons cannot run
> >on Windows.  As part of an earlier jira, HADOOP-8962, there was a new
> >test committed on trunk covering the case of a local file system
> >interaction on a file containing a ':'.  On Windows, ':' in a path
Tsuyoshi OZAWA 2013-03-04, 02:09
Harsh J 2013-03-04, 04:50
Suresh Srinivas 2013-03-04, 18:09
Harsh J 2013-03-05, 01:42
Matt Foley 2013-03-05, 01:49
Steve Loughran 2013-03-06, 13:54
Konstantin Boudnik 2013-03-05, 00:35
Matt Foley 2013-03-04, 20:22
Konstantin Shvachko 2013-03-04, 22:30
Matt Foley 2013-03-04, 23:29
Konstantin Shvachko 2013-03-05, 01:00
Matt Foley 2013-03-05, 01:41
Konstantin Boudnik 2013-03-25, 20:17
Suresh Srinivas 2013-03-26, 00:09
Konstantin Boudnik 2013-03-26, 02:14
Konstantin Boudnik 2013-03-26, 05:49
Konstantin Shvachko 2013-03-25, 21:25
Andrew Purtell 2013-03-25, 21:53
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB