Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # dev >> [Vote] Merge branch-trunk-win to trunk

Suresh Srinivas 2013-02-26, 22:55
Robert Evans 2013-02-27, 16:17
Eli Collins 2013-02-27, 19:56
Copy link to this message
Re: [Vote] Merge branch-trunk-win to trunk
Thanks for raising good questions.

Currently the merge patch passes all the tests on Linux, hence
the proposal for merging the patch to trunk. But as Bobby, Harsh
and Eli pointed out, before declaring support for Windows, we need the
discussion on the following:

1. Precommit and development process
Jenkins infrastructure for Windows build will be made available.
Giri and Microsoft contributors have volunteered to help make
this happen.

With that we need to decide how our precommit process looks.
My inclination is to wait for +1 from precommit builds on
both the platforms to ensure no issues are introduced.

2. Feature development impact
Some questions have been raised about would new features
need to be supported on both the platforms. Yes. I do not see a
reason why features cannot work on both the platforms, with
the exception of platform specific optimizations. This what Java
gives us.

3. Platform specific features/optimizations
As regards platform specific optimization, each platform can
evolve at its own pace and should not block progress of a
specific platform.

As indicated in my earlier email, there is a sizable number
of contributors to work on issues and support of Hadoop on Windows
platform. I am excited to see Hadoop reach the other large
part of server market.

Eli, as pointed out by you, the TODO items need to be addressed.
Also we realized we still need to add information on how to
build on Windows in BUILDING.txt. We will address this ASAP.
Giri and Matt have some expirience with this and should be able
to provide more information.

On Wed, Feb 27, 2013 at 11:56 AM, Eli Collins <[EMAIL PROTECTED]> wrote:

> Bobby raises some good questions.  A related one, since most current
> developers won't add Windows support for new features that are
> platform specific is it assumed that Windows development will either
> lag or will people actively work on keeping Windows up with the
> latest?  And vice versa in case Windows support is implemented first.
> Is there a jira for resolving the outstanding TODOs in the code base
> (similar to HDFS-2148)?  Looks like this merge doesn't introduce many
> which is great (just did a quick diff and grep).
> Thanks,
> Eli
> On Wed, Feb 27, 2013 at 8:17 AM, Robert Evans <[EMAIL PROTECTED]> wrote:
> > After this is merged in is Windows still going to be a second class
> > citizen but happens to work for more than just development or is it a
> > fully supported platform where if something breaks it can block a
> release?
> >  How do we as a community intend to keep Windows support from breaking?
> > We don't have any Jenkins slaves to be able to run nightly tests to
> > validate everything still compiles/runs.  This is not a blocker for me
> > because we often rely on individuals and groups to test Hadoop, but I do
> > think we need to have this discussion before we put it in.
> >
> > --Bobby
> >
> > On 2/26/13 4:55 PM, "Suresh Srinivas" <[EMAIL PROTECTED]> wrote:
> >
> >>I had posted heads up about merging branch-trunk-win to trunk on Feb 8th.
> >>I
> >>am happy to announce that we are ready for the merge.
> >>
> >>Here is a brief recap on the highlights of the work done:
> >>- Command-line scripts for the Hadoop surface area
> >>- Mapping the HDFS permissions model to Windows
> >>- Abstracted and reconciled mismatches around differences in Path
> >>semantics
> >>in Java and Windows
> >>- Native Task Controller for Windows
> >>- Implementation of a Block Placement Policy to support cloud
> >>environments,
> >>more specifically Azure.
> >>- Implementation of Hadoop native libraries for Windows (compression
> >>codecs, native I/O)
> >>- Several reliability issues, including race-conditions, intermittent
> test
> >>failures, resource leaks.
> >>- Several new unit test cases written for the above changes
> >>
> >>Please find the details of the work in CHANGES.branch-trunk-win.txt -
> >>Common changes<http://bit.ly/Xe7Ynv>, HDFS changes<http://bit.ly/13QOSo9
> >,

Todd Lipcon 2013-02-27, 23:43
Ivan Mitic 2013-02-28, 02:32
John Gordon 2013-02-28, 11:25
Chris Nauroth 2013-02-28, 16:45
Robert Evans 2013-02-28, 19:29
Chris Nauroth 2013-02-28, 19:50
Harsh J 2013-02-27, 19:20
Konstantin Shvachko 2013-03-02, 03:14
Konstantin Shvachko 2013-03-01, 20:18
Chris Douglas 2013-03-01, 21:28
Konstantin Shvachko 2013-03-01, 21:57
Chris Douglas 2013-03-02, 00:57
Konstantin Boudnik 2013-03-01, 23:47
Suresh Srinivas 2013-03-02, 01:55
Konstantin Boudnik 2013-03-02, 03:33
Matt Foley 2013-03-02, 20:32
Konstantin Shvachko 2013-03-03, 03:32
Matt Foley 2013-03-03, 19:08
Konstantin Shvachko 2013-03-03, 20:16
Bikas Saha 2013-03-01, 17:55
Konstantin Boudnik 2013-03-01, 01:04
Ramya Sunil 2013-03-01, 00:59
Suresh Srinivas 2013-03-06, 19:50
Andrew Purtell 2013-03-25, 20:36
Suresh Srinivas 2013-03-26, 00:08
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB