Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> RE: [Vote] Merge branch-trunk-win to trunk

Bikas Saha 2013-02-27, 00:30
Chris Nauroth 2013-02-27, 06:05
Raja Aluri 2013-02-28, 19:17
Copy link to this message
Re: [Vote] Merge branch-trunk-win to trunk
+1 (non-binding)

A few of observations:

- Windows has actually been a supported platform for Hadoop since 0.1 .  Doug championed supporting windows then and we've continued to do it with varying vigor over time.  To my knowledge we've never made a decision to drop windows support.  The change here is improving our support and dropping the requirement of cigwin.  We had Nutch windows users on the list in 2006 and we've been supporting windows FS requirements since inception.

- A little pragmatism will go a long way.  As a community we've got to stay committed to keeping hadoop simple (so it does work on many platforms) and extending it to take advantage of key emerging OS/hardware features, such as containers, new FSs, virtualization, flash ...  We should all plan to let new features & optimizations emerge that don't work everywhere, if they are compelling and central to hadoop's mission of being THE best fabric for storing and processing big data.  

- A UI project like KDE has to deal with the MANY differences between windows and linux UI APIs.  Hadoop faces no such complex challenge and hence can be maintained from a single codeline IMO.  It is mostly abstracted from the OS APIs via Java and our design choices.  Where it is not we can continue to add plugable abstractions.

On Feb 28, 2013, at 6:01 PM, Matt Foley <[EMAIL PROTECTED]> wrote:

> +1 (binding)
> Apache is supposed to be about the community.  We have here a community of
> developers, who have actively and openly worked to add a major improvement
> to Hadoop: the ability to work cross-platform.  Furthermore, the size of
> the substantive part of the needed patch is only about 1500 lines, much
> smaller than quite a few other additions to Hadoop over the last few
> months.  We should welcome and support this change, and make sure that the
> code stays cross-platform going forward by extending our CI practices,
> especially pre-commit "test-patch", to also include Windows.
> As most of you know, my colleague Giri Kesavan (PMC member) helps maintain
> the Linux CI capability for Hadoop.  I've talked with him, and he and I are
> committing to getting test-patch implemented for Windows, so that along
> with the current automated "+1"s required to commit, we can add two more,
> for javac build in Windows and core unit tests in Windows.
> Members of the team implementing cross-platform compatibility, including
> Microsoft employees, have opened the discussion for providing hardware or
> VM resources to perform this additional CI testing.  I will assist them to
> work with the Apache Infra team and figure out how to make it happen.
> I understand there is some concern about the additional platform test.
> My going-in
> presumption, based on Java's intrinsic, pretty-good, cross-platform
> compatibility, is that patches to Hadoop will by default also have
> cross-platform compatibility, unless they are written in an explicitly
> platform-dependent way.  I also believe that in the vast majority of cases
> the cross-platform compatibility of Java will carry thru to Hadoop patches,
> without additional effort on the developer's part.
> Let's try it, and see what happens.  If we actually find a frequent
> difficulty, we'll change to engineer around it.  But I believe that, in the
> rare cases where a Windows-specific failure occurs, there will be a number
> of people (new, enthusiastic members of the community! :-) willing to help.
> If such help is not forthcoming, then we can discuss work-arounds, but
> like a previous poster, I am confident in the community.
> Regards,
> --Matt
> On Thu, Feb 28, 2013 at 12:21 PM, Chuan Liu <[EMAIL PROTECTED]> wrote:
>> +1 (non-binding)
>>> As someone also contributed to porting Hadoop to Windows, I think Java
>>> already provided a very good platform independent platform.
>>> For features that are not available in Java, we will try to provide our
>>> platform independent APIs that abstract OS tasks away.
Chuan Liu 2013-02-28, 20:21
Tsuyoshi OZAWA 2013-03-04, 02:09
Harsh J 2013-03-04, 04:50
Suresh Srinivas 2013-03-04, 18:09
Harsh J 2013-03-05, 01:42
Matt Foley 2013-03-05, 01:49
Steve Loughran 2013-03-06, 13:54
Konstantin Boudnik 2013-03-05, 00:35
Matt Foley 2013-03-04, 20:22
Konstantin Shvachko 2013-03-04, 22:30
Matt Foley 2013-03-04, 23:29
Konstantin Shvachko 2013-03-05, 01:00
Matt Foley 2013-03-05, 01:41
Konstantin Boudnik 2013-03-25, 20:17
Suresh Srinivas 2013-03-26, 00:09
Konstantin Boudnik 2013-03-26, 02:14
Konstantin Boudnik 2013-03-26, 05:49
Konstantin Shvachko 2013-03-25, 21:25
Andrew Purtell 2013-03-25, 21:53