Milind.Bhandarkar@...
2011-10-07, 22:33
Todd Lipcon
2011-10-08, 09:00
Ted Dunning
2011-10-08, 18:19
Brian Bockelman
2011-10-10, 00:49
Ted Dunning
2011-10-10, 00:55
Konstantin Boudnik
2011-10-10, 03:35
Scott Carey
2011-10-10, 04:22
Scott Carey
2011-10-10, 04:34
Steve Loughran
2011-10-10, 10:45
Scott Carey
2011-10-10, 18:57
Steve Loughran
2011-10-11, 11:18
Scott Carey
2011-11-03, 21:35
Scott Carey
2011-10-10, 04:33
Milind.Bhandarkar@...
2011-10-10, 21:32
Steve Loughran
2011-10-11, 11:20
Milind.Bhandarkar@...
2011-10-11, 17:31
Arun C Murthy
2011-10-11, 17:32
Milind.Bhandarkar@...
2011-10-11, 18:57
Arun C Murthy
2011-10-11, 22:16
-
Java Versions and HadoopMilind.Bhandarkar@... 2011-10-07, 22:33
Hi Folks,
While I have seen the wiki on which java versions to use currently to run
Hadoop, I have not seen any discussion about the roadmap of java version
compatibility with future hadoop versions.
Recently, Oracle retired the "Operating System Distributor License for
Java" (DLJ) [http://robilad.livejournal.com/90792.html,
http://jdk-distros.java.net/] and Linux vendors have started making
OpenJDK (6/7) as the default java version bundled with their OSs
[http://www.java7developer.com/blog/?p=361]. Also, all future Java SE
updates will be delivered through OpenJDK updates project.
I see that OpenJDK6 (6b20pre) cannot be used to compile hadoop trunk. Has
anyone tried OpenJDK7 ?
Additionally, I have a few small projects in mind which can really make
use of the new (esp I/O) features of Java 7.
What, if any, timeline do hadoop developers have in mind to make Java 7 as
required (and tested with OpenJDK 7) ?
Thanks,
- milind
---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)
While I have seen the wiki on which java versions to use currently to run
Hadoop, I have not seen any discussion about the roadmap of java version
compatibility with future hadoop versions.
Recently, Oracle retired the "Operating System Distributor License for
Java" (DLJ) [http://robilad.livejournal.com/90792.html,
http://jdk-distros.java.net/] and Linux vendors have started making
OpenJDK (6/7) as the default java version bundled with their OSs
[http://www.java7developer.com/blog/?p=361]. Also, all future Java SE
updates will be delivered through OpenJDK updates project.
I see that OpenJDK6 (6b20pre) cannot be used to compile hadoop trunk. Has
anyone tried OpenJDK7 ?
Additionally, I have a few small projects in mind which can really make
use of the new (esp I/O) features of Java 7.
What, if any, timeline do hadoop developers have in mind to make Java 7 as
required (and tested with OpenJDK 7) ?
Thanks,
- milind
---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)
+
Milind.Bhandarkar@... 2011-10-07, 22:33
-
Re: Java Versions and HadoopTodd Lipcon 2011-10-08, 09:00
I think requiring Java 7 is years off... I think most people have
doubts as to Java 7's stability until it's been adopted by a majority
of applications, and the new features aren't compelling enough to jump
ship, IMO.
-Todd
On Fri, Oct 7, 2011 at 3:33 PM, <[EMAIL PROTECTED]> wrote:
> Hi Folks,
>
> While I have seen the wiki on which java versions to use currently to run
> Hadoop, I have not seen any discussion about the roadmap of java version
> compatibility with future hadoop versions.
>
> Recently, Oracle retired the "Operating System Distributor License for
> Java" (DLJ) [http://robilad.livejournal.com/90792.html,
> http://jdk-distros.java.net/] and Linux vendors have started making
> OpenJDK (6/7) as the default java version bundled with their OSs
> [http://www.java7developer.com/blog/?p=361]. Also, all future Java SE
> updates will be delivered through OpenJDK updates project.
>
> I see that OpenJDK6 (6b20pre) cannot be used to compile hadoop trunk. Has
> anyone tried OpenJDK7 ?
>
> Additionally, I have a few small projects in mind which can really make
> use of the new (esp I/O) features of Java 7.
>
> What, if any, timeline do hadoop developers have in mind to make Java 7 as
> required (and tested with OpenJDK 7) ?
>
> Thanks,
>
> - milind
>
> ---
> Milind Bhandarkar
> Greenplum Labs, EMC
> (Disclaimer: Opinions expressed in this email are those of the author, and
> do not necessarily represent the views of any organization, past or
> present, the author might be affiliated with.)
>
>
--
Todd Lipcon
Software Engineer, Cloudera
doubts as to Java 7's stability until it's been adopted by a majority
of applications, and the new features aren't compelling enough to jump
ship, IMO.
-Todd
On Fri, Oct 7, 2011 at 3:33 PM, <[EMAIL PROTECTED]> wrote:
> Hi Folks,
>
> While I have seen the wiki on which java versions to use currently to run
> Hadoop, I have not seen any discussion about the roadmap of java version
> compatibility with future hadoop versions.
>
> Recently, Oracle retired the "Operating System Distributor License for
> Java" (DLJ) [http://robilad.livejournal.com/90792.html,
> http://jdk-distros.java.net/] and Linux vendors have started making
> OpenJDK (6/7) as the default java version bundled with their OSs
> [http://www.java7developer.com/blog/?p=361]. Also, all future Java SE
> updates will be delivered through OpenJDK updates project.
>
> I see that OpenJDK6 (6b20pre) cannot be used to compile hadoop trunk. Has
> anyone tried OpenJDK7 ?
>
> Additionally, I have a few small projects in mind which can really make
> use of the new (esp I/O) features of Java 7.
>
> What, if any, timeline do hadoop developers have in mind to make Java 7 as
> required (and tested with OpenJDK 7) ?
>
> Thanks,
>
> - milind
>
> ---
> Milind Bhandarkar
> Greenplum Labs, EMC
> (Disclaimer: Opinions expressed in this email are those of the author, and
> do not necessarily represent the views of any organization, past or
> present, the author might be affiliated with.)
>
>
--
Todd Lipcon
Software Engineer, Cloudera
+
Todd Lipcon 2011-10-08, 09:00
-
Re: Java Versions and HadoopTed Dunning 2011-10-08, 18:19
I hate to sound like the folks who only recently stopped using 1.4, but I am
afraid that Todd is right on here.
The folks who are desperate for new features are being siphoned off by Scala
and Clojure which is leaving a core of recalcitrant termagants like me. I
think that it is going to take something as major as the EOL of Java 6 to
get big projects to allow code that requires Java 7.
On Sat, Oct 8, 2011 at 2:00 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> I think requiring Java 7 is years off... I think most people have
> doubts as to Java 7's stability until it's been adopted by a majority
> of applications, and the new features aren't compelling enough to jump
> ship, IMO.
>
> -Todd
>
> On Fri, Oct 7, 2011 at 3:33 PM, <[EMAIL PROTECTED]> wrote:
> > Hi Folks,
> >
> > While I have seen the wiki on which java versions to use currently to run
> > Hadoop, I have not seen any discussion about the roadmap of java version
> > compatibility with future hadoop versions.
> >
> > Recently, Oracle retired the "Operating System Distributor License for
> > Java" (DLJ) [http://robilad.livejournal.com/90792.html,
> > http://jdk-distros.java.net/] and Linux vendors have started making
> > OpenJDK (6/7) as the default java version bundled with their OSs
> > [http://www.java7developer.com/blog/?p=361]. Also, all future Java SE
> > updates will be delivered through OpenJDK updates project.
> >
> > I see that OpenJDK6 (6b20pre) cannot be used to compile hadoop trunk. Has
> > anyone tried OpenJDK7 ?
> >
> > Additionally, I have a few small projects in mind which can really make
> > use of the new (esp I/O) features of Java 7.
> >
> > What, if any, timeline do hadoop developers have in mind to make Java 7
> as
> > required (and tested with OpenJDK 7) ?
> >
> > Thanks,
> >
> > - milind
> >
> > ---
> > Milind Bhandarkar
> > Greenplum Labs, EMC
> > (Disclaimer: Opinions expressed in this email are those of the author,
> and
> > do not necessarily represent the views of any organization, past or
> > present, the author might be affiliated with.)
> >
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
afraid that Todd is right on here.
The folks who are desperate for new features are being siphoned off by Scala
and Clojure which is leaving a core of recalcitrant termagants like me. I
think that it is going to take something as major as the EOL of Java 6 to
get big projects to allow code that requires Java 7.
On Sat, Oct 8, 2011 at 2:00 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> I think requiring Java 7 is years off... I think most people have
> doubts as to Java 7's stability until it's been adopted by a majority
> of applications, and the new features aren't compelling enough to jump
> ship, IMO.
>
> -Todd
>
> On Fri, Oct 7, 2011 at 3:33 PM, <[EMAIL PROTECTED]> wrote:
> > Hi Folks,
> >
> > While I have seen the wiki on which java versions to use currently to run
> > Hadoop, I have not seen any discussion about the roadmap of java version
> > compatibility with future hadoop versions.
> >
> > Recently, Oracle retired the "Operating System Distributor License for
> > Java" (DLJ) [http://robilad.livejournal.com/90792.html,
> > http://jdk-distros.java.net/] and Linux vendors have started making
> > OpenJDK (6/7) as the default java version bundled with their OSs
> > [http://www.java7developer.com/blog/?p=361]. Also, all future Java SE
> > updates will be delivered through OpenJDK updates project.
> >
> > I see that OpenJDK6 (6b20pre) cannot be used to compile hadoop trunk. Has
> > anyone tried OpenJDK7 ?
> >
> > Additionally, I have a few small projects in mind which can really make
> > use of the new (esp I/O) features of Java 7.
> >
> > What, if any, timeline do hadoop developers have in mind to make Java 7
> as
> > required (and tested with OpenJDK 7) ?
> >
> > Thanks,
> >
> > - milind
> >
> > ---
> > Milind Bhandarkar
> > Greenplum Labs, EMC
> > (Disclaimer: Opinions expressed in this email are those of the author,
> and
> > do not necessarily represent the views of any organization, past or
> > present, the author might be affiliated with.)
> >
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
+
Ted Dunning 2011-10-08, 18:19
-
Re: Java Versions and HadoopBrian Bockelman 2011-10-10, 00:49
You know, I was looking around for Java EOL information a few weeks ago and found this:
http://www.oracle.com/technetwork/java/eol-135779.html
I might not be reading closely enough, but that says Java 6 is EOL in 9 months. In terms of major releases and getting things stable… that's just around the corner.
Brian
On Oct 8, 2011, at 1:19 PM, Ted Dunning wrote:
> I hate to sound like the folks who only recently stopped using 1.4, but I am
> afraid that Todd is right on here.
>
> The folks who are desperate for new features are being siphoned off by Scala
> and Clojure which is leaving a core of recalcitrant termagants like me. I
> think that it is going to take something as major as the EOL of Java 6 to
> get big projects to allow code that requires Java 7.
>
> On Sat, Oct 8, 2011 at 2:00 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>
>> I think requiring Java 7 is years off... I think most people have
>> doubts as to Java 7's stability until it's been adopted by a majority
>> of applications, and the new features aren't compelling enough to jump
>> ship, IMO.
>>
>> -Todd
>>
>> On Fri, Oct 7, 2011 at 3:33 PM, <[EMAIL PROTECTED]> wrote:
>>> Hi Folks,
>>>
>>> While I have seen the wiki on which java versions to use currently to run
>>> Hadoop, I have not seen any discussion about the roadmap of java version
>>> compatibility with future hadoop versions.
>>>
>>> Recently, Oracle retired the "Operating System Distributor License for
>>> Java" (DLJ) [http://robilad.livejournal.com/90792.html,
>>> http://jdk-distros.java.net/] and Linux vendors have started making
>>> OpenJDK (6/7) as the default java version bundled with their OSs
>>> [http://www.java7developer.com/blog/?p=361]. Also, all future Java SE
>>> updates will be delivered through OpenJDK updates project.
>>>
>>> I see that OpenJDK6 (6b20pre) cannot be used to compile hadoop trunk. Has
>>> anyone tried OpenJDK7 ?
>>>
>>> Additionally, I have a few small projects in mind which can really make
>>> use of the new (esp I/O) features of Java 7.
>>>
>>> What, if any, timeline do hadoop developers have in mind to make Java 7
>> as
>>> required (and tested with OpenJDK 7) ?
>>>
>>> Thanks,
>>>
>>> - milind
>>>
>>> ---
>>> Milind Bhandarkar
>>> Greenplum Labs, EMC
>>> (Disclaimer: Opinions expressed in this email are those of the author,
>> and
>>> do not necessarily represent the views of any organization, past or
>>> present, the author might be affiliated with.)
>>>
>>>
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
http://www.oracle.com/technetwork/java/eol-135779.html
I might not be reading closely enough, but that says Java 6 is EOL in 9 months. In terms of major releases and getting things stable… that's just around the corner.
Brian
On Oct 8, 2011, at 1:19 PM, Ted Dunning wrote:
> I hate to sound like the folks who only recently stopped using 1.4, but I am
> afraid that Todd is right on here.
>
> The folks who are desperate for new features are being siphoned off by Scala
> and Clojure which is leaving a core of recalcitrant termagants like me. I
> think that it is going to take something as major as the EOL of Java 6 to
> get big projects to allow code that requires Java 7.
>
> On Sat, Oct 8, 2011 at 2:00 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>
>> I think requiring Java 7 is years off... I think most people have
>> doubts as to Java 7's stability until it's been adopted by a majority
>> of applications, and the new features aren't compelling enough to jump
>> ship, IMO.
>>
>> -Todd
>>
>> On Fri, Oct 7, 2011 at 3:33 PM, <[EMAIL PROTECTED]> wrote:
>>> Hi Folks,
>>>
>>> While I have seen the wiki on which java versions to use currently to run
>>> Hadoop, I have not seen any discussion about the roadmap of java version
>>> compatibility with future hadoop versions.
>>>
>>> Recently, Oracle retired the "Operating System Distributor License for
>>> Java" (DLJ) [http://robilad.livejournal.com/90792.html,
>>> http://jdk-distros.java.net/] and Linux vendors have started making
>>> OpenJDK (6/7) as the default java version bundled with their OSs
>>> [http://www.java7developer.com/blog/?p=361]. Also, all future Java SE
>>> updates will be delivered through OpenJDK updates project.
>>>
>>> I see that OpenJDK6 (6b20pre) cannot be used to compile hadoop trunk. Has
>>> anyone tried OpenJDK7 ?
>>>
>>> Additionally, I have a few small projects in mind which can really make
>>> use of the new (esp I/O) features of Java 7.
>>>
>>> What, if any, timeline do hadoop developers have in mind to make Java 7
>> as
>>> required (and tested with OpenJDK 7) ?
>>>
>>> Thanks,
>>>
>>> - milind
>>>
>>> ---
>>> Milind Bhandarkar
>>> Greenplum Labs, EMC
>>> (Disclaimer: Opinions expressed in this email are those of the author,
>> and
>>> do not necessarily represent the views of any organization, past or
>>> present, the author might be affiliated with.)
>>>
>>>
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
+
Brian Bockelman 2011-10-10, 00:49
-
Re: Java Versions and HadoopTed Dunning 2011-10-10, 00:55
On Sun, Oct 9, 2011 at 5:49 PM, Brian Bockelman <[EMAIL PROTECTED]>wrote:
> You know, I was looking around for Java EOL information a few weeks ago and
> found this:
>
> http://www.oracle.com/technetwork/java/eol-135779.html
>
> I might not be reading closely enough, but that says Java 6 is EOL in 9
> months. In terms of major releases and getting things stable… that's just
> around the corner.
>
> Brian
>
> On Oct 8, 2011, at 1:19 PM, Ted Dunning wrote:
>
> > I hate to sound like the folks who only recently stopped using 1.4, but I
> am
> > afraid that Todd is right on here.
> >
> > The folks who are desperate for new features are being siphoned off by
> Scala
> > and Clojure which is leaving a core of recalcitrant termagants like me.
> I
> > think that it is going to take something as major as the EOL of Java 6 to
> > get big projects to allow code that requires Java 7.
>
Classic. The conversations about requiring Java 6 are just barely dying
down in time for EOL of Java 6. Maybe we ought start talking about
requiring java 9 NOW so that we can finish in time for it to end.
Yowza.
> You know, I was looking around for Java EOL information a few weeks ago and
> found this:
>
> http://www.oracle.com/technetwork/java/eol-135779.html
>
> I might not be reading closely enough, but that says Java 6 is EOL in 9
> months. In terms of major releases and getting things stable… that's just
> around the corner.
>
> Brian
>
> On Oct 8, 2011, at 1:19 PM, Ted Dunning wrote:
>
> > I hate to sound like the folks who only recently stopped using 1.4, but I
> am
> > afraid that Todd is right on here.
> >
> > The folks who are desperate for new features are being siphoned off by
> Scala
> > and Clojure which is leaving a core of recalcitrant termagants like me.
> I
> > think that it is going to take something as major as the EOL of Java 6 to
> > get big projects to allow code that requires Java 7.
>
Classic. The conversations about requiring Java 6 are just barely dying
down in time for EOL of Java 6. Maybe we ought start talking about
requiring java 9 NOW so that we can finish in time for it to end.
Yowza.
+
Ted Dunning 2011-10-10, 00:55
-
Re: Java Versions and HadoopKonstantin Boudnik 2011-10-10, 03:35
On Sun, Oct 09, 2011 at 07:49PM, Brian Bockelman wrote:
> You know, I was looking around for Java EOL information a few weeks ago and found this:
>
> http://www.oracle.com/technetwork/java/eol-135779.html
>
> I might not be reading closely enough, but that says Java 6 is EOL in 9
> months. In terms of major releases and getting things stable… that's just
> around the corner.
there's many ways to "eol" a particular java version. Stopping updates of
HotSpot on JDK6 might (and most likely will be( one of them.
Enjoy while you can
Cos
> Brian
>
> On Oct 8, 2011, at 1:19 PM, Ted Dunning wrote:
>
> > I hate to sound like the folks who only recently stopped using 1.4, but I am
> > afraid that Todd is right on here.
> >
> > The folks who are desperate for new features are being siphoned off by Scala
> > and Clojure which is leaving a core of recalcitrant termagants like me. I
> > think that it is going to take something as major as the EOL of Java 6 to
> > get big projects to allow code that requires Java 7.
> >
> > On Sat, Oct 8, 2011 at 2:00 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> >
> >> I think requiring Java 7 is years off... I think most people have
> >> doubts as to Java 7's stability until it's been adopted by a majority
> >> of applications, and the new features aren't compelling enough to jump
> >> ship, IMO.
> >>
> >> -Todd
> >>
> >> On Fri, Oct 7, 2011 at 3:33 PM, <[EMAIL PROTECTED]> wrote:
> >>> Hi Folks,
> >>>
> >>> While I have seen the wiki on which java versions to use currently to run
> >>> Hadoop, I have not seen any discussion about the roadmap of java version
> >>> compatibility with future hadoop versions.
> >>>
> >>> Recently, Oracle retired the "Operating System Distributor License for
> >>> Java" (DLJ) [http://robilad.livejournal.com/90792.html,
> >>> http://jdk-distros.java.net/] and Linux vendors have started making
> >>> OpenJDK (6/7) as the default java version bundled with their OSs
> >>> [http://www.java7developer.com/blog/?p=361]. Also, all future Java SE
> >>> updates will be delivered through OpenJDK updates project.
> >>>
> >>> I see that OpenJDK6 (6b20pre) cannot be used to compile hadoop trunk. Has
> >>> anyone tried OpenJDK7 ?
> >>>
> >>> Additionally, I have a few small projects in mind which can really make
> >>> use of the new (esp I/O) features of Java 7.
> >>>
> >>> What, if any, timeline do hadoop developers have in mind to make Java 7
> >> as
> >>> required (and tested with OpenJDK 7) ?
> >>>
> >>> Thanks,
> >>>
> >>> - milind
> >>>
> >>> ---
> >>> Milind Bhandarkar
> >>> Greenplum Labs, EMC
> >>> (Disclaimer: Opinions expressed in this email are those of the author,
> >> and
> >>> do not necessarily represent the views of any organization, past or
> >>> present, the author might be affiliated with.)
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >> Todd Lipcon
> >> Software Engineer, Cloudera
> >>
>
> You know, I was looking around for Java EOL information a few weeks ago and found this:
>
> http://www.oracle.com/technetwork/java/eol-135779.html
>
> I might not be reading closely enough, but that says Java 6 is EOL in 9
> months. In terms of major releases and getting things stable… that's just
> around the corner.
there's many ways to "eol" a particular java version. Stopping updates of
HotSpot on JDK6 might (and most likely will be( one of them.
Enjoy while you can
Cos
> Brian
>
> On Oct 8, 2011, at 1:19 PM, Ted Dunning wrote:
>
> > I hate to sound like the folks who only recently stopped using 1.4, but I am
> > afraid that Todd is right on here.
> >
> > The folks who are desperate for new features are being siphoned off by Scala
> > and Clojure which is leaving a core of recalcitrant termagants like me. I
> > think that it is going to take something as major as the EOL of Java 6 to
> > get big projects to allow code that requires Java 7.
> >
> > On Sat, Oct 8, 2011 at 2:00 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> >
> >> I think requiring Java 7 is years off... I think most people have
> >> doubts as to Java 7's stability until it's been adopted by a majority
> >> of applications, and the new features aren't compelling enough to jump
> >> ship, IMO.
> >>
> >> -Todd
> >>
> >> On Fri, Oct 7, 2011 at 3:33 PM, <[EMAIL PROTECTED]> wrote:
> >>> Hi Folks,
> >>>
> >>> While I have seen the wiki on which java versions to use currently to run
> >>> Hadoop, I have not seen any discussion about the roadmap of java version
> >>> compatibility with future hadoop versions.
> >>>
> >>> Recently, Oracle retired the "Operating System Distributor License for
> >>> Java" (DLJ) [http://robilad.livejournal.com/90792.html,
> >>> http://jdk-distros.java.net/] and Linux vendors have started making
> >>> OpenJDK (6/7) as the default java version bundled with their OSs
> >>> [http://www.java7developer.com/blog/?p=361]. Also, all future Java SE
> >>> updates will be delivered through OpenJDK updates project.
> >>>
> >>> I see that OpenJDK6 (6b20pre) cannot be used to compile hadoop trunk. Has
> >>> anyone tried OpenJDK7 ?
> >>>
> >>> Additionally, I have a few small projects in mind which can really make
> >>> use of the new (esp I/O) features of Java 7.
> >>>
> >>> What, if any, timeline do hadoop developers have in mind to make Java 7
> >> as
> >>> required (and tested with OpenJDK 7) ?
> >>>
> >>> Thanks,
> >>>
> >>> - milind
> >>>
> >>> ---
> >>> Milind Bhandarkar
> >>> Greenplum Labs, EMC
> >>> (Disclaimer: Opinions expressed in this email are those of the author,
> >> and
> >>> do not necessarily represent the views of any organization, past or
> >>> present, the author might be affiliated with.)
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >> Todd Lipcon
> >> Software Engineer, Cloudera
> >>
>
+
Konstantin Boudnik 2011-10-10, 03:35
-
Re: Java Versions and HadoopScott Carey 2011-10-10, 04:22
On 10/8/11 11:19 AM, "Ted Dunning" <[EMAIL PROTECTED]> wrote:
>I hate to sound like the folks who only recently stopped using 1.4, but I
>am
>afraid that Todd is right on here.
>
>The folks who are desperate for new features are being siphoned off by
>Scala
>and Clojure which is leaving a core of recalcitrant termagants like me. I
>think that it is going to take something as major as the EOL of Java 6 to
>get big projects to allow code that requires Java 7.
That was not the original user's question. The question was not "When
will Hadoop REQUIRE Java 7" but "When will Hadoop SUPPORT using a Java 7
JRE".
The JVM in Java 7 is near identical to Java 6. Sun/Oracle started
updating the JVM for JRE 6 to lag slightly (6-12 weeks) behind the (then
in development) Java 7 about 3 years ago. After the next Java 6 update,
they will essentially be identical other than a handful of default
parameters and the presence of the invokevirtual JVM infrastructure.
The JDK has changed in many ways however -- there may be more risk there
over the next year than differences in the JVM.
Next, now that Hadoop is using Maven, it would be trivial to create a new
(optional) module that required Java 7 and provided useful things, like an
implementation of the new Java 7 File System API for HDFS. Hadoop would
not require a Java 7 JRE, but a user that was using a Java 7 JRE would
have some extra API's available.
Java 8 will be a bigger beast that will likely result in more demands for
Hadoop to adopt somewhat rapidly.
Project Lambda will result in many requests for API enhancements around
Map/Reduce.
Project Jigsaw may finally rescue Hadoop from itself and allow proper
isolation of user code from framework code.
>
>On Sat, Oct 8, 2011 at 2:00 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>
>> I think requiring Java 7 is years off... I think most people have
>> doubts as to Java 7's stability until it's been adopted by a majority
>> of applications, and the new features aren't compelling enough to jump
>> ship, IMO.
>>
>> -Todd
>>
>> On Fri, Oct 7, 2011 at 3:33 PM, <[EMAIL PROTECTED]> wrote:
>> > Hi Folks,
>> >
>> > While I have seen the wiki on which java versions to use currently to
>>run
>> > Hadoop, I have not seen any discussion about the roadmap of java
>>version
>> > compatibility with future hadoop versions.
>> >
>> > Recently, Oracle retired the "Operating System Distributor License for
>> > Java" (DLJ) [http://robilad.livejournal.com/90792.html,
>> > http://jdk-distros.java.net/] and Linux vendors have started making
>> > OpenJDK (6/7) as the default java version bundled with their OSs
>> > [http://www.java7developer.com/blog/?p=361]. Also, all future Java SE
>> > updates will be delivered through OpenJDK updates project.
>> >
>> > I see that OpenJDK6 (6b20pre) cannot be used to compile hadoop trunk.
>>Has
>> > anyone tried OpenJDK7 ?
>> >
>> > Additionally, I have a few small projects in mind which can really
>>make
>> > use of the new (esp I/O) features of Java 7.
>> >
>> > What, if any, timeline do hadoop developers have in mind to make Java
>>7
>> as
>> > required (and tested with OpenJDK 7) ?
>> >
>> > Thanks,
>> >
>> > - milind
>> >
>> > ---
>> > Milind Bhandarkar
>> > Greenplum Labs, EMC
>> > (Disclaimer: Opinions expressed in this email are those of the author,
>> and
>> > do not necessarily represent the views of any organization, past or
>> > present, the author might be affiliated with.)
>> >
>> >
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
+
Scott Carey 2011-10-10, 04:22
-
Re: Java Versions and HadoopScott Carey 2011-10-10, 04:34
On 10/9/11 9:22 PM, "Scott Carey" <[EMAIL PROTECTED]> wrote:
>
>
>On 10/8/11 11:19 AM, "Ted Dunning" <[EMAIL PROTECTED]> wrote:
>
>>I hate to sound like the folks who only recently stopped using 1.4, but I
>>am
>>afraid that Todd is right on here.
>>
>>The folks who are desperate for new features are being siphoned off by
>>Scala
>>and Clojure which is leaving a core of recalcitrant termagants like me.
>>I
>>think that it is going to take something as major as the EOL of Java 6 to
>>get big projects to allow code that requires Java 7.
>
>
>That was not the original user's question. The question was not "When
>will Hadoop REQUIRE Java 7" but "When will Hadoop SUPPORT using a Java 7
>JRE".
Apologies, The quotes were not from the original question -- they were how
I interpreted the question.
>
>
>The JVM in Java 7 is near identical to Java 6. Sun/Oracle started
>updating the JVM for JRE 6 to lag slightly (6-12 weeks) behind the (then
>in development) Java 7 about 3 years ago. After the next Java 6 update,
>they will essentially be identical other than a handful of default
>parameters and the presence of the invokevirtual JVM infrastructure.
>The JDK has changed in many ways however -- there may be more risk there
>over the next year than differences in the JVM.
>
>Next, now that Hadoop is using Maven, it would be trivial to create a new
>(optional) module that required Java 7 and provided useful things, like an
>implementation of the new Java 7 File System API for HDFS. Hadoop would
>not require a Java 7 JRE, but a user that was using a Java 7 JRE would
>have some extra API's available.
>
>Java 8 will be a bigger beast that will likely result in more demands for
>Hadoop to adopt somewhat rapidly.
>Project Lambda will result in many requests for API enhancements around
>Map/Reduce.
>Project Jigsaw may finally rescue Hadoop from itself and allow proper
>isolation of user code from framework code.
>
>
>>
>>On Sat, Oct 8, 2011 at 2:00 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>>
>>> I think requiring Java 7 is years off... I think most people have
>>> doubts as to Java 7's stability until it's been adopted by a majority
>>> of applications, and the new features aren't compelling enough to jump
>>> ship, IMO.
>>>
>>> -Todd
>>>
>>> On Fri, Oct 7, 2011 at 3:33 PM, <[EMAIL PROTECTED]> wrote:
>>> > Hi Folks,
>>> >
>>> > While I have seen the wiki on which java versions to use currently to
>>>run
>>> > Hadoop, I have not seen any discussion about the roadmap of java
>>>version
>>> > compatibility with future hadoop versions.
>>> >
>>> > Recently, Oracle retired the "Operating System Distributor License
>>>for
>>> > Java" (DLJ) [http://robilad.livejournal.com/90792.html,
>>> > http://jdk-distros.java.net/] and Linux vendors have started making
>>> > OpenJDK (6/7) as the default java version bundled with their OSs
>>> > [http://www.java7developer.com/blog/?p=361]. Also, all future Java SE
>>> > updates will be delivered through OpenJDK updates project.
>>> >
>>> > I see that OpenJDK6 (6b20pre) cannot be used to compile hadoop trunk.
>>>Has
>>> > anyone tried OpenJDK7 ?
>>> >
>>> > Additionally, I have a few small projects in mind which can really
>>>make
>>> > use of the new (esp I/O) features of Java 7.
>>> >
>>> > What, if any, timeline do hadoop developers have in mind to make Java
>>>7
>>> as
>>> > required (and tested with OpenJDK 7) ?
>>> >
>>> > Thanks,
>>> >
>>> > - milind
>>> >
>>> > ---
>>> > Milind Bhandarkar
>>> > Greenplum Labs, EMC
>>> > (Disclaimer: Opinions expressed in this email are those of the
>>>author,
>>> and
>>> > do not necessarily represent the views of any organization, past or
>>> > present, the author might be affiliated with.)
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Todd Lipcon
>>> Software Engineer, Cloudera
>>>
>
+
Scott Carey 2011-10-10, 04:34
-
Re: Java Versions and HadoopSteve Loughran 2011-10-10, 10:45
On 08/10/11 19:19, Ted Dunning wrote:
> I hate to sound like the folks who only recently stopped using 1.4, but I am
> afraid that Todd is right on here.
>
> The folks who are desperate for new features are being siphoned off by Scala
> and Clojure which is leaving a core of recalcitrant termagants like me.
Language wise, you are right. I actually use Groovy under Java6 for a
lot of work these days, as it offers many of the features of the Java7
language today yet runs on Java6, and lets you subclass java classesand
vice versa.
Here, for example, is a MiniDFSCluster for Hadoop 0.20.20x that fixes
its need for a JVM property to set the dest dir for data (fixed in
trunk, BTW)
http://bit.ly/nEUGKY
> I
> think that it is going to take something as major as the EOL of Java 6 to
> get big projects to allow code that requires Java 7.
JVM-wise, people have problems with the hotspot compiler, which means
"don't use Hadoop with it"
http://www.lucidimagination.com/blog/2011/07/28/dont-use-java-7-for-anything/
I don't know if the updates fix this. Eventually they shall.
For people to be confident that it works someone needs to bring up a
large cluster and run it on Hadoop long enough -with a more complex
workload than terasort -to find the bugs, the race conditions, the
problems that only surface once you serve up 2+PB of data.
I hope someone volunteers for this. As Oracle have announced a
Hadoop-based system, they could be the people to step up here, or they
could pay someone else to do the work.
In an ideal world, the Hadoop stack would be part of the test stack for
a JVM release.
In the meantime, even if Oracle say Java6 is EOL, if people pay money to
keep it alive -and they will have to in any project you don't want to
have to requalify for java7- then it may keep going for longer, except
the updates won't be so widely available.
I don't know if anyone with a Hadoop cluster has a support contract with
Oracle for it. I know Yahoo! did, but I don't know it's current state.
> I hate to sound like the folks who only recently stopped using 1.4, but I am
> afraid that Todd is right on here.
>
> The folks who are desperate for new features are being siphoned off by Scala
> and Clojure which is leaving a core of recalcitrant termagants like me.
Language wise, you are right. I actually use Groovy under Java6 for a
lot of work these days, as it offers many of the features of the Java7
language today yet runs on Java6, and lets you subclass java classesand
vice versa.
Here, for example, is a MiniDFSCluster for Hadoop 0.20.20x that fixes
its need for a JVM property to set the dest dir for data (fixed in
trunk, BTW)
http://bit.ly/nEUGKY
> I
> think that it is going to take something as major as the EOL of Java 6 to
> get big projects to allow code that requires Java 7.
JVM-wise, people have problems with the hotspot compiler, which means
"don't use Hadoop with it"
http://www.lucidimagination.com/blog/2011/07/28/dont-use-java-7-for-anything/
I don't know if the updates fix this. Eventually they shall.
For people to be confident that it works someone needs to bring up a
large cluster and run it on Hadoop long enough -with a more complex
workload than terasort -to find the bugs, the race conditions, the
problems that only surface once you serve up 2+PB of data.
I hope someone volunteers for this. As Oracle have announced a
Hadoop-based system, they could be the people to step up here, or they
could pay someone else to do the work.
In an ideal world, the Hadoop stack would be part of the test stack for
a JVM release.
In the meantime, even if Oracle say Java6 is EOL, if people pay money to
keep it alive -and they will have to in any project you don't want to
have to requalify for java7- then it may keep going for longer, except
the updates won't be so widely available.
I don't know if anyone with a Hadoop cluster has a support contract with
Oracle for it. I know Yahoo! did, but I don't know it's current state.
+
Steve Loughran 2011-10-10, 10:45
-
Re: Java Versions and HadoopScott Carey 2011-10-10, 18:57
On 10/10/11 3:45 AM, "Steve Loughran" <[EMAIL PROTECTED]> wrote:
>On 08/10/11 19:19, Ted Dunning wrote:
>> I hate to sound like the folks who only recently stopped using 1.4, but
>>I am
>> afraid that Todd is right on here.
>>
>> The folks who are desperate for new features are being siphoned off by
>>Scala
>> and Clojure which is leaving a core of recalcitrant termagants like me.
>
>Language wise, you are right. I actually use Groovy under Java6 for a
>lot of work these days, as it offers many of the features of the Java7
>language today yet runs on Java6, and lets you subclass java classesand
>vice versa.
>
>Here, for example, is a MiniDFSCluster for Hadoop 0.20.20x that fixes
>its need for a JVM property to set the dest dir for data (fixed in
>trunk, BTW)
>
>http://bit.ly/nEUGKY
>
>
>> I
>> think that it is going to take something as major as the EOL of Java 6
>>to
>> get big projects to allow code that requires Java 7.
>
>JVM-wise, people have problems with the hotspot compiler, which means
>"don't use Hadoop with it"
>
>http://www.lucidimagination.com/blog/2011/07/28/dont-use-java-7-for-anythi
>ng/
>
>I don't know if the updates fix this. Eventually they shall.
It was fixed days before that blog post in OpenJDK trunk and for the JRE6
and JRE7 'next' releases (due out very soon now). Part of those bugs
exist in JRE 6, but are not exposed due to the default configuration
parameters differing. JDK7 is safe if you disable the loop predication
optimizations (which first appeared ~JRE 6u21).
>
>For people to be confident that it works someone needs to bring up a
>large cluster and run it on Hadoop long enough -with a more complex
>workload than terasort -to find the bugs, the race conditions, the
>problems that only surface once you serve up 2+PB of data.
This is the same requirement for a normal Hadoop release. The testing
needs to be done, regardless of the JRE version. Simply testing 0.23 or
0.24 with JRE 7u2 (which has the bugfixes above) or later is all that is
needed.
What JRE (6 update ?) is planned to be used when testing 0.23 at scale?
Should JRE 7u2 also be tested? Both a new update to JRE 6 and 7 is due
out very soon. 0.23 will be code complete after that. If I had enough
resources and time, I'd test both the latest JRE 6 and JRE 7.
>
>I hope someone volunteers for this. As Oracle have announced a
>Hadoop-based system, they could be the people to step up here, or they
>could pay someone else to do the work.
>
>In an ideal world, the Hadoop stack would be part of the test stack for
>a JVM release.
A performance regression for Hadoop's pure java CRC32 happened in a recent
JRE6 update, and a bug was filed, and they fixed it and now include that
algorithm in their test suite. JVM releases don't include whole stacks,
but someone could engage the OpenJDK developers to find out what kind of
contributions OpenJDK can accept for test code -- I'm not sure how
compatible it is with Apache.
http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2011-July/00597
1.html
http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2011-September/
006289.html
>
>In the meantime, even if Oracle say Java6 is EOL, if people pay money to
>keep it alive -and they will have to in any project you don't want to
>have to requalify for java7- then it may keep going for longer, except
>the updates won't be so widely available.
You can always keep running on the old JVM with the old version of Hadoop
you have had in your cluster, but if you upgrade Hadoop to a new version,
you might as well upgrade your JVM at the same time and pay the testing
cost once.
>
>I don't know if anyone with a Hadoop cluster has a support contract with
>Oracle for it. I know Yahoo! did, but I don't know it's current state.
+
Scott Carey 2011-10-10, 18:57
-
Re: Java Versions and HadoopSteve Loughran 2011-10-11, 11:18
On 10/10/11 19:57, Scott Carey wrote:
> What JRE (6 update ?) is planned to be used when testing 0.23 at scale?
> Should JRE 7u2 also be tested? Both a new update to JRE 6 and 7 is due
> out very soon. 0.23 will be code complete after that. If I had enough
> resources and time, I'd test both the latest JRE 6 and JRE 7.
Makes sense. Ideally anyone planning to move to 0.23 should bring up
some kind of cluster running that code on their chosen JVM, with their
own algorithms, just to see what the outcome is. Too bad nobody has a
large idle cluster with data they don't care about. EMC have just
announced one though, and HortonWorks and Cloudera will also have
clusters. That doesn't mean you shouldn't test on your own
hardware/OS/network/application setup.
>
> A performance regression for Hadoop's pure java CRC32 happened in a recent
> JRE6 update, and a bug was filed, and they fixed it and now include that
> algorithm in their test suite. JVM releases don't include whole stacks,
> but someone could engage the OpenJDK developers to find out what kind of
> contributions OpenJDK can accept for test code -- I'm not sure how
> compatible it is with Apache.
> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2011-July/00597
> 1.html
> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2011-September/
> 006289.html
Thanks, Scott, that is really informative
>
>>
>> In the meantime, even if Oracle say Java6 is EOL, if people pay money to
>> keep it alive -and they will have to in any project you don't want to
>> have to requalify for java7- then it may keep going for longer, except
>> the updates won't be so widely available.
>
> You can always keep running on the old JVM with the old version of Hadoop
> you have had in your cluster, but if you upgrade Hadoop to a new version,
> you might as well upgrade your JVM at the same time and pay the testing
> cost once.
I have mixed feelings about that. You may be introducing too many
variables at once.
> What JRE (6 update ?) is planned to be used when testing 0.23 at scale?
> Should JRE 7u2 also be tested? Both a new update to JRE 6 and 7 is due
> out very soon. 0.23 will be code complete after that. If I had enough
> resources and time, I'd test both the latest JRE 6 and JRE 7.
Makes sense. Ideally anyone planning to move to 0.23 should bring up
some kind of cluster running that code on their chosen JVM, with their
own algorithms, just to see what the outcome is. Too bad nobody has a
large idle cluster with data they don't care about. EMC have just
announced one though, and HortonWorks and Cloudera will also have
clusters. That doesn't mean you shouldn't test on your own
hardware/OS/network/application setup.
>
> A performance regression for Hadoop's pure java CRC32 happened in a recent
> JRE6 update, and a bug was filed, and they fixed it and now include that
> algorithm in their test suite. JVM releases don't include whole stacks,
> but someone could engage the OpenJDK developers to find out what kind of
> contributions OpenJDK can accept for test code -- I'm not sure how
> compatible it is with Apache.
> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2011-July/00597
> 1.html
> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2011-September/
> 006289.html
Thanks, Scott, that is really informative
>
>>
>> In the meantime, even if Oracle say Java6 is EOL, if people pay money to
>> keep it alive -and they will have to in any project you don't want to
>> have to requalify for java7- then it may keep going for longer, except
>> the updates won't be so widely available.
>
> You can always keep running on the old JVM with the old version of Hadoop
> you have had in your cluster, but if you upgrade Hadoop to a new version,
> you might as well upgrade your JVM at the same time and pay the testing
> cost once.
I have mixed feelings about that. You may be introducing too many
variables at once.
+
Steve Loughran 2011-10-11, 11:18
-
Re: Java Versions and HadoopScott Carey 2011-11-03, 21:35
On 10/11/11 4:18 AM, "Steve Loughran" <[EMAIL PROTECTED]> wrote:
>On 10/10/11 19:57, Scott Carey wrote:
>
>> What JRE (6 update ?) is planned to be used when testing 0.23 at scale?
>> Should JRE 7u2 also be tested? Both a new update to JRE 6 and 7 is due
>> out very soon. 0.23 will be code complete after that. If I had enough
>> resources and time, I'd test both the latest JRE 6 and JRE 7.
>
>Makes sense. Ideally anyone planning to move to 0.23 should bring up
>some kind of cluster running that code on their chosen JVM, with their
>own algorithms, just to see what the outcome is. Too bad nobody has a
>large idle cluster with data they don't care about. EMC have just
>announced one though, and HortonWorks and Cloudera will also have
>clusters. That doesn't mean you shouldn't test on your own
>hardware/OS/network/application setup.
>
>
>>
>> A performance regression for Hadoop's pure java CRC32 happened in a
>>recent
>> JRE6 update, and a bug was filed, and they fixed it and now include that
>> algorithm in their test suite. JVM releases don't include whole stacks,
>> but someone could engage the OpenJDK developers to find out what kind of
>> contributions OpenJDK can accept for test code -- I'm not sure how
>> compatible it is with Apache.
>>
>>http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2011-July/005
>>97
>> 1.html
>>
>>http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2011-Septembe
>>r/
>> 006289.html
>
>Thanks, Scott, that is really informative
FYI, Java 7u1 fixes the lucene related loop predicate bugs:
http://lucene.apache.org/solr/#26+October+2011+-+Java+7u1+fixes+index+corru
ption+and+crash+bugs+in+Apache+Lucene+Core+and+Apache+Solr
Java 6u29 fixes them as well (they are only exposed in Java 6 if you used
+AggressiveOpts or the equivalent).
7u2 is relatively far along: http://jdk7.java.net/download.html
>
>>
>>>
>>> In the meantime, even if Oracle say Java6 is EOL, if people pay money
>>>to
>>> keep it alive -and they will have to in any project you don't want to
>>> have to requalify for java7- then it may keep going for longer, except
>>> the updates won't be so widely available.
>>
>> You can always keep running on the old JVM with the old version of
>>Hadoop
>> you have had in your cluster, but if you upgrade Hadoop to a new
>>version,
>> you might as well upgrade your JVM at the same time and pay the testing
>> cost once.
>
>I have mixed feelings about that. You may be introducing too many
>variables at once.
+
Scott Carey 2011-11-03, 21:35
-
Re: Java Versions and HadoopScott Carey 2011-10-10, 04:33
On 10/7/11 3:33 PM, "[EMAIL PROTECTED]"
<[EMAIL PROTECTED]> wrote:
>Hi Folks,
>
>While I have seen the wiki on which java versions to use currently to run
>Hadoop, I have not seen any discussion about the roadmap of java version
>compatibility with future hadoop versions.
>
>Recently, Oracle retired the "Operating System Distributor License for
>Java" (DLJ) [http://robilad.livejournal.com/90792.html,
>http://jdk-distros.java.net/] and Linux vendors have started making
>OpenJDK (6/7) as the default java version bundled with their OSs
>[http://www.java7developer.com/blog/?p=361]. Also, all future Java SE
>updates will be delivered through OpenJDK updates project.
For Linux, Oracle will sill provide their version of the JVM available for
download from their site. Almost all Java SE updates (especially any that
affect Hadoop -- no sound or Swing) have been going through OpenJDK for
years.
You won't be getting the Oracle/Sun JVM directly from your Linux OS
vendor, but it is not going away. OpenJDK will be the only option that an
OS vendor can use to package Java themselves, but you can freely download
Oracle's version and run it.
>
>I see that OpenJDK6 (6b20pre) cannot be used to compile hadoop trunk. Has
>anyone tried OpenJDK7 ?
>
>Additionally, I have a few small projects in mind which can really make
>use of the new (esp I/O) features of Java 7.
>
>What, if any, timeline do hadoop developers have in mind to make Java 7 as
>required (and tested with OpenJDK 7) ?
I believe you mean when will it be compatible with Java 7, not require it
(based on all you mention above, other than the word 'required'). Is that
the case?
It will take some testing by some with at least medium sized clusters to
be confident that JRE 7 works with Hadoop, but not much else. The current
release of JRE 7 has some bugs, Oracle's JRE 7 update 2 (there was no
update 1) appears to be very close to release.
>
>Thanks,
>
>- milind
>
>---
>Milind Bhandarkar
>Greenplum Labs, EMC
>(Disclaimer: Opinions expressed in this email are those of the author, and
>do not necessarily represent the views of any organization, past or
>present, the author might be affiliated with.)
>
+
Scott Carey 2011-10-10, 04:33
-
Re: Java Versions and HadoopMilind.Bhandarkar@... 2011-10-10, 21:32
>I believe you mean when will it be compatible with Java 7, not require it
>(based on all you mention above, other than the word 'required'). Is that
>the case?
I meant, "when will I be able to use some of the JDK 7 APIs in Hadoop code
?", which means JDK7 is required for Hadoop.
- milind
---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)
+
Milind.Bhandarkar@... 2011-10-10, 21:32
-
Re: Java Versions and HadoopSteve Loughran 2011-10-11, 11:20
On 10/10/11 22:32, [EMAIL PROTECTED] wrote:
>
>> I believe you mean when will it be compatible with Java 7, not require it
>> (based on all you mention above, other than the word 'required'). Is that
>> the case?
>
> I meant, "when will I be able to use some of the JDK 7 APIs in Hadoop code
> ?", which means JDK7 is required for Hadoop.
>
There are enough language changes in Java7 to make considering this
worthwhile; the Java5->Java6 changes were so minor (@Override on
interface) that it wasn't that compelling.
I would suggest moving up projects on the side first, layers on top,
like your MPI-on-YARN stuff, where there is little or no installed base
to worry about
>
>> I believe you mean when will it be compatible with Java 7, not require it
>> (based on all you mention above, other than the word 'required'). Is that
>> the case?
>
> I meant, "when will I be able to use some of the JDK 7 APIs in Hadoop code
> ?", which means JDK7 is required for Hadoop.
>
There are enough language changes in Java7 to make considering this
worthwhile; the Java5->Java6 changes were so minor (@Override on
interface) that it wasn't that compelling.
I would suggest moving up projects on the side first, layers on top,
like your MPI-on-YARN stuff, where there is little or no installed base
to worry about
+
Steve Loughran 2011-10-11, 11:20
-
Re: Java Versions and HadoopMilind.Bhandarkar@... 2011-10-11, 17:31
Steve,
>I would suggest moving up projects on the side first, layers on top,
>like your MPI-on-YARN stuff, where there is little or no installed base
>to worry about
That is exactly the project for which I though some of the new I/O
interfaces would reduce the amount of code, (plus auto-freed resources on
exceptions, with try-with blocks). But my goal is to make it work in
current hadoop deployments, rather than having a special deploy with jdk7.
So, I need to worry about the current install-base of hadoop.
- milind
---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)
>I would suggest moving up projects on the side first, layers on top,
>like your MPI-on-YARN stuff, where there is little or no installed base
>to worry about
That is exactly the project for which I though some of the new I/O
interfaces would reduce the amount of code, (plus auto-freed resources on
exceptions, with try-with blocks). But my goal is to make it work in
current hadoop deployments, rather than having a special deploy with jdk7.
So, I need to worry about the current install-base of hadoop.
- milind
---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)
+
Milind.Bhandarkar@... 2011-10-11, 17:31
-
Re: Java Versions and HadoopArun C Murthy 2011-10-11, 17:32
MR2 supports different JVMs for user-apps.
Arun
On Oct 11, 2011, at 10:31 AM, <[EMAIL PROTECTED]> <[EMAIL PROTECTED]> wrote:
> Steve,
>
>> I would suggest moving up projects on the side first, layers on top,
>> like your MPI-on-YARN stuff, where there is little or no installed base
>> to worry about
>
> That is exactly the project for which I though some of the new I/O
> interfaces would reduce the amount of code, (plus auto-freed resources on
> exceptions, with try-with blocks). But my goal is to make it work in
> current hadoop deployments, rather than having a special deploy with jdk7.
> So, I need to worry about the current install-base of hadoop.
>
> - milind
>
> ---
> Milind Bhandarkar
> Greenplum Labs, EMC
> (Disclaimer: Opinions expressed in this email are those of the author, and
> do not necessarily represent the views of any organization, past or
> present, the author might be affiliated with.)
>
Arun
On Oct 11, 2011, at 10:31 AM, <[EMAIL PROTECTED]> <[EMAIL PROTECTED]> wrote:
> Steve,
>
>> I would suggest moving up projects on the side first, layers on top,
>> like your MPI-on-YARN stuff, where there is little or no installed base
>> to worry about
>
> That is exactly the project for which I though some of the new I/O
> interfaces would reduce the amount of code, (plus auto-freed resources on
> exceptions, with try-with blocks). But my goal is to make it work in
> current hadoop deployments, rather than having a special deploy with jdk7.
> So, I need to worry about the current install-base of hadoop.
>
> - milind
>
> ---
> Milind Bhandarkar
> Greenplum Labs, EMC
> (Disclaimer: Opinions expressed in this email are those of the author, and
> do not necessarily represent the views of any organization, past or
> present, the author might be affiliated with.)
>
+
Arun C Murthy 2011-10-11, 17:32
-
Re: Java Versions and HadoopMilind.Bhandarkar@... 2011-10-11, 18:57
Oh, that is very interesting ! Would I have to write my own
container-launcher for that ? Or does existing container launcher support
it ?
- Milind
---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)
On 10/11/11 10:32 AM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:
>MR2 supports different JVMs for user-apps.
>
>Arun
>
>On Oct 11, 2011, at 10:31 AM, <[EMAIL PROTECTED]>
><[EMAIL PROTECTED]> wrote:
>
>> Steve,
>>
>>> I would suggest moving up projects on the side first, layers on top,
>>> like your MPI-on-YARN stuff, where there is little or no installed base
>>> to worry about
>>
>> That is exactly the project for which I though some of the new I/O
>> interfaces would reduce the amount of code, (plus auto-freed resources
>>on
>> exceptions, with try-with blocks). But my goal is to make it work in
>> current hadoop deployments, rather than having a special deploy with
>>jdk7.
>> So, I need to worry about the current install-base of hadoop.
>>
>> - milind
>>
>> ---
>> Milind Bhandarkar
>> Greenplum Labs, EMC
>> (Disclaimer: Opinions expressed in this email are those of the author,
>>and
>> do not necessarily represent the views of any organization, past or
>> present, the author might be affiliated with.)
>>
>
>
container-launcher for that ? Or does existing container launcher support
it ?
- Milind
---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)
On 10/11/11 10:32 AM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:
>MR2 supports different JVMs for user-apps.
>
>Arun
>
>On Oct 11, 2011, at 10:31 AM, <[EMAIL PROTECTED]>
><[EMAIL PROTECTED]> wrote:
>
>> Steve,
>>
>>> I would suggest moving up projects on the side first, layers on top,
>>> like your MPI-on-YARN stuff, where there is little or no installed base
>>> to worry about
>>
>> That is exactly the project for which I though some of the new I/O
>> interfaces would reduce the amount of code, (plus auto-freed resources
>>on
>> exceptions, with try-with blocks). But my goal is to make it work in
>> current hadoop deployments, rather than having a special deploy with
>>jdk7.
>> So, I need to worry about the current install-base of hadoop.
>>
>> - milind
>>
>> ---
>> Milind Bhandarkar
>> Greenplum Labs, EMC
>> (Disclaimer: Opinions expressed in this email are those of the author,
>>and
>> do not necessarily represent the views of any organization, past or
>> present, the author might be affiliated with.)
>>
>
>
+
Milind.Bhandarkar@... 2011-10-11, 18:57
-
Re: Java Versions and HadoopArun C Murthy 2011-10-11, 22:16
On Oct 11, 2011, at 11:57 AM, <[EMAIL PROTECTED]> <[EMAIL PROTECTED]> wrote:
> Oh, that is very interesting ! Would I have to write my own
> container-launcher for that ? Or does existing container launcher support
> it ?
>
In MR2 the command line for the container is the AM's own responsibility.
So, you can easily set whatever JAVA_HOME for your container in the cmd-line. Or something other than java too... :)
Also, you can either choose to have a alternate java install on all nodes or ship them with the Container itself (i.e. think DistributedCache).
@see ContainerLaunchContext.
More details here:
http://people.apache.org/~acmurthy/yarn/WritingYarnApplications.html
We can take this discussion to mapreduce-dev@.
Arun
PS: That link will point to hadoop.apache.org once we have a release of 0.23, something I host as an temp. alternative till then.
> - Milind
>
> ---
> Milind Bhandarkar
> Greenplum Labs, EMC
> (Disclaimer: Opinions expressed in this email are those of the author, and
> do not necessarily represent the views of any organization, past or
> present, the author might be affiliated with.)
>
>
>
> On 10/11/11 10:32 AM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:
>
>> MR2 supports different JVMs for user-apps.
>>
>> Arun
>>
>> On Oct 11, 2011, at 10:31 AM, <[EMAIL PROTECTED]>
>> <[EMAIL PROTECTED]> wrote:
>>
>>> Steve,
>>>
>>>> I would suggest moving up projects on the side first, layers on top,
>>>> like your MPI-on-YARN stuff, where there is little or no installed base
>>>> to worry about
>>>
>>> That is exactly the project for which I though some of the new I/O
>>> interfaces would reduce the amount of code, (plus auto-freed resources
>>> on
>>> exceptions, with try-with blocks). But my goal is to make it work in
>>> current hadoop deployments, rather than having a special deploy with
>>> jdk7.
>>> So, I need to worry about the current install-base of hadoop.
>>>
>>> - milind
>>>
>>> ---
>>> Milind Bhandarkar
>>> Greenplum Labs, EMC
>>> (Disclaimer: Opinions expressed in this email are those of the author,
>>> and
>>> do not necessarily represent the views of any organization, past or
>>> present, the author might be affiliated with.)
>>>
>>
>>
>
> Oh, that is very interesting ! Would I have to write my own
> container-launcher for that ? Or does existing container launcher support
> it ?
>
In MR2 the command line for the container is the AM's own responsibility.
So, you can easily set whatever JAVA_HOME for your container in the cmd-line. Or something other than java too... :)
Also, you can either choose to have a alternate java install on all nodes or ship them with the Container itself (i.e. think DistributedCache).
@see ContainerLaunchContext.
More details here:
http://people.apache.org/~acmurthy/yarn/WritingYarnApplications.html
We can take this discussion to mapreduce-dev@.
Arun
PS: That link will point to hadoop.apache.org once we have a release of 0.23, something I host as an temp. alternative till then.
> - Milind
>
> ---
> Milind Bhandarkar
> Greenplum Labs, EMC
> (Disclaimer: Opinions expressed in this email are those of the author, and
> do not necessarily represent the views of any organization, past or
> present, the author might be affiliated with.)
>
>
>
> On 10/11/11 10:32 AM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:
>
>> MR2 supports different JVMs for user-apps.
>>
>> Arun
>>
>> On Oct 11, 2011, at 10:31 AM, <[EMAIL PROTECTED]>
>> <[EMAIL PROTECTED]> wrote:
>>
>>> Steve,
>>>
>>>> I would suggest moving up projects on the side first, layers on top,
>>>> like your MPI-on-YARN stuff, where there is little or no installed base
>>>> to worry about
>>>
>>> That is exactly the project for which I though some of the new I/O
>>> interfaces would reduce the amount of code, (plus auto-freed resources
>>> on
>>> exceptions, with try-with blocks). But my goal is to make it work in
>>> current hadoop deployments, rather than having a special deploy with
>>> jdk7.
>>> So, I need to worry about the current install-base of hadoop.
>>>
>>> - milind
>>>
>>> ---
>>> Milind Bhandarkar
>>> Greenplum Labs, EMC
>>> (Disclaimer: Opinions expressed in this email are those of the author,
>>> and
>>> do not necessarily represent the views of any organization, past or
>>> present, the author might be affiliated with.)
>>>
>>
>>
>
+
Arun C Murthy 2011-10-11, 22:16
