Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - Java Versions and Hadoop


Copy link to this message
-
Re: Java Versions and Hadoop
Steve Loughran 2011-10-10, 10:45
On 08/10/11 19:19, Ted Dunning wrote:
> I hate to sound like the folks who only recently stopped using 1.4, but I am
> afraid that Todd is right on here.
>
> The folks who are desperate for new features are being siphoned off by Scala
> and Clojure which is leaving a core of recalcitrant termagants like me.

Language wise, you are right. I actually use Groovy under Java6 for a
lot of work these days, as it offers many of the features of the Java7
language today yet runs on Java6, and lets you subclass java classesand
vice versa.

Here, for example, is a MiniDFSCluster for Hadoop 0.20.20x that fixes
its need for a JVM property to set the dest dir for data (fixed in
trunk, BTW)

http://bit.ly/nEUGKY
> I
> think that it is going to take something as major as the EOL of Java 6 to
> get big projects to allow code that requires Java 7.

JVM-wise, people have problems with the hotspot compiler, which means
"don't use Hadoop with it"

http://www.lucidimagination.com/blog/2011/07/28/dont-use-java-7-for-anything/

I don't know if the updates fix this. Eventually they shall.

For people to be confident that it works someone needs to bring up a
large cluster and run it on Hadoop long enough -with a more complex
workload than terasort -to find the bugs, the race conditions, the
problems that only surface once you serve up 2+PB of data.

I hope someone volunteers for this. As Oracle have announced a
Hadoop-based system, they could be the people to step up here, or they
could pay someone else to do the work.

In an ideal world, the Hadoop stack would be part of the test stack for
a JVM release.

In the meantime, even if Oracle say Java6 is EOL, if people pay money to
keep it alive -and they will have to in any project you don't want to
have to requalify for java7- then it may keep going for longer, except
the updates won't be so widely available.

I don't know if anyone with a Hadoop cluster has a support contract with
Oracle for it. I know Yahoo! did, but I don't know it's current state.