Jonathan Coveney
2012-04-12, 18:55
Gianmarco De Francisci Mo...
2012-04-12, 19:02
Scott Carey
2012-04-12, 23:10
Gianmarco De Francisci Mo...
2012-04-13, 08:30
Alan Gates
2012-04-16, 16:58
Scott Carey
2012-04-16, 17:18
Dmitriy Ryaboy
2012-04-16, 17:28
Scott Carey
2012-04-16, 17:46
-
Java 7 and Pig, hijacked from PIG-2643Jonathan Coveney 2012-04-12, 18:55
Scott Carey brought Java 7 up in PIG-2643, and I think it's something we
need to think about. When do we want to start taking advantage of new
features that may not exist on Java 6? Do we ever?
2012/4/12 Scott Carey (Commented) (JIRA) <[EMAIL PROTECTED]>
>
> [
> https://issues.apache.org/jira/browse/PIG-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252706#comment-13252706]
>
> Scott Carey commented on PIG-2643:
> ----------------------------------
>
> Another thought for this sort of thing:
>
> This might be achievable without bytecode generation and good performance
> with Java 7 MethodHandles [1][2]. Of course, that would require Java 7,
> but Java 6 support ends later year [3], about the time Pig 0.11 would be
> out anyway.
>
>
> [1]
> http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.html
> [2]
> http://stackoverflow.com/questions/8823793/methodhandle-what-is-it-all-about
> [3] https://blogs.oracle.com/henrik/entry/updated_java_6_eol_date
>
> > Use bytecode generation to make a performance replacement for
> InvokeForLong, InvokeForString, etc
> >
> -------------------------------------------------------------------------------------------------
> >
> > Key: PIG-2643
> > URL: https://issues.apache.org/jira/browse/PIG-2643
> > Project: Pig
> > Issue Type: Improvement
> > Reporter: Jonathan Coveney
> > Assignee: Jonathan Coveney
> > Priority: Minor
> > Labels: codegen
> > Fix For: 0.11, 0.10.1
> >
> > Attachments: PIG-2643-0.patch
> >
> >
> > This is basically to cut my teeth for much more ambitious code
> generation down the line, but I think it could be performance and useful.
> > the new syntax is:
> > {code}a = load 'thing' as (x:chararray);
> > define concat InvokerGenerator('java.lang.String','concat','String');
> > define valueOf InvokerGenerator('java.lang.Integer','valueOf','String');
> > define valueOfRadix
> InvokerGenerator('java.lang.Integer','valueOf','String,int');
> > b = foreach a generate x, valueOf(x) as vOf;
> > c = foreach b generate x, vOf, valueOfRadix(x, 16) as vOfR;
> > d = foreach c generate x, vOf, vOfR, concat(concat(x, (chararray)vOf),
> (chararray)vOfR);
> > dump d;
> > {code}
> > There are some differences between this version and Dmitriy's
> implementation:
> > - it is no longer necessary to declare whether the method is static or
> not. This is gleaned via reflection.
> > - as per the above, it is no longer necessary to make the first argument
> be the type of the object to invoke the method on. If it is not a static
> method, then the type will implicitly be the type you need. So in the case
> of concat, it would need to be passed a tuple of two inputs: one for the
> method to be called against (as it is not static), and then the 'string'
> that was specified. In the case of valueOf, because it IS static, then the
> 'String' is the only value.
> > - The arguments are type sensitive. Integer means the Object Integer,
> whereas int (or long, or float, or boolean, etc) refer to the primitive.
> This is necessary to properly reflect the arguments. Values passed in WILL,
> however, be properly unboxed as necessary.
> > - The return type will be reflected.
> > This uses the ASM API to generate the bytecode, and then a custom
> classloader to load it in. I will add caching of the generated code based
> on the input strings, etc, but I wanted to get eyes and opinions on this. I
> also need to benchmark, but it should be native speed (excluding a little
> startup time to make the bytecode, but ASM is really fast).
> > Another nice benefit is that this bypasses the need for the JDK, though
> it adds a dependency on ASM (which is a super tiny dependency).
> > Patch incoming.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators:
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
need to think about. When do we want to start taking advantage of new
features that may not exist on Java 6? Do we ever?
2012/4/12 Scott Carey (Commented) (JIRA) <[EMAIL PROTECTED]>
>
> [
> https://issues.apache.org/jira/browse/PIG-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252706#comment-13252706]
>
> Scott Carey commented on PIG-2643:
> ----------------------------------
>
> Another thought for this sort of thing:
>
> This might be achievable without bytecode generation and good performance
> with Java 7 MethodHandles [1][2]. Of course, that would require Java 7,
> but Java 6 support ends later year [3], about the time Pig 0.11 would be
> out anyway.
>
>
> [1]
> http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.html
> [2]
> http://stackoverflow.com/questions/8823793/methodhandle-what-is-it-all-about
> [3] https://blogs.oracle.com/henrik/entry/updated_java_6_eol_date
>
> > Use bytecode generation to make a performance replacement for
> InvokeForLong, InvokeForString, etc
> >
> -------------------------------------------------------------------------------------------------
> >
> > Key: PIG-2643
> > URL: https://issues.apache.org/jira/browse/PIG-2643
> > Project: Pig
> > Issue Type: Improvement
> > Reporter: Jonathan Coveney
> > Assignee: Jonathan Coveney
> > Priority: Minor
> > Labels: codegen
> > Fix For: 0.11, 0.10.1
> >
> > Attachments: PIG-2643-0.patch
> >
> >
> > This is basically to cut my teeth for much more ambitious code
> generation down the line, but I think it could be performance and useful.
> > the new syntax is:
> > {code}a = load 'thing' as (x:chararray);
> > define concat InvokerGenerator('java.lang.String','concat','String');
> > define valueOf InvokerGenerator('java.lang.Integer','valueOf','String');
> > define valueOfRadix
> InvokerGenerator('java.lang.Integer','valueOf','String,int');
> > b = foreach a generate x, valueOf(x) as vOf;
> > c = foreach b generate x, vOf, valueOfRadix(x, 16) as vOfR;
> > d = foreach c generate x, vOf, vOfR, concat(concat(x, (chararray)vOf),
> (chararray)vOfR);
> > dump d;
> > {code}
> > There are some differences between this version and Dmitriy's
> implementation:
> > - it is no longer necessary to declare whether the method is static or
> not. This is gleaned via reflection.
> > - as per the above, it is no longer necessary to make the first argument
> be the type of the object to invoke the method on. If it is not a static
> method, then the type will implicitly be the type you need. So in the case
> of concat, it would need to be passed a tuple of two inputs: one for the
> method to be called against (as it is not static), and then the 'string'
> that was specified. In the case of valueOf, because it IS static, then the
> 'String' is the only value.
> > - The arguments are type sensitive. Integer means the Object Integer,
> whereas int (or long, or float, or boolean, etc) refer to the primitive.
> This is necessary to properly reflect the arguments. Values passed in WILL,
> however, be properly unboxed as necessary.
> > - The return type will be reflected.
> > This uses the ASM API to generate the bytecode, and then a custom
> classloader to load it in. I will add caching of the generated code based
> on the input strings, etc, but I wanted to get eyes and opinions on this. I
> also need to benchmark, but it should be native speed (excluding a little
> startup time to make the bytecode, but ASM is really fast).
> > Another nice benefit is that this bypasses the need for the JDK, though
> it adds a dependency on ASM (which is a super tiny dependency).
> > Patch incoming.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators:
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
-
Re: Java 7 and Pig, hijacked from PIG-2643Gianmarco De Francisci Mo... 2012-04-12, 19:02
My personal opinion is Yes, if there is a compelling reason, but not now.
Still not mature enough.
Cheers,
--
Gianmarco
On Thu, Apr 12, 2012 at 20:55, Jonathan Coveney <[EMAIL PROTECTED]> wrote:
> Scott Carey brought Java 7 up in PIG-2643, and I think it's something we
> need to think about. When do we want to start taking advantage of new
> features that may not exist on Java 6? Do we ever?
>
> 2012/4/12 Scott Carey (Commented) (JIRA) <[EMAIL PROTECTED]>
>
> >
> > [
> >
> https://issues.apache.org/jira/browse/PIG-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252706#comment-13252706
> ]
> >
> > Scott Carey commented on PIG-2643:
> > ----------------------------------
> >
> > Another thought for this sort of thing:
> >
> > This might be achievable without bytecode generation and good performance
> > with Java 7 MethodHandles [1][2]. Of course, that would require Java 7,
> > but Java 6 support ends later year [3], about the time Pig 0.11 would be
> > out anyway.
> >
> >
> > [1]
> >
> http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.html
> > [2]
> >
> http://stackoverflow.com/questions/8823793/methodhandle-what-is-it-all-about
> > [3] https://blogs.oracle.com/henrik/entry/updated_java_6_eol_date
> >
> > > Use bytecode generation to make a performance replacement for
> > InvokeForLong, InvokeForString, etc
> > >
> >
> -------------------------------------------------------------------------------------------------
> > >
> > > Key: PIG-2643
> > > URL: https://issues.apache.org/jira/browse/PIG-2643
> > > Project: Pig
> > > Issue Type: Improvement
> > > Reporter: Jonathan Coveney
> > > Assignee: Jonathan Coveney
> > > Priority: Minor
> > > Labels: codegen
> > > Fix For: 0.11, 0.10.1
> > >
> > > Attachments: PIG-2643-0.patch
> > >
> > >
> > > This is basically to cut my teeth for much more ambitious code
> > generation down the line, but I think it could be performance and useful.
> > > the new syntax is:
> > > {code}a = load 'thing' as (x:chararray);
> > > define concat InvokerGenerator('java.lang.String','concat','String');
> > > define valueOf
> InvokerGenerator('java.lang.Integer','valueOf','String');
> > > define valueOfRadix
> > InvokerGenerator('java.lang.Integer','valueOf','String,int');
> > > b = foreach a generate x, valueOf(x) as vOf;
> > > c = foreach b generate x, vOf, valueOfRadix(x, 16) as vOfR;
> > > d = foreach c generate x, vOf, vOfR, concat(concat(x, (chararray)vOf),
> > (chararray)vOfR);
> > > dump d;
> > > {code}
> > > There are some differences between this version and Dmitriy's
> > implementation:
> > > - it is no longer necessary to declare whether the method is static or
> > not. This is gleaned via reflection.
> > > - as per the above, it is no longer necessary to make the first
> argument
> > be the type of the object to invoke the method on. If it is not a static
> > method, then the type will implicitly be the type you need. So in the
> case
> > of concat, it would need to be passed a tuple of two inputs: one for the
> > method to be called against (as it is not static), and then the 'string'
> > that was specified. In the case of valueOf, because it IS static, then
> the
> > 'String' is the only value.
> > > - The arguments are type sensitive. Integer means the Object Integer,
> > whereas int (or long, or float, or boolean, etc) refer to the primitive.
> > This is necessary to properly reflect the arguments. Values passed in
> WILL,
> > however, be properly unboxed as necessary.
> > > - The return type will be reflected.
> > > This uses the ASM API to generate the bytecode, and then a custom
> > classloader to load it in. I will add caching of the generated code based
> > on the input strings, etc, but I wanted to get eyes and opinions on
> this. I
> > also need to benchmark, but it should be native speed (excluding a little
Still not mature enough.
Cheers,
--
Gianmarco
On Thu, Apr 12, 2012 at 20:55, Jonathan Coveney <[EMAIL PROTECTED]> wrote:
> Scott Carey brought Java 7 up in PIG-2643, and I think it's something we
> need to think about. When do we want to start taking advantage of new
> features that may not exist on Java 6? Do we ever?
>
> 2012/4/12 Scott Carey (Commented) (JIRA) <[EMAIL PROTECTED]>
>
> >
> > [
> >
> https://issues.apache.org/jira/browse/PIG-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252706#comment-13252706
> ]
> >
> > Scott Carey commented on PIG-2643:
> > ----------------------------------
> >
> > Another thought for this sort of thing:
> >
> > This might be achievable without bytecode generation and good performance
> > with Java 7 MethodHandles [1][2]. Of course, that would require Java 7,
> > but Java 6 support ends later year [3], about the time Pig 0.11 would be
> > out anyway.
> >
> >
> > [1]
> >
> http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.html
> > [2]
> >
> http://stackoverflow.com/questions/8823793/methodhandle-what-is-it-all-about
> > [3] https://blogs.oracle.com/henrik/entry/updated_java_6_eol_date
> >
> > > Use bytecode generation to make a performance replacement for
> > InvokeForLong, InvokeForString, etc
> > >
> >
> -------------------------------------------------------------------------------------------------
> > >
> > > Key: PIG-2643
> > > URL: https://issues.apache.org/jira/browse/PIG-2643
> > > Project: Pig
> > > Issue Type: Improvement
> > > Reporter: Jonathan Coveney
> > > Assignee: Jonathan Coveney
> > > Priority: Minor
> > > Labels: codegen
> > > Fix For: 0.11, 0.10.1
> > >
> > > Attachments: PIG-2643-0.patch
> > >
> > >
> > > This is basically to cut my teeth for much more ambitious code
> > generation down the line, but I think it could be performance and useful.
> > > the new syntax is:
> > > {code}a = load 'thing' as (x:chararray);
> > > define concat InvokerGenerator('java.lang.String','concat','String');
> > > define valueOf
> InvokerGenerator('java.lang.Integer','valueOf','String');
> > > define valueOfRadix
> > InvokerGenerator('java.lang.Integer','valueOf','String,int');
> > > b = foreach a generate x, valueOf(x) as vOf;
> > > c = foreach b generate x, vOf, valueOfRadix(x, 16) as vOfR;
> > > d = foreach c generate x, vOf, vOfR, concat(concat(x, (chararray)vOf),
> > (chararray)vOfR);
> > > dump d;
> > > {code}
> > > There are some differences between this version and Dmitriy's
> > implementation:
> > > - it is no longer necessary to declare whether the method is static or
> > not. This is gleaned via reflection.
> > > - as per the above, it is no longer necessary to make the first
> argument
> > be the type of the object to invoke the method on. If it is not a static
> > method, then the type will implicitly be the type you need. So in the
> case
> > of concat, it would need to be passed a tuple of two inputs: one for the
> > method to be called against (as it is not static), and then the 'string'
> > that was specified. In the case of valueOf, because it IS static, then
> the
> > 'String' is the only value.
> > > - The arguments are type sensitive. Integer means the Object Integer,
> > whereas int (or long, or float, or boolean, etc) refer to the primitive.
> > This is necessary to properly reflect the arguments. Values passed in
> WILL,
> > however, be properly unboxed as necessary.
> > > - The return type will be reflected.
> > > This uses the ASM API to generate the bytecode, and then a custom
> > classloader to load it in. I will add caching of the generated code based
> > on the input strings, etc, but I wanted to get eyes and opinions on
> this. I
> > also need to benchmark, but it should be native speed (excluding a little
-
Re: Java 7 and Pig, hijacked from PIG-2643Scott Carey 2012-04-12, 23:10
On 4/12/12 12:02 PM, "Gianmarco De Francisci Morales" <[EMAIL PROTECTED]>
wrote:
>My personal opinion is Yes, if there is a compelling reason, but not now.
>Still not mature enough.
What is the definition of "mature enough"?
I'd argue that out of Java 7, Hadoop, Avro, and Pig... Java 7 is the most
mature.
Granted, I also think the whole concept of maturity is subjective and not
conducive to good discussion.
What objective characteristics are required of Java 7 to support it? When
approximately would that likely occur? Subtract 6 months from that (a
typical Pig dev cycle), how soon is that?
There is a big difference between _requiring_ Java 7 and having an
optional feature built with Java 7 into its own jar that requires a Java 7
cluster to use. Some performance features might fall into that category.
>
>Cheers,
>--
>Gianmarco
>
>
>
>On Thu, Apr 12, 2012 at 20:55, Jonathan Coveney <[EMAIL PROTECTED]>
>wrote:
>
>> Scott Carey brought Java 7 up in PIG-2643, and I think it's something we
>> need to think about. When do we want to start taking advantage of new
>> features that may not exist on Java 6? Do we ever?
>>
>> 2012/4/12 Scott Carey (Commented) (JIRA) <[EMAIL PROTECTED]>
>>
>> >
>> > [
>> >
>>
>>https://issues.apache.org/jira/browse/PIG-2643?page=com.atlassian.jira.pl
>>ugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252706#com
>>ment-13252706
>> ]
>> >
>> > Scott Carey commented on PIG-2643:
>> > ----------------------------------
>> >
>> > Another thought for this sort of thing:
>> >
>> > This might be achievable without bytecode generation and good
>>performance
>> > with Java 7 MethodHandles [1][2]. Of course, that would require Java
>>7,
>> > but Java 6 support ends later year [3], about the time Pig 0.11 would
>>be
>> > out anyway.
>> >
>> >
>> > [1]
>> >
>>
>>http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.ht
>>ml
>> > [2]
>> >
>>
>>http://stackoverflow.com/questions/8823793/methodhandle-what-is-it-all-ab
>>out
>> > [3] https://blogs.oracle.com/henrik/entry/updated_java_6_eol_date
>> >
>> > > Use bytecode generation to make a performance replacement for
>> > InvokeForLong, InvokeForString, etc
>> > >
>> >
>>
>>-------------------------------------------------------------------------
>>------------------------
>> > >
>> > > Key: PIG-2643
>> > > URL: https://issues.apache.org/jira/browse/PIG-2643
>> > > Project: Pig
>> > > Issue Type: Improvement
>> > > Reporter: Jonathan Coveney
>> > > Assignee: Jonathan Coveney
>> > > Priority: Minor
>> > > Labels: codegen
>> > > Fix For: 0.11, 0.10.1
>> > >
>> > > Attachments: PIG-2643-0.patch
>> > >
>> > >
>> > > This is basically to cut my teeth for much more ambitious code
>> > generation down the line, but I think it could be performance and
>>useful.
>> > > the new syntax is:
>> > > {code}a = load 'thing' as (x:chararray);
>> > > define concat
>>InvokerGenerator('java.lang.String','concat','String');
>> > > define valueOf
>> InvokerGenerator('java.lang.Integer','valueOf','String');
>> > > define valueOfRadix
>> > InvokerGenerator('java.lang.Integer','valueOf','String,int');
>> > > b = foreach a generate x, valueOf(x) as vOf;
>> > > c = foreach b generate x, vOf, valueOfRadix(x, 16) as vOfR;
>> > > d = foreach c generate x, vOf, vOfR, concat(concat(x,
>>(chararray)vOf),
>> > (chararray)vOfR);
>> > > dump d;
>> > > {code}
>> > > There are some differences between this version and Dmitriy's
>> > implementation:
>> > > - it is no longer necessary to declare whether the method is static
>>or
>> > not. This is gleaned via reflection.
>> > > - as per the above, it is no longer necessary to make the first
>> argument
>> > be the type of the object to invoke the method on. If it is not a
>>static
>> > method, then the type will implicitly be the type you need. So in the
>> case
>> > of concat, it would need to be passed a tuple of two inputs: one for
-
Re: Java 7 and Pig, hijacked from PIG-2643Gianmarco De Francisci Mo... 2012-04-13, 08:30
Sure,
probably Java7 is more mature than Pig.
However as far as I know it is a pain to install on Mac.
This is just one small example of not mature enough.
I am under the impression that Java7 not yet mainstream, correct me if I am
wrong.
I am fine with having optional performance improvements enabled by Java7. +1
I am not fine with having a dependency on Java7 for users. -1
I am not comfortable with having a dependency on Java7 for pig developers.
I am afraid it will put off new contributors. -0
Cheers,
--
Gianmarco
On Fri, Apr 13, 2012 at 01:10, Scott Carey <[EMAIL PROTECTED]> wrote:
>
> On 4/12/12 12:02 PM, "Gianmarco De Francisci Morales" <[EMAIL PROTECTED]>
> wrote:
>
> >My personal opinion is Yes, if there is a compelling reason, but not now.
> >Still not mature enough.
>
> What is the definition of "mature enough"?
>
> I'd argue that out of Java 7, Hadoop, Avro, and Pig... Java 7 is the most
> mature.
>
> Granted, I also think the whole concept of maturity is subjective and not
> conducive to good discussion.
>
>
> What objective characteristics are required of Java 7 to support it? When
> approximately would that likely occur? Subtract 6 months from that (a
> typical Pig dev cycle), how soon is that?
>
> There is a big difference between _requiring_ Java 7 and having an
> optional feature built with Java 7 into its own jar that requires a Java 7
> cluster to use. Some performance features might fall into that category.
>
> >
> >Cheers,
> >--
> >Gianmarco
> >
> >
> >
> >On Thu, Apr 12, 2012 at 20:55, Jonathan Coveney <[EMAIL PROTECTED]>
> >wrote:
> >
> >> Scott Carey brought Java 7 up in PIG-2643, and I think it's something we
> >> need to think about. When do we want to start taking advantage of new
> >> features that may not exist on Java 6? Do we ever?
> >>
> >> 2012/4/12 Scott Carey (Commented) (JIRA) <[EMAIL PROTECTED]>
> >>
> >> >
> >> > [
> >> >
> >>
> >>
> https://issues.apache.org/jira/browse/PIG-2643?page=com.atlassian.jira.pl
> >>ugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252706#com
> >>ment-13252706
> >> ]
> >> >
> >> > Scott Carey commented on PIG-2643:
> >> > ----------------------------------
> >> >
> >> > Another thought for this sort of thing:
> >> >
> >> > This might be achievable without bytecode generation and good
> >>performance
> >> > with Java 7 MethodHandles [1][2]. Of course, that would require Java
> >>7,
> >> > but Java 6 support ends later year [3], about the time Pig 0.11 would
> >>be
> >> > out anyway.
> >> >
> >> >
> >> > [1]
> >> >
> >>
> >>
> http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.ht
> >>ml
> >> > [2]
> >> >
> >>
> >>
> http://stackoverflow.com/questions/8823793/methodhandle-what-is-it-all-ab
> >>out
> >> > [3] https://blogs.oracle.com/henrik/entry/updated_java_6_eol_date
> >> >
> >> > > Use bytecode generation to make a performance replacement for
> >> > InvokeForLong, InvokeForString, etc
> >> > >
> >> >
> >>
> >>-------------------------------------------------------------------------
> >>------------------------
> >> > >
> >> > > Key: PIG-2643
> >> > > URL: https://issues.apache.org/jira/browse/PIG-2643
> >> > > Project: Pig
> >> > > Issue Type: Improvement
> >> > > Reporter: Jonathan Coveney
> >> > > Assignee: Jonathan Coveney
> >> > > Priority: Minor
> >> > > Labels: codegen
> >> > > Fix For: 0.11, 0.10.1
> >> > >
> >> > > Attachments: PIG-2643-0.patch
> >> > >
> >> > >
> >> > > This is basically to cut my teeth for much more ambitious code
> >> > generation down the line, but I think it could be performance and
> >>useful.
> >> > > the new syntax is:
> >> > > {code}a = load 'thing' as (x:chararray);
> >> > > define concat
> >>InvokerGenerator('java.lang.String','concat','String');
> >> > > define valueOf
> >> InvokerGenerator('java.lang.Integer','valueOf','String');
> >> > > define valueOfRadix
> >> > InvokerGenerator('java.lang.Integer','valueOf','String,int');
probably Java7 is more mature than Pig.
However as far as I know it is a pain to install on Mac.
This is just one small example of not mature enough.
I am under the impression that Java7 not yet mainstream, correct me if I am
wrong.
I am fine with having optional performance improvements enabled by Java7. +1
I am not fine with having a dependency on Java7 for users. -1
I am not comfortable with having a dependency on Java7 for pig developers.
I am afraid it will put off new contributors. -0
Cheers,
--
Gianmarco
On Fri, Apr 13, 2012 at 01:10, Scott Carey <[EMAIL PROTECTED]> wrote:
>
> On 4/12/12 12:02 PM, "Gianmarco De Francisci Morales" <[EMAIL PROTECTED]>
> wrote:
>
> >My personal opinion is Yes, if there is a compelling reason, but not now.
> >Still not mature enough.
>
> What is the definition of "mature enough"?
>
> I'd argue that out of Java 7, Hadoop, Avro, and Pig... Java 7 is the most
> mature.
>
> Granted, I also think the whole concept of maturity is subjective and not
> conducive to good discussion.
>
>
> What objective characteristics are required of Java 7 to support it? When
> approximately would that likely occur? Subtract 6 months from that (a
> typical Pig dev cycle), how soon is that?
>
> There is a big difference between _requiring_ Java 7 and having an
> optional feature built with Java 7 into its own jar that requires a Java 7
> cluster to use. Some performance features might fall into that category.
>
> >
> >Cheers,
> >--
> >Gianmarco
> >
> >
> >
> >On Thu, Apr 12, 2012 at 20:55, Jonathan Coveney <[EMAIL PROTECTED]>
> >wrote:
> >
> >> Scott Carey brought Java 7 up in PIG-2643, and I think it's something we
> >> need to think about. When do we want to start taking advantage of new
> >> features that may not exist on Java 6? Do we ever?
> >>
> >> 2012/4/12 Scott Carey (Commented) (JIRA) <[EMAIL PROTECTED]>
> >>
> >> >
> >> > [
> >> >
> >>
> >>
> https://issues.apache.org/jira/browse/PIG-2643?page=com.atlassian.jira.pl
> >>ugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252706#com
> >>ment-13252706
> >> ]
> >> >
> >> > Scott Carey commented on PIG-2643:
> >> > ----------------------------------
> >> >
> >> > Another thought for this sort of thing:
> >> >
> >> > This might be achievable without bytecode generation and good
> >>performance
> >> > with Java 7 MethodHandles [1][2]. Of course, that would require Java
> >>7,
> >> > but Java 6 support ends later year [3], about the time Pig 0.11 would
> >>be
> >> > out anyway.
> >> >
> >> >
> >> > [1]
> >> >
> >>
> >>
> http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.ht
> >>ml
> >> > [2]
> >> >
> >>
> >>
> http://stackoverflow.com/questions/8823793/methodhandle-what-is-it-all-ab
> >>out
> >> > [3] https://blogs.oracle.com/henrik/entry/updated_java_6_eol_date
> >> >
> >> > > Use bytecode generation to make a performance replacement for
> >> > InvokeForLong, InvokeForString, etc
> >> > >
> >> >
> >>
> >>-------------------------------------------------------------------------
> >>------------------------
> >> > >
> >> > > Key: PIG-2643
> >> > > URL: https://issues.apache.org/jira/browse/PIG-2643
> >> > > Project: Pig
> >> > > Issue Type: Improvement
> >> > > Reporter: Jonathan Coveney
> >> > > Assignee: Jonathan Coveney
> >> > > Priority: Minor
> >> > > Labels: codegen
> >> > > Fix For: 0.11, 0.10.1
> >> > >
> >> > > Attachments: PIG-2643-0.patch
> >> > >
> >> > >
> >> > > This is basically to cut my teeth for much more ambitious code
> >> > generation down the line, but I think it could be performance and
> >>useful.
> >> > > the new syntax is:
> >> > > {code}a = load 'thing' as (x:chararray);
> >> > > define concat
> >>InvokerGenerator('java.lang.String','concat','String');
> >> > > define valueOf
> >> InvokerGenerator('java.lang.Integer','valueOf','String');
> >> > > define valueOfRadix
> >> > InvokerGenerator('java.lang.Integer','valueOf','String,int');
-
Re: Java 7 and Pig, hijacked from PIG-2643Alan Gates 2012-04-16, 16:58
There are some exciting new features in Java 7. However, realistically we can't start using it until Hadoop does. I don't recall any discussion on it on their list, though I may have missed it. But AFAIK they have no migration plans at this time.
Alan.
On Apr 12, 2012, at 11:55 AM, Jonathan Coveney wrote:
> Scott Carey brought Java 7 up in PIG-2643, and I think it's something we
> need to think about. When do we want to start taking advantage of new
> features that may not exist on Java 6? Do we ever?
>
> 2012/4/12 Scott Carey (Commented) (JIRA) <[EMAIL PROTECTED]>
>
>>
>> [
>> https://issues.apache.org/jira/browse/PIG-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252706#comment-13252706]
>>
>> Scott Carey commented on PIG-2643:
>> ----------------------------------
>>
>> Another thought for this sort of thing:
>>
>> This might be achievable without bytecode generation and good performance
>> with Java 7 MethodHandles [1][2]. Of course, that would require Java 7,
>> but Java 6 support ends later year [3], about the time Pig 0.11 would be
>> out anyway.
>>
>>
>> [1]
>> http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.html
>> [2]
>> http://stackoverflow.com/questions/8823793/methodhandle-what-is-it-all-about
>> [3] https://blogs.oracle.com/henrik/entry/updated_java_6_eol_date
>>
>>> Use bytecode generation to make a performance replacement for
>> InvokeForLong, InvokeForString, etc
>>>
>> -------------------------------------------------------------------------------------------------
>>>
>>> Key: PIG-2643
>>> URL: https://issues.apache.org/jira/browse/PIG-2643
>>> Project: Pig
>>> Issue Type: Improvement
>>> Reporter: Jonathan Coveney
>>> Assignee: Jonathan Coveney
>>> Priority: Minor
>>> Labels: codegen
>>> Fix For: 0.11, 0.10.1
>>>
>>> Attachments: PIG-2643-0.patch
>>>
>>>
>>> This is basically to cut my teeth for much more ambitious code
>> generation down the line, but I think it could be performance and useful.
>>> the new syntax is:
>>> {code}a = load 'thing' as (x:chararray);
>>> define concat InvokerGenerator('java.lang.String','concat','String');
>>> define valueOf InvokerGenerator('java.lang.Integer','valueOf','String');
>>> define valueOfRadix
>> InvokerGenerator('java.lang.Integer','valueOf','String,int');
>>> b = foreach a generate x, valueOf(x) as vOf;
>>> c = foreach b generate x, vOf, valueOfRadix(x, 16) as vOfR;
>>> d = foreach c generate x, vOf, vOfR, concat(concat(x, (chararray)vOf),
>> (chararray)vOfR);
>>> dump d;
>>> {code}
>>> There are some differences between this version and Dmitriy's
>> implementation:
>>> - it is no longer necessary to declare whether the method is static or
>> not. This is gleaned via reflection.
>>> - as per the above, it is no longer necessary to make the first argument
>> be the type of the object to invoke the method on. If it is not a static
>> method, then the type will implicitly be the type you need. So in the case
>> of concat, it would need to be passed a tuple of two inputs: one for the
>> method to be called against (as it is not static), and then the 'string'
>> that was specified. In the case of valueOf, because it IS static, then the
>> 'String' is the only value.
>>> - The arguments are type sensitive. Integer means the Object Integer,
>> whereas int (or long, or float, or boolean, etc) refer to the primitive.
>> This is necessary to properly reflect the arguments. Values passed in WILL,
>> however, be properly unboxed as necessary.
>>> - The return type will be reflected.
>>> This uses the ASM API to generate the bytecode, and then a custom
>> classloader to load it in. I will add caching of the generated code based
>> on the input strings, etc, but I wanted to get eyes and opinions on this. I
>> also need to benchmark, but it should be native speed (excluding a little
Alan.
On Apr 12, 2012, at 11:55 AM, Jonathan Coveney wrote:
> Scott Carey brought Java 7 up in PIG-2643, and I think it's something we
> need to think about. When do we want to start taking advantage of new
> features that may not exist on Java 6? Do we ever?
>
> 2012/4/12 Scott Carey (Commented) (JIRA) <[EMAIL PROTECTED]>
>
>>
>> [
>> https://issues.apache.org/jira/browse/PIG-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252706#comment-13252706]
>>
>> Scott Carey commented on PIG-2643:
>> ----------------------------------
>>
>> Another thought for this sort of thing:
>>
>> This might be achievable without bytecode generation and good performance
>> with Java 7 MethodHandles [1][2]. Of course, that would require Java 7,
>> but Java 6 support ends later year [3], about the time Pig 0.11 would be
>> out anyway.
>>
>>
>> [1]
>> http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.html
>> [2]
>> http://stackoverflow.com/questions/8823793/methodhandle-what-is-it-all-about
>> [3] https://blogs.oracle.com/henrik/entry/updated_java_6_eol_date
>>
>>> Use bytecode generation to make a performance replacement for
>> InvokeForLong, InvokeForString, etc
>>>
>> -------------------------------------------------------------------------------------------------
>>>
>>> Key: PIG-2643
>>> URL: https://issues.apache.org/jira/browse/PIG-2643
>>> Project: Pig
>>> Issue Type: Improvement
>>> Reporter: Jonathan Coveney
>>> Assignee: Jonathan Coveney
>>> Priority: Minor
>>> Labels: codegen
>>> Fix For: 0.11, 0.10.1
>>>
>>> Attachments: PIG-2643-0.patch
>>>
>>>
>>> This is basically to cut my teeth for much more ambitious code
>> generation down the line, but I think it could be performance and useful.
>>> the new syntax is:
>>> {code}a = load 'thing' as (x:chararray);
>>> define concat InvokerGenerator('java.lang.String','concat','String');
>>> define valueOf InvokerGenerator('java.lang.Integer','valueOf','String');
>>> define valueOfRadix
>> InvokerGenerator('java.lang.Integer','valueOf','String,int');
>>> b = foreach a generate x, valueOf(x) as vOf;
>>> c = foreach b generate x, vOf, valueOfRadix(x, 16) as vOfR;
>>> d = foreach c generate x, vOf, vOfR, concat(concat(x, (chararray)vOf),
>> (chararray)vOfR);
>>> dump d;
>>> {code}
>>> There are some differences between this version and Dmitriy's
>> implementation:
>>> - it is no longer necessary to declare whether the method is static or
>> not. This is gleaned via reflection.
>>> - as per the above, it is no longer necessary to make the first argument
>> be the type of the object to invoke the method on. If it is not a static
>> method, then the type will implicitly be the type you need. So in the case
>> of concat, it would need to be passed a tuple of two inputs: one for the
>> method to be called against (as it is not static), and then the 'string'
>> that was specified. In the case of valueOf, because it IS static, then the
>> 'String' is the only value.
>>> - The arguments are type sensitive. Integer means the Object Integer,
>> whereas int (or long, or float, or boolean, etc) refer to the primitive.
>> This is necessary to properly reflect the arguments. Values passed in WILL,
>> however, be properly unboxed as necessary.
>>> - The return type will be reflected.
>>> This uses the ASM API to generate the bytecode, and then a custom
>> classloader to load it in. I will add caching of the generated code based
>> on the input strings, etc, but I wanted to get eyes and opinions on this. I
>> also need to benchmark, but it should be native speed (excluding a little
-
Re: Java 7 and Pig, hijacked from PIG-2643Scott Carey 2012-04-16, 17:18
On 4/13/12 1:30 AM, "Gianmarco De Francisci Morales" <[EMAIL PROTECTED]>
wrote:
>Sure,
>probably Java7 is more mature than Pig.
>However as far as I know it is a pain to install on Mac.
Yes, you have to install using the developer preview, but it is available
from the same place all other JRE's from Oracle are. [1]
7u4 should be out in a few weeks [2], which will include a Mac OSX version
(Unfortunately only for Lion, and not Snow Leopard).
Oracle's official stance is that JRE 6 can go EOL 6 months after Java 7
is widely supported on the mac. They will likely wait a couple months
more than that. However, that still puts JRE 6's end this year.
[1] http://www.oracle.com/technetwork/java/javase/downloads/index.html
[2] http://jdk7.java.net/download.html (build b20, as of April 12, 2012)
>This is just one small example of not mature enough.
Availability on Mac is probably the biggest example in my opinion. What
else is there?
Most of the risky features in Java 7's JVM are already back-ported to Java
6. This leaves the new APIs and features as the biggest risks, but most
of these should reduce risk (try-with-resources, much better file system
API, better exception handling). This leaves mostly MethodHandle and
invokevirtual as the new features that are risky -- these are being
heavily tested and used by others however (e.g. JRuby).
>I am under the impression that Java7 not yet mainstream, correct me if I
>am
>wrong.
What is the definition of mainstream in this situation? Why does it
matter? I am struggling to imagine a concrete definition of mainstream
that would be useful in this context.
Other than fear of new things and a lack of a corporate backed JVM on Mac,
we have the real issue that there are significant resources required to
test such a change in Pig and its related execution and development
environments. However, this issue does not go away -- Java 6 can't be the
only supported flavor forever. Some day in the not too distant future, a
Java 7 JRE/JDK will be the default and preferred JVM in Pig's ecosystem.
It would be nice if high value features that could take advantage of Java
7 features started development early enough to be available at about the
time Java 7 was common in the ecosystem. Waiting until Java 7 is
mainstream to start such features means adding a 12 month delay into the
process.
>
>I am fine with having optional performance improvements enabled by Java7.
>+1
>I am not fine with having a dependency on Java7 for users. -1
>I am not comfortable with having a dependency on Java7 for pig developers.
>I am afraid it will put off new contributors. -0
How many potential contributors are interested in being able to try new
technologies, versus those only wanting to use what they currently know?
>
>Cheers,
>--
>Gianmarco
>
>
>
>On Fri, Apr 13, 2012 at 01:10, Scott Carey <[EMAIL PROTECTED]>
>wrote:
>
>>
>> On 4/12/12 12:02 PM, "Gianmarco De Francisci Morales" <[EMAIL PROTECTED]>
>> wrote:
>>
>> >My personal opinion is Yes, if there is a compelling reason, but not
>>now.
>> >Still not mature enough.
>>
>> What is the definition of "mature enough"?
>>
>> I'd argue that out of Java 7, Hadoop, Avro, and Pig... Java 7 is the
>>most
>> mature.
>>
>> Granted, I also think the whole concept of maturity is subjective and
>>not
>> conducive to good discussion.
>>
>>
>> What objective characteristics are required of Java 7 to support it?
>>When
>> approximately would that likely occur? Subtract 6 months from that (a
>> typical Pig dev cycle), how soon is that?
>>
>> There is a big difference between _requiring_ Java 7 and having an
>> optional feature built with Java 7 into its own jar that requires a
>>Java 7
>> cluster to use. Some performance features might fall into that
>>category.
>>
>> >
>> >Cheers,
>> >--
>> >Gianmarco
>> >
>> >
>> >
>> >On Thu, Apr 12, 2012 at 20:55, Jonathan Coveney <[EMAIL PROTECTED]>
>> >wrote:
>> >
>> >> Scott Carey brought Java 7 up in PIG-2643, and I think it's
>>something we
-
Re: Java 7 and Pig, hijacked from PIG-2643Dmitriy Ryaboy 2012-04-16, 17:28
Last time I tried to run a Java 7 jar on a java 6 jvm, things blew up.
This suggests to me that before Pig can use Java 7, Hadoop has to be
able to (since hadoop spins out the jvm process that runs the pig
map/reduce jobs). Do you know if Hadoop can run on Java 7?
D
On Mon, Apr 16, 2012 at 10:18 AM, Scott Carey <[EMAIL PROTECTED]> wrote:
>
>
> On 4/13/12 1:30 AM, "Gianmarco De Francisci Morales" <[EMAIL PROTECTED]>
> wrote:
>
>>Sure,
>>probably Java7 is more mature than Pig.
>>However as far as I know it is a pain to install on Mac.
>
> Yes, you have to install using the developer preview, but it is available
> from the same place all other JRE's from Oracle are. [1]
>
> 7u4 should be out in a few weeks [2], which will include a Mac OSX version
> (Unfortunately only for Lion, and not Snow Leopard).
>
> Oracle's official stance is that JRE 6 can go EOL 6 months after Java 7
> is widely supported on the mac. They will likely wait a couple months
> more than that. However, that still puts JRE 6's end this year.
>
> [1] http://www.oracle.com/technetwork/java/javase/downloads/index.html
> [2] http://jdk7.java.net/download.html (build b20, as of April 12, 2012)
>
>>This is just one small example of not mature enough.
>
> Availability on Mac is probably the biggest example in my opinion. What
> else is there?
>
> Most of the risky features in Java 7's JVM are already back-ported to Java
> 6. This leaves the new APIs and features as the biggest risks, but most
> of these should reduce risk (try-with-resources, much better file system
> API, better exception handling). This leaves mostly MethodHandle and
> invokevirtual as the new features that are risky -- these are being
> heavily tested and used by others however (e.g. JRuby).
>
>>I am under the impression that Java7 not yet mainstream, correct me if I
>>am
>>wrong.
>
> What is the definition of mainstream in this situation? Why does it
> matter? I am struggling to imagine a concrete definition of mainstream
> that would be useful in this context.
>
> Other than fear of new things and a lack of a corporate backed JVM on Mac,
> we have the real issue that there are significant resources required to
> test such a change in Pig and its related execution and development
> environments. However, this issue does not go away -- Java 6 can't be the
> only supported flavor forever. Some day in the not too distant future, a
> Java 7 JRE/JDK will be the default and preferred JVM in Pig's ecosystem.
>
> It would be nice if high value features that could take advantage of Java
> 7 features started development early enough to be available at about the
> time Java 7 was common in the ecosystem. Waiting until Java 7 is
> mainstream to start such features means adding a 12 month delay into the
> process.
>
>>
>>I am fine with having optional performance improvements enabled by Java7.
>>+1
>>I am not fine with having a dependency on Java7 for users. -1
>>I am not comfortable with having a dependency on Java7 for pig developers.
>>I am afraid it will put off new contributors. -0
>
> How many potential contributors are interested in being able to try new
> technologies, versus those only wanting to use what they currently know?
>
>>
>>Cheers,
>>--
>>Gianmarco
>>
>>
>>
>>On Fri, Apr 13, 2012 at 01:10, Scott Carey <[EMAIL PROTECTED]>
>>wrote:
>>
>>>
>>> On 4/12/12 12:02 PM, "Gianmarco De Francisci Morales" <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>> >My personal opinion is Yes, if there is a compelling reason, but not
>>>now.
>>> >Still not mature enough.
>>>
>>> What is the definition of "mature enough"?
>>>
>>> I'd argue that out of Java 7, Hadoop, Avro, and Pig... Java 7 is the
>>>most
>>> mature.
>>>
>>> Granted, I also think the whole concept of maturity is subjective and
>>>not
>>> conducive to good discussion.
>>>
>>>
>>> What objective characteristics are required of Java 7 to support it?
>>>When
>>> approximately would that likely occur? Subtract 6 months from that (a
>>> typical Pig dev cycle), how soon is that?
This suggests to me that before Pig can use Java 7, Hadoop has to be
able to (since hadoop spins out the jvm process that runs the pig
map/reduce jobs). Do you know if Hadoop can run on Java 7?
D
On Mon, Apr 16, 2012 at 10:18 AM, Scott Carey <[EMAIL PROTECTED]> wrote:
>
>
> On 4/13/12 1:30 AM, "Gianmarco De Francisci Morales" <[EMAIL PROTECTED]>
> wrote:
>
>>Sure,
>>probably Java7 is more mature than Pig.
>>However as far as I know it is a pain to install on Mac.
>
> Yes, you have to install using the developer preview, but it is available
> from the same place all other JRE's from Oracle are. [1]
>
> 7u4 should be out in a few weeks [2], which will include a Mac OSX version
> (Unfortunately only for Lion, and not Snow Leopard).
>
> Oracle's official stance is that JRE 6 can go EOL 6 months after Java 7
> is widely supported on the mac. They will likely wait a couple months
> more than that. However, that still puts JRE 6's end this year.
>
> [1] http://www.oracle.com/technetwork/java/javase/downloads/index.html
> [2] http://jdk7.java.net/download.html (build b20, as of April 12, 2012)
>
>>This is just one small example of not mature enough.
>
> Availability on Mac is probably the biggest example in my opinion. What
> else is there?
>
> Most of the risky features in Java 7's JVM are already back-ported to Java
> 6. This leaves the new APIs and features as the biggest risks, but most
> of these should reduce risk (try-with-resources, much better file system
> API, better exception handling). This leaves mostly MethodHandle and
> invokevirtual as the new features that are risky -- these are being
> heavily tested and used by others however (e.g. JRuby).
>
>>I am under the impression that Java7 not yet mainstream, correct me if I
>>am
>>wrong.
>
> What is the definition of mainstream in this situation? Why does it
> matter? I am struggling to imagine a concrete definition of mainstream
> that would be useful in this context.
>
> Other than fear of new things and a lack of a corporate backed JVM on Mac,
> we have the real issue that there are significant resources required to
> test such a change in Pig and its related execution and development
> environments. However, this issue does not go away -- Java 6 can't be the
> only supported flavor forever. Some day in the not too distant future, a
> Java 7 JRE/JDK will be the default and preferred JVM in Pig's ecosystem.
>
> It would be nice if high value features that could take advantage of Java
> 7 features started development early enough to be available at about the
> time Java 7 was common in the ecosystem. Waiting until Java 7 is
> mainstream to start such features means adding a 12 month delay into the
> process.
>
>>
>>I am fine with having optional performance improvements enabled by Java7.
>>+1
>>I am not fine with having a dependency on Java7 for users. -1
>>I am not comfortable with having a dependency on Java7 for pig developers.
>>I am afraid it will put off new contributors. -0
>
> How many potential contributors are interested in being able to try new
> technologies, versus those only wanting to use what they currently know?
>
>>
>>Cheers,
>>--
>>Gianmarco
>>
>>
>>
>>On Fri, Apr 13, 2012 at 01:10, Scott Carey <[EMAIL PROTECTED]>
>>wrote:
>>
>>>
>>> On 4/12/12 12:02 PM, "Gianmarco De Francisci Morales" <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>> >My personal opinion is Yes, if there is a compelling reason, but not
>>>now.
>>> >Still not mature enough.
>>>
>>> What is the definition of "mature enough"?
>>>
>>> I'd argue that out of Java 7, Hadoop, Avro, and Pig... Java 7 is the
>>>most
>>> mature.
>>>
>>> Granted, I also think the whole concept of maturity is subjective and
>>>not
>>> conducive to good discussion.
>>>
>>>
>>> What objective characteristics are required of Java 7 to support it?
>>>When
>>> approximately would that likely occur? Subtract 6 months from that (a
>>> typical Pig dev cycle), how soon is that?
-
Re: Java 7 and Pig, hijacked from PIG-2643Scott Carey 2012-04-16, 17:46
We are starting to test it internally in our cluster.
This is a broader hadoop issue. Note, Oracle's last public (free) release
of a Java 6 JRE will be towards the end of this year. After that, even
security updates will ONLY be available in a Java 7 JRE.
On 4/16/12 10:28 AM, "Dmitriy Ryaboy" <[EMAIL PROTECTED]> wrote:
>Last time I tried to run a Java 7 jar on a java 6 jvm, things blew up.
>This suggests to me that before Pig can use Java 7, Hadoop has to be
>able to (since hadoop spins out the jvm process that runs the pig
>map/reduce jobs). Do you know if Hadoop can run on Java 7?
>
>D
>
>On Mon, Apr 16, 2012 at 10:18 AM, Scott Carey <[EMAIL PROTECTED]>
>wrote:
>>
>>
>> On 4/13/12 1:30 AM, "Gianmarco De Francisci Morales" <[EMAIL PROTECTED]>
>> wrote:
>>
>>>Sure,
>>>probably Java7 is more mature than Pig.
>>>However as far as I know it is a pain to install on Mac.
>>
>> Yes, you have to install using the developer preview, but it is
>>available
>> from the same place all other JRE's from Oracle are. [1]
>>
>> 7u4 should be out in a few weeks [2], which will include a Mac OSX
>>version
>> (Unfortunately only for Lion, and not Snow Leopard).
>>
>> Oracle's official stance is that JRE 6 can go EOL 6 months after Java 7
>> is widely supported on the mac. They will likely wait a couple months
>> more than that. However, that still puts JRE 6's end this year.
>>
>> [1] http://www.oracle.com/technetwork/java/javase/downloads/index.html
>> [2] http://jdk7.java.net/download.html (build b20, as of April 12, 2012)
>>
>>>This is just one small example of not mature enough.
>>
>> Availability on Mac is probably the biggest example in my opinion. What
>> else is there?
>>
>> Most of the risky features in Java 7's JVM are already back-ported to
>>Java
>> 6. This leaves the new APIs and features as the biggest risks, but most
>> of these should reduce risk (try-with-resources, much better file system
>> API, better exception handling). This leaves mostly MethodHandle and
>> invokevirtual as the new features that are risky -- these are being
>> heavily tested and used by others however (e.g. JRuby).
>>
>>>I am under the impression that Java7 not yet mainstream, correct me if I
>>>am
>>>wrong.
>>
>> What is the definition of mainstream in this situation? Why does it
>> matter? I am struggling to imagine a concrete definition of mainstream
>> that would be useful in this context.
>>
>> Other than fear of new things and a lack of a corporate backed JVM on
>>Mac,
>> we have the real issue that there are significant resources required to
>> test such a change in Pig and its related execution and development
>> environments. However, this issue does not go away -- Java 6 can't be
>>the
>> only supported flavor forever. Some day in the not too distant future, a
>> Java 7 JRE/JDK will be the default and preferred JVM in Pig's ecosystem.
>>
>> It would be nice if high value features that could take advantage of
>>Java
>> 7 features started development early enough to be available at about the
>> time Java 7 was common in the ecosystem. Waiting until Java 7 is
>> mainstream to start such features means adding a 12 month delay into the
>> process.
>>
>>>
>>>I am fine with having optional performance improvements enabled by
>>>Java7.
>>>+1
>>>I am not fine with having a dependency on Java7 for users. -1
>>>I am not comfortable with having a dependency on Java7 for pig
>>>developers.
>>>I am afraid it will put off new contributors. -0
>>
>> How many potential contributors are interested in being able to try new
>> technologies, versus those only wanting to use what they currently know?
>>
>>>
>>>Cheers,
>>>--
>>>Gianmarco
>>>
>>>
>>>
>>>On Fri, Apr 13, 2012 at 01:10, Scott Carey <[EMAIL PROTECTED]>
>>>wrote:
>>>
>>>>
>>>> On 4/12/12 12:02 PM, "Gianmarco De Francisci Morales"
>>>><[EMAIL PROTECTED]>
>>>> wrote:
>>>>
>>>> >My personal opinion is Yes, if there is a compelling reason, but not
>>>>now.
>>>> >Still not mature enough.
>>>
This is a broader hadoop issue. Note, Oracle's last public (free) release
of a Java 6 JRE will be towards the end of this year. After that, even
security updates will ONLY be available in a Java 7 JRE.
On 4/16/12 10:28 AM, "Dmitriy Ryaboy" <[EMAIL PROTECTED]> wrote:
>Last time I tried to run a Java 7 jar on a java 6 jvm, things blew up.
>This suggests to me that before Pig can use Java 7, Hadoop has to be
>able to (since hadoop spins out the jvm process that runs the pig
>map/reduce jobs). Do you know if Hadoop can run on Java 7?
>
>D
>
>On Mon, Apr 16, 2012 at 10:18 AM, Scott Carey <[EMAIL PROTECTED]>
>wrote:
>>
>>
>> On 4/13/12 1:30 AM, "Gianmarco De Francisci Morales" <[EMAIL PROTECTED]>
>> wrote:
>>
>>>Sure,
>>>probably Java7 is more mature than Pig.
>>>However as far as I know it is a pain to install on Mac.
>>
>> Yes, you have to install using the developer preview, but it is
>>available
>> from the same place all other JRE's from Oracle are. [1]
>>
>> 7u4 should be out in a few weeks [2], which will include a Mac OSX
>>version
>> (Unfortunately only for Lion, and not Snow Leopard).
>>
>> Oracle's official stance is that JRE 6 can go EOL 6 months after Java 7
>> is widely supported on the mac. They will likely wait a couple months
>> more than that. However, that still puts JRE 6's end this year.
>>
>> [1] http://www.oracle.com/technetwork/java/javase/downloads/index.html
>> [2] http://jdk7.java.net/download.html (build b20, as of April 12, 2012)
>>
>>>This is just one small example of not mature enough.
>>
>> Availability on Mac is probably the biggest example in my opinion. What
>> else is there?
>>
>> Most of the risky features in Java 7's JVM are already back-ported to
>>Java
>> 6. This leaves the new APIs and features as the biggest risks, but most
>> of these should reduce risk (try-with-resources, much better file system
>> API, better exception handling). This leaves mostly MethodHandle and
>> invokevirtual as the new features that are risky -- these are being
>> heavily tested and used by others however (e.g. JRuby).
>>
>>>I am under the impression that Java7 not yet mainstream, correct me if I
>>>am
>>>wrong.
>>
>> What is the definition of mainstream in this situation? Why does it
>> matter? I am struggling to imagine a concrete definition of mainstream
>> that would be useful in this context.
>>
>> Other than fear of new things and a lack of a corporate backed JVM on
>>Mac,
>> we have the real issue that there are significant resources required to
>> test such a change in Pig and its related execution and development
>> environments. However, this issue does not go away -- Java 6 can't be
>>the
>> only supported flavor forever. Some day in the not too distant future, a
>> Java 7 JRE/JDK will be the default and preferred JVM in Pig's ecosystem.
>>
>> It would be nice if high value features that could take advantage of
>>Java
>> 7 features started development early enough to be available at about the
>> time Java 7 was common in the ecosystem. Waiting until Java 7 is
>> mainstream to start such features means adding a 12 month delay into the
>> process.
>>
>>>
>>>I am fine with having optional performance improvements enabled by
>>>Java7.
>>>+1
>>>I am not fine with having a dependency on Java7 for users. -1
>>>I am not comfortable with having a dependency on Java7 for pig
>>>developers.
>>>I am afraid it will put off new contributors. -0
>>
>> How many potential contributors are interested in being able to try new
>> technologies, versus those only wanting to use what they currently know?
>>
>>>
>>>Cheers,
>>>--
>>>Gianmarco
>>>
>>>
>>>
>>>On Fri, Apr 13, 2012 at 01:10, Scott Carey <[EMAIL PROTECTED]>
>>>wrote:
>>>
>>>>
>>>> On 4/12/12 12:02 PM, "Gianmarco De Francisci Morales"
>>>><[EMAIL PROTECTED]>
>>>> wrote:
>>>>
>>>> >My personal opinion is Yes, if there is a compelling reason, but not
>>>>now.
>>>> >Still not mature enough.
>>>
