Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Fixing a broken dependency // can we include a patched piece of JRuby source code in Pig?

Copy link to this message
Re: Fixing a broken dependency // can we include a patched piece of JRuby source code in Pig?
One confusion I have is do we really need require "pig.jar"?
PIG-2317-8.patch+PIG-2317-8_plus.patch works for me last time, and I
drop require "pig.jar". Did we change anything in PIG-2317-9.patch?


On Fri, Mar 23, 2012 at 2:50 PM, Jonathan Coveney <[EMAIL PROTECTED]> wrote:
> Alan,
> My idea wasn't to have a patched version included with Pig, just one
> specific file that needed to be patched, that way it would be earlier in
> the classpath than the version in JRuby. If you think it is cleaner to just
> have a patched version of JRuby that we depend on, it would accomplish the
> same thing. But I definitely agree that we shouldn't include JRuby itself
> with Pig. But right now I have a file at src/org/jruby/path/to/file.java,
> and since there will be this lone JRuby file, in the case that it gets
> called, we are good
> Daniel,
> The issue only cropped up because, in a sense, another bug was hiding it. I
> thought 1.9 was the default version of Ruby that JRuby intepreted, but I
> was wrong (1.8 was the default). I fixed this, and then the issue cropped
> up.
> For people who aren't rubyists and don't know much about Ruby versions:
> 1.8.7 is the equivalent of Python 2.7, and 1.9.2 is the equivalent of
> Python 3.3, the difference being that most everyone with new projects uses
> 1.9.2 because it is better in many ways, and didn't break the language like
> Python did (this is where the analogy breaks down). Another way to think of
> it might be that we want to use Pig 0.9 (1.9) instead of Pig 0.8 (1.8),
> especially because 1.9 is getting all of the active development, so in the
> future most bug fixes are going to be for 1.9 etc. That said, the 1.9 piece
> has this one bug (much as people using Pig 0.9 instead of Pig 0.8 ran into
> some weird bugs b/c of the parser changes).
> The bug is as follows:
> If you have a file on your classpath, and then "require" a resource with
> that exact same name, you get a String index error. This crops up when,
> internally to Pig, it loads the pigudf.rb file into the ruby runtime on the
> same JVM. pig.jar is on the classpath, but the script has to 'require
> pig.jar' because it needs to know about things like DataBags and whatnot.
> This causes an error because of a subtle bug that rarely comes up in pure
> JRuby projects, since pure JRuby projects usually don't set the classpath
> on the jvm level. This is a pretty difficult bug to workaround: pig.jar has
> to be on the classpath, and requiring it in ruby is also necessary from
> within pig.
> Either way, the fix is trivial, but 1.7.0 won't be out for a month or two,
> and in the meantime, it's pretty bad. I can try to come up with some weird
> hack workaround, but depending on a fork of JRuby seems like it would be a
> lot more reasonable. Having the one .java file that needs to be fixed is
> also easy, but might be ugly. I personally would really prefer to just go
> with the fixed version, because the workaround is going to be pretty
> difficult (since the error is baked deep in there) and ugly. If we have a
> forked dependency, when 1.7.0 comes out we can just depend on that and bam.
> Sorry if that's a bit long, I just wanted to thoroughly document the issue.
> Jon
> 2012/3/23 Alan Gates <[EMAIL PROTECTED]>
>> Won't a lot of people already have their version of JRuby and not want a
>> special one?  I'm fine with having a patched version on github and
>> referring it in our release notes.  I'm not wild about including a version
>> of JRuby with Pig, for both licensing reasons and because our tar file is
>> bloated enough as it is.
>> Alan.
>> On Mar 23, 2012, at 11:38 AM, Daniel Dai wrote:
>> > Hi, Jonathan,
>> > What bug is it? Last time when I try, it seems work well for me. We
>> > can leave a small hole and describe the limitation clearly in release
>> > notes/code comments/javadocs, we can also provide a link to the ticket
>> > tracking the issue. I remember we did something similar for javacc