Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Possible Pig 9.1 globing bug in parameter substitution


+
Ayon Sinha 2011-12-15, 19:18
+
william.dowling@... 2011-12-15, 19:25
+
Corbin Hoenes 2011-12-27, 23:35
Copy link to this message
-
Re: Possible Pig 9.1 globing bug in parameter substitution
Aniket Mokashi 2011-12-28, 01:15
I tried
pig --param "input=s3n://bucket_path/*/" test.pig

It worked for me. I am on EMR Pig 0.9.1.

Thanks,
Aniket

On Tue, Dec 27, 2011 at 3:35 PM, Corbin Hoenes <[EMAIL PROTECTED]> wrote:

> I am not sure Ayon doesn't have something here.  I am seeing a similar
> problem with the 0.9.1 build of pig.  But when I run with 0.9.0 it doesn't
> have that problem.
>
> Did something with pattern substitution change from 0.9.0 --> 0.9.1?
>  Haven't run it through a debugger yet but that is the next step tomorrow
> if someone doesn't know of some patch I'm missing?
>
> On Dec 15, 2011, at 12:25 PM, <[EMAIL PROTECTED]> <
> [EMAIL PROTECTED]> wrote:
>
> > If
> >  -param input=s3n://foo/bar/baz/*/ blah.pig
> > is part of a command line, you'd have to add quotes:
> >  -param 'input=s3n://foo/bar/baz/*/' blah.pig
> > to inhibit your shell from trying to interpret the *.
> >
> >
> > William F Dowling
> > Senior Technologist
> > Thomson Reuters
> > 0 +1 215 823 3853
> >
> >
> > -----Original Message-----
> > From: Ayon Sinha [mailto:[EMAIL PROTECTED]]
> > Sent: Thursday, December 15, 2011 2:18 PM
> > To: Pig Mailinglist
> > Subject: Possible Pig 9.1 globing bug in parameter substitution
> >
> > when using -param input=s3n://foo/bar/baz/*/ blah.pig
> > it throws
> >
> > java.lang.NullPointerException
> > at
> org.apache.pig.tools.parameters.ParameterSubstitutionPreprocessor.genSubstitutedFile(ParameterSubstitutionPreprocessor.java:79)
> > at org.apache.pig.Main.runParamPreprocessor(Main.java:710)
> > at org.apache.pig.Main.run(Main.java:517)
> > at org.apache.pig.Main.main(Main.java:108)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> >
> > It works when my load statement is changed from:
> > a = load '$input' using PigStorage();
> >
> > to
> >
> > a = load 's3n://foo/bar/baz/*/' using PigStorage();
> >
> > (I'm under a deadline so can't file a JIRA bug rightaway)
> >
> > -Ayon
> > See My Photos on Flickr
> > Also check out my Blog for answers to commonly asked questions.
>
>
--
"...:::Aniket:::... Quetzalco@tl"
+
Corbin Hoenes 2011-12-28, 19:27