Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Possible Pig 9.1 globing bug in parameter substitution


Copy link to this message
-
Re: Possible Pig 9.1 globing bug in parameter substitution
I tried
pig --param "input=s3n://bucket_path/*/" test.pig

It worked for me. I am on EMR Pig 0.9.1.

Thanks,
Aniket

On Tue, Dec 27, 2011 at 3:35 PM, Corbin Hoenes <[EMAIL PROTECTED]> wrote:

> I am not sure Ayon doesn't have something here.  I am seeing a similar
> problem with the 0.9.1 build of pig.  But when I run with 0.9.0 it doesn't
> have that problem.
>
> Did something with pattern substitution change from 0.9.0 --> 0.9.1?
>  Haven't run it through a debugger yet but that is the next step tomorrow
> if someone doesn't know of some patch I'm missing?
>
> On Dec 15, 2011, at 12:25 PM, <[EMAIL PROTECTED]> <
> [EMAIL PROTECTED]> wrote:
>
> > If
> >  -param input=s3n://foo/bar/baz/*/ blah.pig
> > is part of a command line, you'd have to add quotes:
> >  -param 'input=s3n://foo/bar/baz/*/' blah.pig
> > to inhibit your shell from trying to interpret the *.
> >
> >
> > William F Dowling
> > Senior Technologist
> > Thomson Reuters
> > 0 +1 215 823 3853
> >
> >
> > -----Original Message-----
> > From: Ayon Sinha [mailto:[EMAIL PROTECTED]]
> > Sent: Thursday, December 15, 2011 2:18 PM
> > To: Pig Mailinglist
> > Subject: Possible Pig 9.1 globing bug in parameter substitution
> >
> > when using -param input=s3n://foo/bar/baz/*/ blah.pig
> > it throws
> >
> > java.lang.NullPointerException
> > at
> org.apache.pig.tools.parameters.ParameterSubstitutionPreprocessor.genSubstitutedFile(ParameterSubstitutionPreprocessor.java:79)
> > at org.apache.pig.Main.runParamPreprocessor(Main.java:710)
> > at org.apache.pig.Main.run(Main.java:517)
> > at org.apache.pig.Main.main(Main.java:108)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> >
> > It works when my load statement is changed from:
> > a = load '$input' using PigStorage();
> >
> > to
> >
> > a = load 's3n://foo/bar/baz/*/' using PigStorage();
> >
> > (I'm under a deadline so can't file a JIRA bug rightaway)
> >
> > -Ayon
> > See My Photos on Flickr
> > Also check out my Blog for answers to commonly asked questions.
>
>
--
"...:::Aniket:::... Quetzalco@tl"
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB