Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Possible Pig 9.1 globing bug in parameter substitution


+
Ayon Sinha 2011-12-15, 19:18
Copy link to this message
-
RE: Possible Pig 9.1 globing bug in parameter substitution
If
  -param input=s3n://foo/bar/baz/*/ blah.pig
is part of a command line, you'd have to add quotes:
  -param 'input=s3n://foo/bar/baz/*/' blah.pig
to inhibit your shell from trying to interpret the *.
William F Dowling
Senior Technologist
Thomson Reuters
0 +1 215 823 3853
-----Original Message-----
From: Ayon Sinha [mailto:[EMAIL PROTECTED]]
Sent: Thursday, December 15, 2011 2:18 PM
To: Pig Mailinglist
Subject: Possible Pig 9.1 globing bug in parameter substitution

when using -param input=s3n://foo/bar/baz/*/ blah.pig
it throws 

java.lang.NullPointerException
at org.apache.pig.tools.parameters.ParameterSubstitutionPreprocessor.genSubstitutedFile(ParameterSubstitutionPreprocessor.java:79)
at org.apache.pig.Main.runParamPreprocessor(Main.java:710)
at org.apache.pig.Main.run(Main.java:517)
at org.apache.pig.Main.main(Main.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

It works when my load statement is changed from:
a = load '$input' using PigStorage();

to

a = load 's3n://foo/bar/baz/*/' using PigStorage();

(I'm under a deadline so can't file a JIRA bug rightaway)
 
-Ayon
See My Photos on Flickr
Also check out my Blog for answers to commonly asked questions.
+
Corbin Hoenes 2011-12-27, 23:35
+
Aniket Mokashi 2011-12-28, 01:15
+
Corbin Hoenes 2011-12-28, 19:27