Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Possible Pig 9.1 globing bug in parameter substitution


Copy link to this message
-
RE: Possible Pig 9.1 globing bug in parameter substitution
If
  -param input=s3n://foo/bar/baz/*/ blah.pig
is part of a command line, you'd have to add quotes:
  -param 'input=s3n://foo/bar/baz/*/' blah.pig
to inhibit your shell from trying to interpret the *.
William F Dowling
Senior Technologist
Thomson Reuters
0 +1 215 823 3853
-----Original Message-----
From: Ayon Sinha [mailto:[EMAIL PROTECTED]]
Sent: Thursday, December 15, 2011 2:18 PM
To: Pig Mailinglist
Subject: Possible Pig 9.1 globing bug in parameter substitution

when using -param input=s3n://foo/bar/baz/*/ blah.pig
it throws 

java.lang.NullPointerException
at org.apache.pig.tools.parameters.ParameterSubstitutionPreprocessor.genSubstitutedFile(ParameterSubstitutionPreprocessor.java:79)
at org.apache.pig.Main.runParamPreprocessor(Main.java:710)
at org.apache.pig.Main.run(Main.java:517)
at org.apache.pig.Main.main(Main.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

It works when my load statement is changed from:
a = load '$input' using PigStorage();

to

a = load 's3n://foo/bar/baz/*/' using PigStorage();

(I'm under a deadline so can't file a JIRA bug rightaway)
 
-Ayon
See My Photos on Flickr
Also check out my Blog for answers to commonly asked questions.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB