Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - PigServer and dynamic invokers


Copy link to this message
-
Re: PigServer and dynamic invokers
Stan Rosenberg 2011-11-17, 04:21
Hi Dimitriy,

The script does run if invoked from command line but only if we set
PIG_CLASSPATH to point at the jar.

stan
On Nov 16, 2011 11:18 PM, "Dmitriy Ryaboy" <[EMAIL PROTECTED]> wrote:

> Does the script run if you launch it from the pig command line instead
> of via PigServer?
>
> On Wed, Nov 16, 2011 at 3:01 PM, Charles Menguy
> <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > We're trying to run a PIG script using the PigServer API in Java, but
> we're
> > having a couple issues.
> >
> > It seems to work well in most cases, but in our case we need to use Pig's
> > dynamic invokers. Basically this looks something like the following:
> > DEFINE isEmail InvokeForInt('com.company.pig.udf.Utils.isEmail',
> 'String');
> >
> > We register the script using PigServer like below:
> > pigServer.registerScript(pigScript, params);
> >
> > The problem is that PigServer doesn't seem to like these dynamic
> invokers.
> > We get an error during the parsing of the script by PigServer. We've
> > checked and the classpath is fine, and the UDF is defined correctly with
> > the right signature. Any help or advice would be very much appreciated on
> > this. I have attached the stack trace below for more details.
> >
> > Thanks,
> >
> > Charles
> >
> > =======================> >
> > Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
> 1000:
> > Error during parsing. could not instantiate 'InvokeForInt' with arguments
> > '[com.company.pig.udf.Utils.isEmail, String]'
> > at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1618)
> > at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1562)
> > at org.apache.pig.PigServer.registerQuery(PigServer.java:534)
> > at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:868)
> > at
> >
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:388)
> > at
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
> > at org.apache.pig.PigServer.registerScript(PigServer.java:619)
> > at org.apache.pig.PigServer.registerScript(PigServer.java:577)
> > at com.proclivitysystems.etl.job.PIGJobRunner.run(PIGJobRunner.java:96)
> > ... 2 more
> > Caused by: org.apache.pig.impl.logicalLayer.parser.ParseException: could
> > not instantiate 'InvokeForInt' with arguments
> > '[com.company.pig.udf.Utils.isEmail, String]'
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.EvalFuncSpec(QueryParser.java:5465)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser.java:5291)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.java:5187)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.java:5133)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(QueryParser.java:5042)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser.java:4968)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.InfixExpr(QueryParser.java:4934)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.PUnaryCond(QueryParser.java:2160)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.PAndCond(QueryParser.java:2096)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.POrCond(QueryParser.java:2040)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.PCond(QueryParser.java:2006)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BinCond(QueryParser.java:5406)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser.java:5300)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.java:5187)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.java:5133)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(QueryParser.java:5042)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser.java:4968)