Hi,

We're trying to run a PIG script using the PigServer API in Java, but we're
having a couple issues.

It seems to work well in most cases, but in our case we need to use Pig's
dynamic invokers. Basically this looks something like the following:
DEFINE isEmail InvokeForInt('com.company.pig.udf.Utils.isEmail', 'String');

We register the script using PigServer like below:
pigServer.registerScript(pigScript, params);

The problem is that PigServer doesn't seem to like these dynamic invokers.
We get an error during the parsing of the script by PigServer. We've
checked and the classpath is fine, and the UDF is defined correctly with
the right signature. Any help or advice would be very much appreciated on
this. I have attached the stack trace below for more details.

Thanks,

Charles

=======================
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000:
Error during parsing. could not instantiate 'InvokeForInt' with arguments
'[com.company.pig.udf.Utils.isEmail, String]'
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1618)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1562)
at org.apache.pig.PigServer.registerQuery(PigServer.java:534)
at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:868)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:388)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
at org.apache.pig.PigServer.registerScript(PigServer.java:619)
at org.apache.pig.PigServer.registerScript(PigServer.java:577)
at com.proclivitysystems.etl.job.PIGJobRunner.run(PIGJobRunner.java:96)
... 2 more
Caused by: org.apache.pig.impl.logicalLayer.parser.ParseException: could
not instantiate 'InvokeForInt' with arguments
'[com.company.pig.udf.Utils.isEmail, String]'
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.EvalFuncSpec(QueryParser.java:5465)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser.java:5291)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.java:5187)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.java:5133)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(QueryParser.java:5042)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser.java:4968)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.InfixExpr(QueryParser.java:4934)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.PUnaryCond(QueryParser.java:2160)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.PAndCond(QueryParser.java:2096)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.POrCond(QueryParser.java:2040)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.PCond(QueryParser.java:2006)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.BinCond(QueryParser.java:5406)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser.java:5300)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.java:5187)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.java:5133)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(QueryParser.java:5042)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser.java:4968)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.InfixExpr(QueryParser.java:4934)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.NestedCommand(QueryParser.java:4117)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.NestedBlock(QueryParser.java:4045)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3433)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1464)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:1013)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:800)
at
org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1612)
... 10 more
Caused by: java.lang.RuntimeException: could not instantiate 'InvokeForInt'
with arguments '[com.company.pig.udf.Utils.isEmail, String]'
at
org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502)
at
org.apache.pig.impl.PigContext.instantiateFuncFromAlias(PigContext.java:529)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.EvalFuncSpec(QueryParser.java:5462)
... 35 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at
org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:470)
... 37 more

Does the script run if you launch it from the pig command line instead
of via PigServer?

On Wed, Nov 16, 2011 at 3:01 PM, Charles Menguy
<[EMAIL PROTECTED]> wrote:
> Hi,
>
> We're trying to run a PIG script using the PigServer API in Java, but we're
> having a couple issues.
>
> It seems to work well in most cases, but in our case we need to use Pig's
> dynamic invokers. Basically this looks something like the following:
> DEFINE isEmail InvokeForInt('com.company.pig.udf.Utils.isEmail', 'String');
>
> We register the script using PigServer like below:
> pigServer.registerScript(pigScript, params);
>
> The problem is that PigServer doesn't seem to like these dynamic invokers.
> We get an error during the parsing of the script by PigServer. We've
> checked and the classpath is fine, and the UDF is defined correctly with
> the right signature. Any help or advice would be very much appreciated on
> this. I have attached the stack trace below for more details.
>
> Thanks,
>
> Charles
>
> =======================>
> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000:
> Error during parsing. could not instantiate 'InvokeForInt' with arguments
> '[com.company.pig.udf.Utils.isEmail, String]'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1618)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1562)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:534)
> at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:868)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:388)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
> at org.apache.pig.PigServer.registerScript(PigServer.java:619)
> at org.apache.pig.PigServer.registerScript(PigServer.java:577)
> at com.proclivitysystems.etl.job.PIGJobRunner.run(PIGJobRunner.java:96)
> ... 2 more
> Caused by: org.apache.pig.impl.logicalLayer.parser.ParseException: could
> not instantiate 'InvokeForInt' with arguments
> '[com.company.pig.udf.Utils.isEmail, String]'
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.EvalFuncSpec(QueryParser.java:5465)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser.java:5291)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.java:5187)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.java:5133)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(QueryParser.java:5042)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser.java:4968)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.InfixExpr(QueryParser.java:4934)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.PUnaryCond(QueryParser.java:2160)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.PAndCond(QueryParser.java:2096)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.POrCond(QueryParser.java:2040)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.PCond(QueryParser.java:2006)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BinCond(QueryParser.java:5406)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser.java:5300)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.java:5187)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.java:5133)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(QueryParser.java:5042)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser.java:4968)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.InfixExpr(QueryParser.java:4934)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.NestedCommand(QueryParser.java:4117)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.NestedBlock(QueryParser.java:4045)
> at
> org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3433)

Hi Dimitriy,

The script does run if invoked from command line but only if we set
PIG_CLASSPATH to point at the jar.

stan
On Nov 16, 2011 11:18 PM, "Dmitriy Ryaboy" <[EMAIL PROTECTED]> wrote:

> Does the script run if you launch it from the pig command line instead
> of via PigServer?
>
> On Wed, Nov 16, 2011 at 3:01 PM, Charles Menguy
> <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > We're trying to run a PIG script using the PigServer API in Java, but
> we're
> > having a couple issues.
> >
> > It seems to work well in most cases, but in our case we need to use Pig's
> > dynamic invokers. Basically this looks something like the following:
> > DEFINE isEmail InvokeForInt('com.company.pig.udf.Utils.isEmail',
> 'String');
> >
> > We register the script using PigServer like below:
> > pigServer.registerScript(pigScript, params);
> >
> > The problem is that PigServer doesn't seem to like these dynamic
> invokers.
> > We get an error during the parsing of the script by PigServer. We've
> > checked and the classpath is fine, and the UDF is defined correctly with
> > the right signature. Any help or advice would be very much appreciated on
> > this. I have attached the stack trace below for more details.
> >
> > Thanks,
> >
> > Charles
> >
> > =======================> >
> > Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
> 1000:
> > Error during parsing. could not instantiate 'InvokeForInt' with arguments
> > '[com.company.pig.udf.Utils.isEmail, String]'
> > at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1618)
> > at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1562)
> > at org.apache.pig.PigServer.registerQuery(PigServer.java:534)
> > at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:868)
> > at
> >
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:388)
> > at
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
> > at org.apache.pig.PigServer.registerScript(PigServer.java:619)
> > at org.apache.pig.PigServer.registerScript(PigServer.java:577)
> > at com.proclivitysystems.etl.job.PIGJobRunner.run(PIGJobRunner.java:96)
> > ... 2 more
> > Caused by: org.apache.pig.impl.logicalLayer.parser.ParseException: could
> > not instantiate 'InvokeForInt' with arguments
> > '[com.company.pig.udf.Utils.isEmail, String]'
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.EvalFuncSpec(QueryParser.java:5465)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser.java:5291)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.java:5187)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.java:5133)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(QueryParser.java:5042)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser.java:4968)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.InfixExpr(QueryParser.java:4934)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.PUnaryCond(QueryParser.java:2160)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.PAndCond(QueryParser.java:2096)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.POrCond(QueryParser.java:2040)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.PCond(QueryParser.java:2006)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BinCond(QueryParser.java:5406)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser.java:5300)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.java:5187)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.java:5133)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(QueryParser.java:5042)
> > at
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser.java:4968)

Can you try applying the patch below to your version of pig?

diff --git src/org/apache/pig/builtin/Invoker.java
src/org/apache/pig/builtin/Invoker.java
index adfc04a..8150165 100644
--- src/org/apache/pig/builtin/Invoker.java
+++ src/org/apache/pig/builtin/Invoker.java
@@ -31,6 +31,7 @@ import org.apache.commons.logging.LogFactory;
 import org.apache.pig.backend.executionengine.ExecException;
 import org.apache.pig.data.DataBag;
 import org.apache.pig.data.Tuple;
+import org.apache.pig.impl.PigContext;
 import org.apache.pig.impl.logicalLayer.FrontendException;

 import com.google.common.collect.Lists;
@@ -67,7 +68,14 @@ public class Invoker<T>  {
     throws ClassNotFoundException, FrontendException,
SecurityException, NoSuchMethodException {
         String className = fullName.substring(0, fullName.lastIndexOf('.'));
         String methodName = fullName.substring(fullName.lastIndexOf('.')+1);
-        Class<?> klazz = Class.forName(className);
+        Class<?> klazz;
+        try {
+            klazz = PigContext.resolveClassName(className);
+        } catch (IOException e) {
+            // the amusing part is that PigContext throws this to wrap one of
+            // the exceptions we declare!
+            throw new FrontendException(e);
+        }
         String[] paramSpecs = "".equals(paramSpecsStr) ? new
String[0] : paramSpecsStr.split(" ");
         isStatic_ = "static".equalsIgnoreCase(isStatic) ||
"true".equals(isStatic);
         paramClasses_ = new Class<?>[paramSpecs.length];

On Wed, Nov 16, 2011 at 8:21 PM, Stan Rosenberg
<[EMAIL PROTECTED]> wrote:
> Hi Dimitriy,
>
> The script does run if invoked from command line but only if we set
> PIG_CLASSPATH to point at the jar.
>
> stan
> On Nov 16, 2011 11:18 PM, "Dmitriy Ryaboy" <[EMAIL PROTECTED]> wrote:
>
>> Does the script run if you launch it from the pig command line instead
>> of via PigServer?
>>
>> On Wed, Nov 16, 2011 at 3:01 PM, Charles Menguy
>> <[EMAIL PROTECTED]> wrote:
>> > Hi,
>> >
>> > We're trying to run a PIG script using the PigServer API in Java, but
>> we're
>> > having a couple issues.
>> >
>> > It seems to work well in most cases, but in our case we need to use Pig's
>> > dynamic invokers. Basically this looks something like the following:
>> > DEFINE isEmail InvokeForInt('com.company.pig.udf.Utils.isEmail',
>> 'String');
>> >
>> > We register the script using PigServer like below:
>> > pigServer.registerScript(pigScript, params);
>> >
>> > The problem is that PigServer doesn't seem to like these dynamic
>> invokers.
>> > We get an error during the parsing of the script by PigServer. We've
>> > checked and the classpath is fine, and the UDF is defined correctly with
>> > the right signature. Any help or advice would be very much appreciated on
>> > this. I have attached the stack trace below for more details.
>> >
>> > Thanks,
>> >
>> > Charles
>> >
>> > =======================>> >
>> > Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
>> 1000:
>> > Error during parsing. could not instantiate 'InvokeForInt' with arguments
>> > '[com.company.pig.udf.Utils.isEmail, String]'
>> > at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1618)
>> > at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1562)
>> > at org.apache.pig.PigServer.registerQuery(PigServer.java:534)
>> > at
>> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:868)
>> > at
>> >
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:388)
>> > at
>> >
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
>> > at org.apache.pig.PigServer.registerScript(PigServer.java:619)
>> > at org.apache.pig.PigServer.registerScript(PigServer.java:577)
>> > at com.proclivitysystems.etl.job.PIGJobRunner.run(PIGJobRunner.java:96)
>> > ... 2 more
>> > Caused by: org.apache.pig.impl.logicalLayer.parser.ParseException: could
>> > not instantiate 'InvokeForInt' with arguments

Thank you very much for the patch Dmitry.

This seems to fix the issue when we run PIG from the command line, we no
longer have to add this jar in the PIG_CLASSPATH, if we only register it it
will work as expected.

However, after pointing to the right patched PIG jar, we're still having
issues using it from PigServer, basically the same issue and same stack
trace.

Do you know of any other workarounds to use dynamic invokers with
PigServer, or we should refrain from using dynamic invokers with PigServer
for now?

Thanks,

Charles

On Thu, Nov 17, 2011 at 12:17 AM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> Can you try applying the patch below to your version of pig?
>
> diff --git src/org/apache/pig/builtin/Invoker.java
> src/org/apache/pig/builtin/Invoker.java
> index adfc04a..8150165 100644
> --- src/org/apache/pig/builtin/Invoker.java
> +++ src/org/apache/pig/builtin/Invoker.java
> @@ -31,6 +31,7 @@ import org.apache.commons.logging.LogFactory;
>  import org.apache.pig.backend.executionengine.ExecException;
>  import org.apache.pig.data.DataBag;
>  import org.apache.pig.data.Tuple;
> +import org.apache.pig.impl.PigContext;
>  import org.apache.pig.impl.logicalLayer.FrontendException;
>
>  import com.google.common.collect.Lists;
> @@ -67,7 +68,14 @@ public class Invoker<T>  {
>     throws ClassNotFoundException, FrontendException,
> SecurityException, NoSuchMethodException {
>         String className = fullName.substring(0,
> fullName.lastIndexOf('.'));
>         String methodName > fullName.substring(fullName.lastIndexOf('.')+1);
> -        Class<?> klazz = Class.forName(className);
> +        Class<?> klazz;
> +        try {
> +            klazz = PigContext.resolveClassName(className);
> +        } catch (IOException e) {
> +            // the amusing part is that PigContext throws this to wrap
> one of
> +            // the exceptions we declare!
> +            throw new FrontendException(e);
> +        }
>         String[] paramSpecs = "".equals(paramSpecsStr) ? new
> String[0] : paramSpecsStr.split(" ");
>         isStatic_ = "static".equalsIgnoreCase(isStatic) ||
> "true".equals(isStatic);
>         paramClasses_ = new Class<?>[paramSpecs.length];
>
> On Wed, Nov 16, 2011 at 8:21 PM, Stan Rosenberg
> <[EMAIL PROTECTED]> wrote:
> > Hi Dimitriy,
> >
> > The script does run if invoked from command line but only if we set
> > PIG_CLASSPATH to point at the jar.
> >
> > stan
> > On Nov 16, 2011 11:18 PM, "Dmitriy Ryaboy" <[EMAIL PROTECTED]> wrote:
> >
> >> Does the script run if you launch it from the pig command line instead
> >> of via PigServer?
> >>
> >> On Wed, Nov 16, 2011 at 3:01 PM, Charles Menguy
> >> <[EMAIL PROTECTED]> wrote:
> >> > Hi,
> >> >
> >> > We're trying to run a PIG script using the PigServer API in Java, but
> >> we're
> >> > having a couple issues.
> >> >
> >> > It seems to work well in most cases, but in our case we need to use
> Pig's
> >> > dynamic invokers. Basically this looks something like the following:
> >> > DEFINE isEmail InvokeForInt('com.company.pig.udf.Utils.isEmail',
> >> 'String');
> >> >
> >> > We register the script using PigServer like below:
> >> > pigServer.registerScript(pigScript, params);
> >> >
> >> > The problem is that PigServer doesn't seem to like these dynamic
> >> invokers.
> >> > We get an error during the parsing of the script by PigServer. We've
> >> > checked and the classpath is fine, and the UDF is defined correctly
> with
> >> > the right signature. Any help or advice would be very much
> appreciated on
> >> > this. I have attached the stack trace below for more details.
> >> >
> >> > Thanks,
> >> >
> >> > Charles
> >> >
> >> > =======================> >> >
> >> > Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
> >> 1000:
> >> > Error during parsing. could not instantiate 'InvokeForInt' with
> arguments
> >> > '[com.company.pig.udf.Utils.isEmail, String]'
> >> > at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1618)

Great, good to know about the command-line issue, I'll make a jira and
apply that patch.

There shouldn't be any issue using dynamic invokers from PigServer, I
suspect an environment issue (but I can't even posit what that issue
would be at the moment). Any chance you can provide a reproducible
test case and file a Jira?  I'm going to be offline for the next
couple of days, but I'll try to fix this asap if we can figure out
what the issue is.

D

On Thu, Nov 17, 2011 at 11:03 AM, Charles Menguy
<[EMAIL PROTECTED]> wrote:
> Thank you very much for the patch Dmitry.
>
> This seems to fix the issue when we run PIG from the command line, we no
> longer have to add this jar in the PIG_CLASSPATH, if we only register it it
> will work as expected.
>
> However, after pointing to the right patched PIG jar, we're still having
> issues using it from PigServer, basically the same issue and same stack
> trace.
>
> Do you know of any other workarounds to use dynamic invokers with
> PigServer, or we should refrain from using dynamic invokers with PigServer
> for now?
>
> Thanks,
>
> Charles
>
> On Thu, Nov 17, 2011 at 12:17 AM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:
>
>> Can you try applying the patch below to your version of pig?
>>
>> diff --git src/org/apache/pig/builtin/Invoker.java
>> src/org/apache/pig/builtin/Invoker.java
>> index adfc04a..8150165 100644
>> --- src/org/apache/pig/builtin/Invoker.java
>> +++ src/org/apache/pig/builtin/Invoker.java
>> @@ -31,6 +31,7 @@ import org.apache.commons.logging.LogFactory;
>>  import org.apache.pig.backend.executionengine.ExecException;
>>  import org.apache.pig.data.DataBag;
>>  import org.apache.pig.data.Tuple;
>> +import org.apache.pig.impl.PigContext;
>>  import org.apache.pig.impl.logicalLayer.FrontendException;
>>
>>  import com.google.common.collect.Lists;
>> @@ -67,7 +68,14 @@ public class Invoker<T>  {
>>     throws ClassNotFoundException, FrontendException,
>> SecurityException, NoSuchMethodException {
>>         String className = fullName.substring(0,
>> fullName.lastIndexOf('.'));
>>         String methodName >> fullName.substring(fullName.lastIndexOf('.')+1);
>> -        Class<?> klazz = Class.forName(className);
>> +        Class<?> klazz;
>> +        try {
>> +            klazz = PigContext.resolveClassName(className);
>> +        } catch (IOException e) {
>> +            // the amusing part is that PigContext throws this to wrap
>> one of
>> +            // the exceptions we declare!
>> +            throw new FrontendException(e);
>> +        }
>>         String[] paramSpecs = "".equals(paramSpecsStr) ? new
>> String[0] : paramSpecsStr.split(" ");
>>         isStatic_ = "static".equalsIgnoreCase(isStatic) ||
>> "true".equals(isStatic);
>>         paramClasses_ = new Class<?>[paramSpecs.length];
>>
>> On Wed, Nov 16, 2011 at 8:21 PM, Stan Rosenberg
>> <[EMAIL PROTECTED]> wrote:
>> > Hi Dimitriy,
>> >
>> > The script does run if invoked from command line but only if we set
>> > PIG_CLASSPATH to point at the jar.
>> >
>> > stan
>> > On Nov 16, 2011 11:18 PM, "Dmitriy Ryaboy" <[EMAIL PROTECTED]> wrote:
>> >
>> >> Does the script run if you launch it from the pig command line instead
>> >> of via PigServer?
>> >>
>> >> On Wed, Nov 16, 2011 at 3:01 PM, Charles Menguy
>> >> <[EMAIL PROTECTED]> wrote:
>> >> > Hi,
>> >> >
>> >> > We're trying to run a PIG script using the PigServer API in Java, but
>> >> we're
>> >> > having a couple issues.
>> >> >
>> >> > It seems to work well in most cases, but in our case we need to use
>> Pig's
>> >> > dynamic invokers. Basically this looks something like the following:
>> >> > DEFINE isEmail InvokeForInt('com.company.pig.udf.Utils.isEmail',
>> >> 'String');
>> >> >
>> >> > We register the script using PigServer like below:
>> >> > pigServer.registerScript(pigScript, params);
>> >> >
>> >> > The problem is that PigServer doesn't seem to like these dynamic
>> >> invokers.
>> >> > We get an error during the parsing of the script by PigServer. We've