Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Adding dependent jars for UDF in the PIG


Copy link to this message
-
Re: Adding dependent jars for UDF in the PIG
Pig uses reflection. The top exception says that there is no such method
signature. The problem is in the way you are trying to call method.
And its better to paste the whole stavktrace
13.08.2013 7:39 пользователь "Darpan R" <[EMAIL PROTECTED]> написал:

> I've a UDF which I use to do custom processing on the records. In the eval
> function I am using a third party jar for processing. I saw the job jar
> file, but it does not include this dependency. Is there any way to include
> dependent jar in the job jar ? (For testing I am running the cluster in the
> local mode).
>
> Or can I use distributed cache to make the dependent jar available to the
> UDF ?
>
> I've tried registering the dependent jars in the pig. For the first
> registered jar (all udfs are bundled in this jar) I do not face the issues.
> But for the second jar, I am facing issues when UDF tries to access the
> class from it.
> REGISTER '/home/user/pig/udfrepository/projectUDF.jar'
> REGISTER '/home/user/thridpartyjars/xyz.jar';
> The logs I get on the console are like this :
> 2013-08-11 10:35:02,485 [Thread-14] WARN
> org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
> java.lang.NoSuchMethodError:
> org.xyz.abc.convertToOtherFormat(Lorg/DateTimeZone;)Lorg/DateTime;
>         at com.myproject.MyUDF.exec(MyUDF.java:70)
> Any help on this is highly appreciated.
> Thanks in advance.
>
> -Darpan
>