I keep getting this error, it seems pig cannot locate the jar that
contains my udf ImageProcessor.How do i build a jar that contains my udf
(p.s the UDF documentation wasnt that helpful

laptop:~/pig-0.7.0/trunk$ javac -cp $CLASSPATH:pig.jar ImageProcessor.java
laptop:~/pig-0.7.0/trunk$ bin/pig specialscript.pig
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.conf.Configuration).
log4j:WARN Please initialize the log4j system properly.
2010-08-11 11:54:15,820 [main] INFO  org.apache.pig.Main - Logging error
messages to: /home/hadoop/pig-0.7.0/trunk/pig_1281542055806.log
2010-08-11 11:54:16,005 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to hadoop file system at: hdfs://localhost:54310
2010-08-11 11:54:16,179 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to map-reduce job tracker at: localhost:54311
2010-08-11 11:54:16,384 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1070: Could not resolve ImageProcessor using imports: [,
org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Details at logfile: /home/hadoop/pig-0.7.0/trunk/pig_1281542055806.log
laptop:~/pig-0.7.0/trunk$

Just compiling it isn't enough. You need to package it in a jar and then
register that jar in your pig script.
Documentation: http://hadoop.apache.org/pig/docs/r0.7.0/udf.html

Ashutosh

On Wed, Aug 11, 2010 at 09:06, Ifeanyichukwu Osuji <[EMAIL PROTECTED]>wrote:

> I keep getting this error, it seems pig cannot locate the jar that
> contains my udf ImageProcessor.How do i build a jar that contains my udf
> (p.s the UDF documentation wasnt that helpful
>
>
>
> laptop:~/pig-0.7.0/trunk$ javac -cp $CLASSPATH:pig.jar ImageProcessor.java
> laptop:~/pig-0.7.0/trunk$ bin/pig specialscript.pig
> log4j:WARN No appenders could be found for logger
> (org.apache.hadoop.conf.Configuration).
> log4j:WARN Please initialize the log4j system properly.
> 2010-08-11 11:54:15,820 [main] INFO  org.apache.pig.Main - Logging error
> messages to: /home/hadoop/pig-0.7.0/trunk/pig_1281542055806.log
> 2010-08-11 11:54:16,005 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to hadoop file system at: hdfs://localhost:54310
> 2010-08-11 11:54:16,179 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to map-reduce job tracker at: localhost:54311
> 2010-08-11 11:54:16,384 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1070: Could not resolve ImageProcessor using imports: [,
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> Details at logfile: /home/hadoop/pig-0.7.0/trunk/pig_1281542055806.log
> laptop:~/pig-0.7.0/trunk$
>
>
>

feanyichukwu Osuji,

Not sure if this help you or not, but we had a similar problem where dependent jars where not included in the deployed jar, so we created a mvn that included the dependencies into the final jar.

If that is your issue the following pom.xml snippet should help

<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>attached</goal>
</goals>
</execution>
</executions>
</plugin>
Then you can register your jar with the pig command:

REGISTER /my/local/path/to/the/jars/my.jar

Define the method you want to call like:

DEFINE MY_STUFF com.mycompany.ufs.MyStuff();

Define the method you want to call like:

x = FOREACH y GENERATE MY_STUFF(x);

On Aug 11, 2010, at 10:06 AM, Ifeanyichukwu Osuji wrote:

> I keep getting this error, it seems pig cannot locate the jar that
> contains my udf ImageProcessor.How do i build a jar that contains my udf
> (p.s the UDF documentation wasnt that helpful
>
>
>
> laptop:~/pig-0.7.0/trunk$ javac -cp $CLASSPATH:pig.jar ImageProcessor.java
> laptop:~/pig-0.7.0/trunk$ bin/pig specialscript.pig
> log4j:WARN No appenders could be found for logger
> (org.apache.hadoop.conf.Configuration).
> log4j:WARN Please initialize the log4j system properly.
> 2010-08-11 11:54:15,820 [main] INFO  org.apache.pig.Main - Logging error
> messages to: /home/hadoop/pig-0.7.0/trunk/pig_1281542055806.log
> 2010-08-11 11:54:16,005 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to hadoop file system at: hdfs://localhost:54310
> 2010-08-11 11:54:16,179 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to map-reduce job tracker at: localhost:54311
> 2010-08-11 11:54:16,384 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1070: Could not resolve ImageProcessor using imports: [,
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> Details at logfile: /home/hadoop/pig-0.7.0/trunk/pig_1281542055806.log
> laptop:~/pig-0.7.0/trunk$
>
>