Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Pig udf help


Copy link to this message
-
Pig udf help
jamal sasha 2012-10-25, 21:03
> Hi,
>
>   I am trying to write a pig udf function.. Basically the data is of
format
>
>
>
> Id,time
>
> What I am trying to do is … parse the time and then see whether its
breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
null as well..
>
>
>
> So here is the udf code for this.
>
>
>
> public class time extends EvalFunc<String>{
>
>
>
>        public String exec(Tuple input) throws IOException {
>
>
>
>               if ((input == null) || (input.size() == 0))
>
>                return null;
>
>            try{
>
>                String time = (String) input.get(0) ;
>
>                DateFormat df = new SimpleDateFormat("hh:mm:ss.000");
>
>                Date date = df.parse(time);
>
>                String timeOfDay = getTimeOfDay(date);
>
>                return timeOfDay;
>
>            } catch (ParseException e) {
>
>                //how will I handle when df.parse(time) fails and throws
ParseException?
>
>                //maybe:
>
>                return null;
>
>            }
>
>
>
>
>
>        }
>
>
>
> After this.. in eclipse.. I did the export of this as a jar called myudfs
and I have a jar file called myudfs.jar
>
>
>
> Then I wrote the pig script as
>
> Time.pig
>
>
>
> REGISTER path/to/udf/myudfs.jar
>
> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
time:chararray);
>
> result = foreach in GENERATE  myudfs.time(time);
>
> STORE result INTO 'path/to/output' using PigStorage(',');
>
>
>
>
>
> So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual
>
>
>
> Now I did this..
>
> java -cp path/to/pig.jar org.apache.pig.Main time.pig
>
>
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
org/pache/pig/Main
>
> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main
>
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>
>         at java.security.AccessController.doPrivileged(Native Method)
>
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>
> Could not find the main class: org.pache.pig.Main.  Program will exit.
>
>
>
>
>
> What am I doing wrong?
>
> Thanks