|
|
-
Re: Better formated.. Pig udf helpjamal sasha 2012-11-12, 15:36
Hi
So I followed the instructions. Echo $PIG_CLASSPATH Points to hadoop conf On pig -f time.pig I still get same error? ? Problem resolving class version number So this is what I did 1 write a program in Java as my udf and export it as udf.jar Wrote a simple pig script and register that udf Now trying to run it thru pig -f time.pig On Friday, October 26, 2012, Prashant Kommireddi <[EMAIL PROTECTED]> wrote: > Do you see a "conf" dir within /path/Hadoop? If yes, point your > PIG_CLASSPATH to it. > > export PIG_CLASSPATH=/path/hadoop/conf. > > Sent from my iPhone > > On Oct 26, 2012, at 10:35 AM, jamal sasha <[EMAIL PROTECTED]> wrote: > >> Hi >> Great catch >> Now I get an error >> Cannot find hadoop configuration in class path ( neither hadoop site XML >> etc) >> >> So I am running the file on a cluster which had say hadoop set up as >> >> /path/hadoop >> /path/pig >> >> And I have account in it >> So I cannot change the hadoop conf files as other users are also using it. >> How do I run this just for me ? >> On Thursday, October 25, 2012, Bill Graham <[EMAIL PROTECTED]> wrote: >>> Somewhere you have a typo, probably in the execution of your program: >>> >>> Exception in thread "main" java.lang.NoClassDefFoundError: >>> org/pache/pig/Main**** >>> >>> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main**** >>> >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)**** >>> >>> Note that the 'a' in apache is missing. >>> >>> >>> On Thu, Oct 25, 2012 at 2:46 PM, jamal sasha <[EMAIL PROTECTED]> >> wrote: >>> >>>> Hi,**** >>>> >>>> I am trying to write a pig udf function.. Basically the data is of >>>> format* >>>> *** >>>> >>>> ** ** >>>> >>>> Id,time**** >>>> >>>> What I am trying to do is … parse the time and then see whether its >>>> breakfast, lunch or dinner.. based on the time stamp. Some entries wil be >>>> null as well..**** >>>> >>>> ** ** >>>> >>>> So here is the udf code for this.**** >>>> >>>> ** ** >>>> >>>> *public* *class* time *extends* EvalFunc<String>{**** >>>> >>>> ** ** >>>> >>>> *public* String exec(Tuple input) *throws* IOException {**** >>>> >>>> **** >>>> >>>> *if* ((*input* == *null*) || (input.size() == 0))**** >>>> >>>> *return* *null*;**** >>>> >>>> *try*{**** >>>> >>>> String time = (String) input.get(0) ;**** >>>> >>>> DateFormat df = *new* >> SimpleDateFormat("hh:mm:ss.000");**** >>>> >>>> Date date = df.parse(time);**** >>>> >>>> String timeOfDay = *getTimeOfDay*(date);**** >>>> >>>> *return* timeOfDay;**** >>>> >>>> } *catch* (ParseException e) {**** >>>> >>>> //how will I handle when df.parse(time) fails and throws >>>> ParseException?**** >>>> >>>> //maybe:**** >>>> >>>> *return* *null*;**** >>>> >>>> }**** >>>> >>>> **** >>>> >>>> **** >>>> >>>> }**** >>>> >>>> ** ** >>>> >>>> After this.. in eclipse.. I did the export of this as a jar called myudfs >>>> and I have a jar file called myudfs.jar**** >>>> >>>> ** ** >>>> >>>> Then I wrote the pig script as **** >>>> >>>> Time.pig**** >>>> >>>> ** ** >>>> >>>> REGISTER path/to/udf/myudfs.jar**** >>>> >>>> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long, >>>> time:chararray);**** >>>> >>>> result = f |