Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Better formated.. Pig udf help


Copy link to this message
-
Re: Better formated.. Pig udf help
Hi
Great catch
Now I get an error
Cannot find hadoop configuration in class path ( neither hadoop site XML
etc)

So I am running the file on a cluster which had say hadoop set up as

/path/hadoop
/path/pig

And I have account in it
So I cannot change the hadoop conf files as other users are also using it.
How do I run this just for me ?
On Thursday, October 25, 2012, Bill Graham <[EMAIL PROTECTED]> wrote:
> Somewhere you have a typo, probably in the execution of your program:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/pache/pig/Main****
>
> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>
> Note that the 'a' in apache is missing.
>
>
> On Thu, Oct 25, 2012 at 2:46 PM, jamal sasha <[EMAIL PROTECTED]>
wrote:
>
>>  Hi,****
>>
>>   I am trying to write a pig udf function.. Basically the data is of
>> format*
>> ***
>>
>> ** **
>>
>> Id,time****
>>
>> What I am trying to do is … parse the time and then see whether its
>> breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
>> null as well..****
>>
>> ** **
>>
>> So here is the udf code for this.****
>>
>> ** **
>>
>> *public* *class* time *extends* EvalFunc<String>{****
>>
>> ** **
>>
>>        *public* String exec(Tuple input) *throws* IOException {****
>>
>>        ****
>>
>>               *if* ((*input* == *null*) || (input.size() == 0))****
>>
>>                *return* *null*;****
>>
>>            *try*{****
>>
>>                String time = (String) input.get(0) ;****
>>
>>                DateFormat df = *new*
SimpleDateFormat("hh:mm:ss.000");****
>>
>>                Date date = df.parse(time);****
>>
>>                String timeOfDay = *getTimeOfDay*(date);****
>>
>>                *return* timeOfDay;****
>>
>>            } *catch* (ParseException e) {****
>>
>>                //how will I handle when df.parse(time) fails and throws
>> ParseException?****
>>
>>                //maybe:****
>>
>>                *return* *null*;****
>>
>>            }****
>>
>>               ****
>>
>>               ****
>>
>>        }****
>>
>> ** **
>>
>> After this.. in eclipse.. I did the export of this as a jar called myudfs
>> and I have a jar file called myudfs.jar****
>>
>> ** **
>>
>> Then I wrote the pig script as ****
>>
>> Time.pig****
>>
>> ** **
>>
>> REGISTER path/to/udf/myudfs.jar****
>>
>> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
>> time:chararray);****
>>
>> result = foreach in GENERATE  myudfs.time(time);****
>>
>> STORE result INTO 'path/to/output' using PigStorage(',');****
>>
>> ** **
>>
>> ** **
>>
>> So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****
>>
>> ** **
>>
>> Now I did this..****
>>
>> java -cp path/to/pig.jar org.apache.pig.Main time.pig****
>>
>> ** **
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/pache/pig/Main****
>>
>> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>>
>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>>
>>         at java.security.AccessController.doPrivileged(Native Method)****
>>
>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
>>
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****
>>
>>         at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
>>
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****
>>
>> Could not find the main class: org.pache.pig.Main.  Program will
exit.****
>>
>> ** **
>>
>> ** **
>>
>> What am I doing wrong?****
>>
>> Thanks****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> American Express made the following annotations on Thu Oct 25 2012
13:50:42
>>
>>
******************************************************************************
>>
>> "This message and any attachments are solely for the intended recipient
and
>> may contain confidential or privileged information. If you are not the
If
confidentiels
par
******************************************************************************