Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Better formated.. Pig udf help


Copy link to this message
-
Re: Better formated.. Pig udf help
Hi
Great catch
Now I get an error
Cannot find hadoop configuration in class path ( neither hadoop site XML
etc)

So I am running the file on a cluster which had say hadoop set up as

/path/hadoop
/path/pig

And I have account in it
So I cannot change the hadoop conf files as other users are also using it.
How do I run this just for me ?
On Thursday, October 25, 2012, Bill Graham <[EMAIL PROTECTED]> wrote:
> Somewhere you have a typo, probably in the execution of your program:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/pache/pig/Main****
>
> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>
> Note that the 'a' in apache is missing.
>
>
> On Thu, Oct 25, 2012 at 2:46 PM, jamal sasha <[EMAIL PROTECTED]>
wrote:
>
>>  Hi,****
>>
>>   I am trying to write a pig udf function.. Basically the data is of
>> format*
>> ***
>>
>> ** **
>>
>> Id,time****
>>
>> What I am trying to do is … parse the time and then see whether its
>> breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
>> null as well..****
>>
>> ** **
>>
>> So here is the udf code for this.****
>>
>> ** **
>>
>> *public* *class* time *extends* EvalFunc<String>{****
>>
>> ** **
>>
>>        *public* String exec(Tuple input) *throws* IOException {****
>>
>>        ****
>>
>>               *if* ((*input* == *null*) || (input.size() == 0))****
>>
>>                *return* *null*;****
>>
>>            *try*{****
>>
>>                String time = (String) input.get(0) ;****
>>
>>                DateFormat df = *new*
SimpleDateFormat("hh:mm:ss.000");****
>>
>>                Date date = df.parse(time);****
>>
>>                String timeOfDay = *getTimeOfDay*(date);****
>>
>>                *return* timeOfDay;****
>>
>>            } *catch* (ParseException e) {****
>>
>>                //how will I handle when df.parse(time) fails and throws
>> ParseException?****
>>
>>                //maybe:****
>>
>>                *return* *null*;****
>>
>>            }****
>>
>>               ****
>>
>>               ****
>>
>>        }****
>>
>> ** **
>>
>> After this.. in eclipse.. I did the export of this as a jar called myudfs
>> and I have a jar file called myudfs.jar****
>>
>> ** **
>>
>> Then I wrote the pig script as ****
>>
>> Time.pig****
>>
>> ** **
>>
>> REGISTER path/to/udf/myudfs.jar****
>>
>> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
>> time:chararray);****
>>
>> result = foreach in GENERATE  myudfs.time(time);****
>>
>> STORE result INTO 'path/to/output' using PigStorage(',');****
>>
>> ** **
>>
>> ** **
>>
>> So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****
>>
>> ** **
>>
>> Now I did this..****
>>
>> java -cp path/to/pig.jar org.apache.pig.Main time.pig****
>>
>> ** **
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/pache/pig/Main****
>>
>> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>>
>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>>
>>         at java.security.AccessController.doPrivileged(Native Method)****
>>
>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
>>
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****
>>
>>         at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
>>
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****
>>
>> Could not find the main class: org.pache.pig.Main.  Program will
exit.****
>>
>> ** **
>>
>> ** **
>>
>> What am I doing wrong?****
>>
>> Thanks****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> American Express made the following annotations on Thu Oct 25 2012
13:50:42
>>
>>
******************************************************************************
>>
>> "This message and any attachments are solely for the intended recipient
and
>> may contain confidential or privileged information. If you are not the
If
confidentiels
par
******************************************************************************
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB