Pig, mail # user - Better formated.. Pig udf help

Better formated.. Pig udf help

jamal sasha 2012-10-25, 21:46

Hi,****

  I am trying to write a pig udf function.. Basically the data is of format*
***

** **

Id,time****

What I am trying to do is … parse the time and then see whether its
breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
null as well..****

** **

So here is the udf code for this.****

** **

*public* *class* time *extends* EvalFunc<String>{****

** **

   *public* String exec(Tuple input) *throws* IOException {****

   ****

   *if* ((*input* == *null*) || (input.size() == 0))****

   *return* *null*;****

   *try*{****

   String time = (String) input.get(0) ;****

   DateFormat df = *new* SimpleDateFormat("hh:mm:ss.000");****

   Date date = df.parse(time);****

   String timeOfDay = *getTimeOfDay*(date);****

   *return* timeOfDay;****

   } *catch* (ParseException e) {****

   //how will I handle when df.parse(time) fails and throws
ParseException?****

   //maybe:****

   *return* *null*;****

   }****

   ****

   ****

   }****

** **

After this.. in eclipse.. I did the export of this as a jar called myudfs
and I have a jar file called myudfs.jar****

** **

Then I wrote the pig script as ****

Time.pig****

** **

REGISTER path/to/udf/myudfs.jar****

in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
time:chararray);****

result = foreach in GENERATE myudfs.time(time);****

STORE result INTO 'path/to/output' using PigStorage(',');****

** **

** **

So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****

** **

Now I did this..****

java -cp path/to/pig.jar org.apache.pig.Main time.pig****

** **

Exception in thread "main" java.lang.NoClassDefFoundError:
org/pache/pig/Main****

Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****

   at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****

   at java.security.AccessController.doPrivileged(Native Method)****

   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****

   at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****

   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****

   at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****

Could not find the main class: org.pache.pig.Main. Program will exit.****

** **

** **

What am I doing wrong?****

Thanks****

** **

** **

** **

** **

** **

American Express made the following annotations on Thu Oct 25 2012 13:50:42
******************************************************************************

"This message and any attachments are solely for the intended recipient and
may contain confidential or privileged information. If you are not the
intended recipient, any disclosure, copying, use, or distribution of the
information included in this message and any attachments is prohibited. If
you have received this communication in error, please notify us by reply
e-mail and immediately and permanently delete this message and any
attachments. Thank you."
American Express a ajouté le commentaire suivant le Thu Oct 25 2012
13:50:42
Ce courrier et toute pièce jointe qu'il contient sont réservés au seul
destinataire indiqué et peuvent renfermer des renseignements confidentiels
et privilégiés. Si vous n'êtes pas le destinataire prévu, toute
divulgation, duplication, utilisation ou distribution du courrier ou de
toute pièce jointe est interdite. Si vous avez reçu cette communication par
erreur, veuillez nous en aviser par courrier et détruire immédiatement le
courrier et les pièces jointes. Merci.
******************************************************************************
-------------------------------------------------------------------------------

Re: Better formated.. Pig udf help

pablomar 2012-10-25, 22:45

to run your script you have to do
pig -f time.pig

On Thu, Oct 25, 2012 at 5:46 PM, jamal sasha <[EMAIL PROTECTED]> wrote:

> Hi,****
>
> I am trying to write a pig udf function.. Basically the data is of
> format*
> ***
>
> ** **
>
> Id,time****
>
> What I am trying to do is … parse the time and then see whether its
> breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
> null as well..****
>
> ** **
>
> So here is the udf code for this.****
>
> ** **
>
> *public* *class* time *extends* EvalFunc<String>{****
>
> ** **
>
> *public* String exec(Tuple input) *throws* IOException {****
>
> ****
>
> *if* ((*input* == *null*) || (input.size() == 0))****
>
> *return* *null*;****
>
> *try*{****
>
> String time = (String) input.get(0) ;****
>
> DateFormat df = *new* SimpleDateFormat("hh:mm:ss.000");****
>
> Date date = df.parse(time);****
>
> String timeOfDay = *getTimeOfDay*(date);****
>
> *return* timeOfDay;****
>
> } *catch* (ParseException e) {****
>
> //how will I handle when df.parse(time) fails and throws
> ParseException?****
>
> //maybe:****
>
> *return* *null*;****
>
> }****
>
> ****
>
> ****
>
> }****
>
> ** **
>
> After this.. in eclipse.. I did the export of this as a jar called myudfs
> and I have a jar file called myudfs.jar****
>
> ** **
>
> Then I wrote the pig script as ****
>
> Time.pig****
>
> ** **
>
> REGISTER path/to/udf/myudfs.jar****
>
> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
> time:chararray);****
>
> result = foreach in GENERATE myudfs.time(time);****
>
> STORE result INTO 'path/to/output' using PigStorage(',');****
>
> ** **
>
> ** **
>
> So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****
>
> ** **
>
> Now I did this..****
>
> java -cp path/to/pig.jar org.apache.pig.Main time.pig****
>
> ** **
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/pache/pig/Main****
>
> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>
> at java.security.AccessController.doPrivileged(Native Method)****
>
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****
>
> at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****
>
> Could not find the main class: org.pache.pig.Main. Program will exit.****
>
> ** **
>
> ** **
>
> What am I doing wrong?****
>
> Thanks****
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> American Express made the following annotations on Thu Oct 25 2012 13:50:42
>
> ******************************************************************************
>
> "This message and any attachments are solely for the intended recipient and
> may contain confidential or privileged information. If you are not the
> intended recipient, any disclosure, copying, use, or distribution of the
> information included in this message and any attachments is prohibited. If
> you have received this communication in error, please notify us by reply
> e-mail and immediately and permanently delete this message and any
> attachments. Thank you."
> American Express a ajouté le commentaire suivant le Thu Oct 25 2012
> 13:50:42
> Ce courrier et toute pièce jointe qu'il contient sont réservés au seul
> destinataire indiqué et peuvent renfermer des renseignements confidentiels
> et privilégiés. Si vous n'êtes pas le destinataire prévu, toute
> divulgation, duplication, utilisation ou distribution du courrier ou de
> toute pièce jointe est interdite. Si vous avez reçu cette communication par
> erreur, veuillez nous en aviser par courrier et détruire immédiatement le

Re: Better formated.. Pig udf help

Bill Graham 2012-10-25, 23:57

Somewhere you have a typo, probably in the execution of your program:

Exception in thread "main" java.lang.NoClassDefFoundError:
org/pache/pig/Main****

Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****

at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****

Note that the 'a' in apache is missing.
On Thu, Oct 25, 2012 at 2:46 PM, jamal sasha <[EMAIL PROTECTED]> wrote:

> Hi,****
>
> I am trying to write a pig udf function.. Basically the data is of
> format*
> ***
>
> ** **
>
> Id,time****
>
> What I am trying to do is … parse the time and then see whether its
> breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
> null as well..****
>
> ** **
>
> So here is the udf code for this.****
>
> ** **
>
> *public* *class* time *extends* EvalFunc<String>{****
>
> ** **
>
> *public* String exec(Tuple input) *throws* IOException {****
>
> ****
>
> *if* ((*input* == *null*) || (input.size() == 0))****
>
> *return* *null*;****
>
> *try*{****
>
> String time = (String) input.get(0) ;****
>
> DateFormat df = *new* SimpleDateFormat("hh:mm:ss.000");****
>
> Date date = df.parse(time);****
>
> String timeOfDay = *getTimeOfDay*(date);****
>
> *return* timeOfDay;****
>
> } *catch* (ParseException e) {****
>
> //how will I handle when df.parse(time) fails and throws
> ParseException?****
>
> //maybe:****
>
> *return* *null*;****
>
> }****
>
> ****
>
> ****
>
> }****
>
> ** **
>
> After this.. in eclipse.. I did the export of this as a jar called myudfs
> and I have a jar file called myudfs.jar****
>
> ** **
>
> Then I wrote the pig script as ****
>
> Time.pig****
>
> ** **
>
> REGISTER path/to/udf/myudfs.jar****
>
> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
> time:chararray);****
>
> result = foreach in GENERATE myudfs.time(time);****
>
> STORE result INTO 'path/to/output' using PigStorage(',');****
>
> ** **
>
> ** **
>
> So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****
>
> ** **
>
> Now I did this..****
>
> java -cp path/to/pig.jar org.apache.pig.Main time.pig****
>
> ** **
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/pache/pig/Main****
>
> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>
> at java.security.AccessController.doPrivileged(Native Method)****
>
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****
>
> at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****
>
> Could not find the main class: org.pache.pig.Main. Program will exit.****
>
> ** **
>
> ** **
>
> What am I doing wrong?****
>
> Thanks****
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> American Express made the following annotations on Thu Oct 25 2012 13:50:42
>
> ******************************************************************************
>
> "This message and any attachments are solely for the intended recipient and
> may contain confidential or privileged information. If you are not the
> intended recipient, any disclosure, copying, use, or distribution of the
> information included in this message and any attachments is prohibited. If
> you have received this communication in error, please notify us by reply
> e-mail and immediately and permanently delete this message and any
> attachments. Thank you."
> American Express a ajouté le commentaire suivant le Thu Oct 25 2012
> 13:50:42
> Ce courrier et toute pièce jointe qu'il contient sont réservés au seul
> destinataire indiqué et peuvent renfermer des renseignements confidentiels

*Note that I'm no longer using my Yahoo! email address. Please email me at
[EMAIL PROTECTED] going forward.*

Re: Better formated.. Pig udf help

jamal sasha 2012-10-26, 17:24

Hi
In this case I get an error
Problem resolving class version numbers for class myudfs.time??

On Thursday, October 25, 2012, pablomar <[EMAIL PROTECTED]>
wrote:
> to run your script you have to do
> pig -f time.pig
>
>
>
> On Thu, Oct 25, 2012 at 5:46 PM, jamal sasha <[EMAIL PROTECTED]>
wrote:
>
>> Hi,****
>>
>> I am trying to write a pig udf function.. Basically the data is of
>> format*
>> ***
>>
>> ** **
>>
>> Id,time****
>>
>> What I am trying to do is … parse the time and then see whether its
>> breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
>> null as well..****
>>
>> ** **
>>
>> So here is the udf code for this.****
>>
>> ** **
>>
>> *public* *class* time *extends* EvalFunc<String>{****
>>
>> ** **
>>
>> *public* String exec(Tuple input) *throws* IOException {****
>>
>> ****
>>
>> *if* ((*input* == *null*) || (input.size() == 0))****
>>
>> *return* *null*;****
>>
>> *try*{****
>>
>> String time = (String) input.get(0) ;****
>>
>> DateFormat df = *new*
SimpleDateFormat("hh:mm:ss.000");****
>>
>> Date date = df.parse(time);****
>>
>> String timeOfDay = *getTimeOfDay*(date);****
>>
>> *return* timeOfDay;****
>>
>> } *catch* (ParseException e) {****
>>
>> //how will I handle when df.parse(time) fails and throws
>> ParseException?****
>>
>> //maybe:****
>>
>> *return* *null*;****
>>
>> }****
>>
>> ****
>>
>> ****
>>
>> }****
>>
>> ** **
>>
>> After this.. in eclipse.. I did the export of this as a jar called myudfs
>> and I have a jar file called myudfs.jar****
>>
>> ** **
>>
>> Then I wrote the pig script as ****
>>
>> Time.pig****
>>
>> ** **
>>
>> REGISTER path/to/udf/myudfs.jar****
>>
>> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
>> time:chararray);****
>>
>> result = foreach in GENERATE myudfs.time(time);****
>>
>> STORE result INTO 'path/to/output' using PigStorage(',');****
>>
>> ** **
>>
>> ** **
>>
>> So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****
>>
>> ** **
>>
>> Now I did this..****
>>
>> java -cp path/to/pig.jar org.apache.pig.Main time.pig****
>>
>> ** **
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/pache/pig/Main****
>>
>> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>>
>> at java.security.AccessController.doPrivileged(Native Method)****
>>
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****
>>
>> at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****
>>
>> Could not find the main class: org.pache.pig.Main. Program will
exit.****
>>
>> ** **
>>
>> ** **
>>
>> What am I doing wrong?****
>>
>> Thanks****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> American Express made the following annotations on Thu Oct 25 2012
13:50:42
>>
>>
******************************************************************************
>>
>> "This message and any attachments are solely for the intended recipient
and
>> may contain confidential or privileged information. If you are not the
>> intended recipient, any disclosure, copying, use, or distribution of the
>> information included in this message and any attachments is prohibited.
If
>> you have received this communication in error, please notify us by reply
>> e-mail and immediately and permanently delete this message and any
>> attachments. Thank you."
>> American Express a ajouté le commentaire suivant le Thu Oct 25 2012
>> 13:50:42
>> Ce courrier et toute pièce jointe qu'il contient sont réservés au seul
>> destinataire indiqué et peuvent renfermer des renseignements
confidentiels
par
******************************************************************************

Re: Better formated.. Pig udf help

jamal sasha 2012-10-26, 17:35

Hi
Great catch
Now I get an error
Cannot find hadoop configuration in class path ( neither hadoop site XML
etc)

So I am running the file on a cluster which had say hadoop set up as

/path/hadoop
/path/pig

And I have account in it
So I cannot change the hadoop conf files as other users are also using it.
How do I run this just for me ?
On Thursday, October 25, 2012, Bill Graham <[EMAIL PROTECTED]> wrote:
> Somewhere you have a typo, probably in the execution of your program:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/pache/pig/Main****
>
> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>
> Note that the 'a' in apache is missing.
>
>
> On Thu, Oct 25, 2012 at 2:46 PM, jamal sasha <[EMAIL PROTECTED]>
wrote:
>
>> Hi,****
>>
>> I am trying to write a pig udf function.. Basically the data is of
>> format*
>> ***
>>
>> ** **
>>
>> Id,time****
>>
>> What I am trying to do is … parse the time and then see whether its
>> breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
>> null as well..****
>>
>> ** **
>>
>> So here is the udf code for this.****
>>
>> ** **
>>
>> *public* *class* time *extends* EvalFunc<String>{****
>>
>> ** **
>>
>> *public* String exec(Tuple input) *throws* IOException {****
>>
>> ****
>>
>> *if* ((*input* == *null*) || (input.size() == 0))****
>>
>> *return* *null*;****
>>
>> *try*{****
>>
>> String time = (String) input.get(0) ;****
>>
>> DateFormat df = *new*
SimpleDateFormat("hh:mm:ss.000");****
>>
>> Date date = df.parse(time);****
>>
>> String timeOfDay = *getTimeOfDay*(date);****
>>
>> *return* timeOfDay;****
>>
>> } *catch* (ParseException e) {****
>>
>> //how will I handle when df.parse(time) fails and throws
>> ParseException?****
>>
>> //maybe:****
>>
>> *return* *null*;****
>>
>> }****
>>
>> ****
>>
>> ****
>>
>> }****
>>
>> ** **
>>
>> After this.. in eclipse.. I did the export of this as a jar called myudfs
>> and I have a jar file called myudfs.jar****
>>
>> ** **
>>
>> Then I wrote the pig script as ****
>>
>> Time.pig****
>>
>> ** **
>>
>> REGISTER path/to/udf/myudfs.jar****
>>
>> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
>> time:chararray);****
>>
>> result = foreach in GENERATE myudfs.time(time);****
>>
>> STORE result INTO 'path/to/output' using PigStorage(',');****
>>
>> ** **
>>
>> ** **
>>
>> So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****
>>
>> ** **
>>
>> Now I did this..****
>>
>> java -cp path/to/pig.jar org.apache.pig.Main time.pig****
>>
>> ** **
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/pache/pig/Main****
>>
>> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>>
>> at java.security.AccessController.doPrivileged(Native Method)****
>>
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****
>>
>> at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****
>>
>> Could not find the main class: org.pache.pig.Main. Program will
exit.****
>>
>> ** **
>>
>> ** **
>>
>> What am I doing wrong?****
>>
>> Thanks****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> American Express made the following annotations on Thu Oct 25 2012
13:50:42
>>
>>
******************************************************************************
>>
>> "This message and any attachments are solely for the intended recipient
and
>> may contain confidential or privileged information. If you are not the
If
confidentiels
par
******************************************************************************

Re: Better formated.. Pig udf help

Prashant Kommireddi 2012-10-26, 17:43

Do you see a "conf" dir within /path/Hadoop? If yes, point your
PIG_CLASSPATH to it.

export PIG_CLASSPATH=/path/hadoop/conf.

Sent from my iPhone

On Oct 26, 2012, at 10:35 AM, jamal sasha <[EMAIL PROTECTED]> wrote:

> Hi
> Great catch
> Now I get an error
> Cannot find hadoop configuration in class path ( neither hadoop site XML
> etc)
>
> So I am running the file on a cluster which had say hadoop set up as
>
> /path/hadoop
> /path/pig
>
> And I have account in it
> So I cannot change the hadoop conf files as other users are also using it.
> How do I run this just for me ?
> On Thursday, October 25, 2012, Bill Graham <[EMAIL PROTECTED]> wrote:
>> Somewhere you have a typo, probably in the execution of your program:
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/pache/pig/Main****
>>
>> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>>
>> Note that the 'a' in apache is missing.
>>
>>
>> On Thu, Oct 25, 2012 at 2:46 PM, jamal sasha <[EMAIL PROTECTED]>
> wrote:
>>
>>> Hi,****
>>>
>>> I am trying to write a pig udf function.. Basically the data is of
>>> format*
>>> ***
>>>
>>> ** **
>>>
>>> Id,time****
>>>
>>> What I am trying to do is … parse the time and then see whether its
>>> breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
>>> null as well..****
>>>
>>> ** **
>>>
>>> So here is the udf code for this.****
>>>
>>> ** **
>>>
>>> *public* *class* time *extends* EvalFunc<String>{****
>>>
>>> ** **
>>>
>>> *public* String exec(Tuple input) *throws* IOException {****
>>>
>>> ****
>>>
>>> *if* ((*input* == *null*) || (input.size() == 0))****
>>>
>>> *return* *null*;****
>>>
>>> *try*{****
>>>
>>> String time = (String) input.get(0) ;****
>>>
>>> DateFormat df = *new*
> SimpleDateFormat("hh:mm:ss.000");****
>>>
>>> Date date = df.parse(time);****
>>>
>>> String timeOfDay = *getTimeOfDay*(date);****
>>>
>>> *return* timeOfDay;****
>>>
>>> } *catch* (ParseException e) {****
>>>
>>> //how will I handle when df.parse(time) fails and throws
>>> ParseException?****
>>>
>>> //maybe:****
>>>
>>> *return* *null*;****
>>>
>>> }****
>>>
>>> ****
>>>
>>> ****
>>>
>>> }****
>>>
>>> ** **
>>>
>>> After this.. in eclipse.. I did the export of this as a jar called myudfs
>>> and I have a jar file called myudfs.jar****
>>>
>>> ** **
>>>
>>> Then I wrote the pig script as ****
>>>
>>> Time.pig****
>>>
>>> ** **
>>>
>>> REGISTER path/to/udf/myudfs.jar****
>>>
>>> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
>>> time:chararray);****
>>>
>>> result = foreach in GENERATE myudfs.time(time);****
>>>
>>> STORE result INTO 'path/to/output' using PigStorage(',');****
>>>
>>> ** **
>>>
>>> ** **
>>>
>>> So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****
>>>
>>> ** **
>>>
>>> Now I did this..****
>>>
>>> java -cp path/to/pig.jar org.apache.pig.Main time.pig****
>>>
>>> ** **
>>>
>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>> org/pache/pig/Main****
>>>
>>> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>>>
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>>>
>>> at java.security.AccessController.doPrivileged(Native Method)****
>>>
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
>>>
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****
>>>
>>> at
>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
>>>
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****
>>>
>>> Could not find the main class: org.pache.pig.Main. Program will
> exit.****
>>>
>>> ** **
>>>
>>> ** **
>>>
>>> What am I doing wrong?****

Re: Better formated.. Pig udf help

jamal sasha 2012-11-12, 15:36

Hi
So I followed the instructions.
Echo $PIG_CLASSPATH
Points to hadoop conf
On pig -f time.pig
I still get same error? ?
Problem resolving class version number
So this is what I did
1 write a program in Java as my udf and export it as udf.jar
Wrote a simple pig script and register that udf

Now trying to run it thru pig -f time.pig
On Friday, October 26, 2012, Prashant Kommireddi <[EMAIL PROTECTED]>
wrote:
> Do you see a "conf" dir within /path/Hadoop? If yes, point your
> PIG_CLASSPATH to it.
>
> export PIG_CLASSPATH=/path/hadoop/conf.
>
> Sent from my iPhone
>
> On Oct 26, 2012, at 10:35 AM, jamal sasha <[EMAIL PROTECTED]> wrote:
>
>> Hi
>> Great catch
>> Now I get an error
>> Cannot find hadoop configuration in class path ( neither hadoop site XML
>> etc)
>>
>> So I am running the file on a cluster which had say hadoop set up as
>>
>> /path/hadoop
>> /path/pig
>>
>> And I have account in it
>> So I cannot change the hadoop conf files as other users are also using
it.
>> How do I run this just for me ?
>> On Thursday, October 25, 2012, Bill Graham <[EMAIL PROTECTED]> wrote:
>>> Somewhere you have a typo, probably in the execution of your program:
>>>
>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>> org/pache/pig/Main****
>>>
>>> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>>>
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>>>
>>> Note that the 'a' in apache is missing.
>>>
>>>
>>> On Thu, Oct 25, 2012 at 2:46 PM, jamal sasha <[EMAIL PROTECTED]>
>> wrote:
>>>
>>>> Hi,****
>>>>
>>>> I am trying to write a pig udf function.. Basically the data is of
>>>> format*
>>>> ***
>>>>
>>>> ** **
>>>>
>>>> Id,time****
>>>>
>>>> What I am trying to do is … parse the time and then see whether its
>>>> breakfast, lunch or dinner.. based on the time stamp. Some entries wil
be
>>>> null as well..****
>>>>
>>>> ** **
>>>>
>>>> So here is the udf code for this.****
>>>>
>>>> ** **
>>>>
>>>> *public* *class* time *extends* EvalFunc<String>{****
>>>>
>>>> ** **
>>>>
>>>> *public* String exec(Tuple input) *throws* IOException {****
>>>>
>>>> ****
>>>>
>>>> *if* ((*input* == *null*) || (input.size() == 0))****
>>>>
>>>> *return* *null*;****
>>>>
>>>> *try*{****
>>>>
>>>> String time = (String) input.get(0) ;****
>>>>
>>>> DateFormat df = *new*
>> SimpleDateFormat("hh:mm:ss.000");****
>>>>
>>>> Date date = df.parse(time);****
>>>>
>>>> String timeOfDay = *getTimeOfDay*(date);****
>>>>
>>>> *return* timeOfDay;****
>>>>
>>>> } *catch* (ParseException e) {****
>>>>
>>>> //how will I handle when df.parse(time) fails and throws
>>>> ParseException?****
>>>>
>>>> //maybe:****
>>>>
>>>> *return* *null*;****
>>>>
>>>> }****
>>>>
>>>> ****
>>>>
>>>> ****
>>>>
>>>> }****
>>>>
>>>> ** **
>>>>
>>>> After this.. in eclipse.. I did the export of this as a jar called
myudfs
>>>> and I have a jar file called myudfs.jar****
>>>>
>>>> ** **
>>>>
>>>> Then I wrote the pig script as ****
>>>>
>>>> Time.pig****
>>>>
>>>> ** **
>>>>
>>>> REGISTER path/to/udf/myudfs.jar****
>>>>
>>>> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
>>>> time:chararray);****
>>>>
>>>> result = f