Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - CDH3 fail python udf


+
Xiaomeng Wan 2011-03-31, 22:07
+
Aniket Mokashi 2011-03-31, 22:38
+
Xiaomeng Wan 2011-04-01, 16:06
+
Aniket Mokashi 2011-04-01, 20:24
+
Xiaomeng Wan 2011-04-01, 20:42
Copy link to this message
-
Re: CDH3 fail python udf
Aniket Mokashi 2011-04-01, 23:10
Hi Shawn,

I think this is more of CDH packaging problem than Pig problem. I suspect
this is related to Java versions of jython and other components.

You may look into
https://docs.cloudera.com/download/attachments/8784980/CDH3b3_Installation_Guide.pdf?version=1&modificationDate=1300229469101
for more details.

Thanks,
Aniket

On Fri, April 1, 2011 4:42 pm, Xiaomeng Wan wrote:
> Hi Aniket,
>
>
> Here is the stacktrace of the exception.
>
>
> java.io.IOException: Deserialization error: could not instantiate
> 'org.apache.pig.scripting.jython.JythonFunction' with arguments
> '[/home/shawn/TESS/code/mypyudfs.py, isStopWord]'
> at
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.ja
> va:55)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.s
> etup(PigMapBase.java:151)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322) at
> org.apache.hadoop.mapred.Child$4.run(Child.java:251)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation
> .java:1115)
> at org.apache.hadoop.mapred.Child.main(Child.java:245) Caused by:
> java.lang.RuntimeException: could not instantiate
> 'org.apache.pig.scripting.jython.JythonFunction' with arguments
> '[/home/shawn/TESS/code/mypyudfs.py, isStopWord]'
> at
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:50
> 2)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOpe
> rators.POUserFunc.instantiateFunc(POUserFunc.java:109)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOpe
> rators.POUserFunc.readObject(POUserFunc.java:451)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java
> :39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI
> mpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597) at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1849) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
>  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329) at
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at java.util.ArrayList.readObject(ArrayList.java:593) at
> sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI
> mpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597) at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1849) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
>  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329) at
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at java.util.HashMap.readObject(HashMap.java:1030) at
> sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI
> mpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597) at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1849) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
>  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329) at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
+
Xiaomeng Wan 2011-04-04, 15:36