Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> CDH3 fail python udf


+
Xiaomeng Wan 2011-03-31, 22:07
+
Aniket Mokashi 2011-03-31, 22:38
+
Xiaomeng Wan 2011-04-01, 16:06
+
Aniket Mokashi 2011-04-01, 20:24
+
Xiaomeng Wan 2011-04-01, 20:42
Copy link to this message
-
Re: CDH3 fail python udf
Hi Shawn,

I think this is more of CDH packaging problem than Pig problem. I suspect
this is related to Java versions of jython and other components.

You may look into
https://docs.cloudera.com/download/attachments/8784980/CDH3b3_Installation_Guide.pdf?version=1&modificationDate=1300229469101
for more details.

Thanks,
Aniket

On Fri, April 1, 2011 4:42 pm, Xiaomeng Wan wrote:
> Hi Aniket,
>
>
> Here is the stacktrace of the exception.
>
>
> java.io.IOException: Deserialization error: could not instantiate
> 'org.apache.pig.scripting.jython.JythonFunction' with arguments
> '[/home/shawn/TESS/code/mypyudfs.py, isStopWord]'
> at
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.ja
> va:55)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.s
> etup(PigMapBase.java:151)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322) at
> org.apache.hadoop.mapred.Child$4.run(Child.java:251)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation
> .java:1115)
> at org.apache.hadoop.mapred.Child.main(Child.java:245) Caused by:
> java.lang.RuntimeException: could not instantiate
> 'org.apache.pig.scripting.jython.JythonFunction' with arguments
> '[/home/shawn/TESS/code/mypyudfs.py, isStopWord]'
> at
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:50
> 2)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOpe
> rators.POUserFunc.instantiateFunc(POUserFunc.java:109)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOpe
> rators.POUserFunc.readObject(POUserFunc.java:451)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java
> :39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI
> mpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597) at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1849) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
>  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329) at
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at java.util.ArrayList.readObject(ArrayList.java:593) at
> sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI
> mpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597) at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1849) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
>  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329) at
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at java.util.HashMap.readObject(HashMap.java:1030) at
> sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI
> mpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597) at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1849) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
>  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329) at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871) at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
+
Xiaomeng Wan 2011-04-04, 15:36
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB