Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Deserialization error when using Jython UDF in Pig 0.10 script


Copy link to this message
-
Re: Deserialization error when using Jython UDF in Pig 0.10 script
Hi,

- ERROR 2997: Unable to recreate exception from backed error:
> java.io.IOException: Deserialization error: could not instantiate
> 'org.apache.pig.scripting.jython.JythonFunction' with arguments

'[/transfer/marek/poc123-code/poc123-udf.py, parse_time_local]'
This is a bug in Pig-0.10. Please refer to PIg-2761 and its related jiras:

https://issues.apache.org/jira/browse/PIG-2761
https://issues.apache.org/jira/browse/PIG-2745

Thanks,
Cheolsoo

On Tue, Jun 19, 2012 at 4:28 PM, Chris Diehl <[EMAIL PROTECTED]> wrote:

> Hi Marek,
>
> I wish I could offer a solution. I tried all sorts of permutations and to
> no avail. I retreated back to our base install of Pig 0.8 where Jython UDFs
> work for me.
> Here's a list of things I've tried:
>
> 1) Setting JYTHON_HOME to the Jython path
> 2) Setting PIG_CLASSPATH to <Jython path>/jython.jar
> 3) Copying jython.jar to <Pig path>/lib
> 4) Using Jython 2.5.0 and 2.5.2
>
> I can't afford to burn any more time right now looking into this. I'm
> hoping the problem can be uncovered. Jython UDF support is critical for my
> productivity.
>
> Chris
>
> On Tue, Jun 19, 2012 at 2:40 PM, Marek Dudek <[EMAIL PROTECTED]
> >wrote:
>
> > Chris,
> >
> > I can confirm the problem; just run into it myself :(
> >
> > In my case I was able to run my code succesfully on MapR cluster (hadoop
> > 0.20.2), with Pig 0.9.1 and Jython 2.5.2.
> > Now, I am getting same "Deserialization error" after upgrading to Pig
> > 0.10.0
> > (see log snippets below).
> >
> > Any ideas/luck so far? Is there anything I can try/test to help?
> >
> > Marek
> >
> >
> >
> > -----------------
> > 2012-06-19 16:24:54,470 [main] INFO  org.apache.pig.Main - Apache Pig
> > version 0.10.0 (r1328203) compiled Apr 20 2012, 05:54:07
> > 2012-06-19 16:24:54,470 [main] INFO  org.apache.pig.Main - Logging error
> > messages to: /transfer/marek/poc123-code/pig_1340141094466.log
> > (...)
> > 2012-06-19 16:26:27,484 [main] ERROR
> > org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
> > 2012-06-19 16:26:27,488 [main] INFO
> > org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
> >
> > HadoopVersion   PigVersion      UserId  StartedAt
> FinishedAt
> > Features
> > 0.20.2-dev      0.10.0          marek   2012-06-19 16:25:13
> 2012-06-19
> > 16:26:27     FILTER
> >
> > Failed!
> > (...)
> >
> > 2012-06-19 16:26:27,488 [main] INFO
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLaunch
> > er - Failed!
> > 2012-06-19 16:26:27,544 [main] ERROR
> org.apache.pig.tools.grunt.GruntParser
> > - ERROR 2997: Unable to recreate exception from backed error:
> > java.io.IOException: Deserialization error: could not instantiate
> > 'org.apache.pig.scripting.jython.JythonFunction' with arguments
> > '[/transfer/marek/poc123-code/poc123-udf.py, parse_time_local]'
> >        at
> >
> >
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:
> > 55)
> >        at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBa
> > se.setup(PigGenericMapBase.java:177)
> >        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
> >        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:654)
> >        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
> >        at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> >        at java.security.AccessController.doPrivileged(Native Method)
> >        at javax.security.auth.Subject.doAs(Subject.java:396)
> >        at
> >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja
> > va:1109)
> >        at org.apache.hadoop.mapred.Child.main(Child.java:264)
> > Caused by: java.lang.RuntimeException: could not instantiate
> > 'org.apache.pig.scripting.jython.JythonFunction' with arguments
> > '[/transfer/marek/poc123-code/poc123-udf.py, parse_time_local]'
> >        at
> >
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:553)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB