Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Using a UDF written in Python


+
Jonathan Coveney 2010-12-27, 21:30
Copy link to this message
-
Re: Using a UDF written in Python
I think decorator used here is incorrect.
In general, "output:chararray" needs to be schema-string-compatible. Also,
you are using "outputSchemaFunction", which is used in case you want to
write a udf that has output schema dependent on input schema (�g -square)
and this should have a function with decorator "schemaFunction" (named
"output" in your case). I think using "outputSchema" decorator would fix
the problem here.

More details can be found at-
http://wiki.apache.org/pig/UDFsUsingScriptingLanguages

Thanks,
Aniket

On Mon, December 27, 2010 4:30 pm, Jonathan Coveney wrote:
> so I have module.py, and I want to be able to use it in a pig script. It
> has no special imports or anything. I do have
> @outputSchemaFunction("output:chararray)
>
>
> In my pig script, I have this
>
>
> register '/my/udf/location/udf.py' using jython as myfunc;
>
> is there any reason why this wouldn't work? here is the error I get:
>
> 2010-12-27 16:29:41,288 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 2998: Unhandled internal error. org/python/util/PythonInterpreter
>
>
> Not the most instructive error, but is there anything more I need to be
> doing to be able to use a python UDF?
>
> As an aside, are simply python UDF's as efficient as Java ones? I like
> Python a lot and love the idea of being able to UDF in it, but can use
> java if necessary.
>
+
Jonathan Coveney 2010-12-28, 03:14
+
Jonathan Coveney 2010-12-28, 03:18
+
soren@...) 2010-12-29, 17:52
+
Jonathan Coveney 2010-12-29, 17:57
+
soren@...) 2010-12-29, 18:35
+
Jonathan Coveney 2010-12-29, 18:57
+
Jonathan Coveney 2010-12-29, 19:32
+
Dmitriy Ryaboy 2010-12-29, 21:53
+
Jonathan Coveney 2010-12-29, 22:00
+
Jonathan Coveney 2010-12-29, 22:01
+
soren@...) 2010-12-29, 22:50
+
Jonathan Coveney 2010-12-29, 23:09
+
Jonathan Coveney 2010-12-29, 23:12
+
Dmitriy Ryaboy 2010-12-30, 00:37
+
Jonathan Coveney 2010-12-30, 02:11
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB