Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> Streaming UDFs


Copy link to this message
-
Streaming UDFs
We've been using pig's jython UDF support and really enjoying it, but we're finding several cases where we need python modules with C extensions, which jython doesn't support.

While we could use the STREAM operator to make this work, it'd be great to have the simplicity, type-checking/casting, and exact-field-using of UDFs.   I think we could get that by adding Streaming UDFs, for which I've sketched an idea on the wiki: https://cwiki.apache.org/confluence/display/PIG/StreamingUDFs

It's still just a sketch, but I'd love feedback on the direction, or any other ideas if people have thought about it in the past.

Best,
Doug
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB