Doug Daniels 2011-11-20, 17:28
We've been using pig's jython UDF support and really enjoying it, but we're finding several cases where we need python modules with C extensions, which jython doesn't support.
While we could use the STREAM operator to make this work, it'd be great to have the simplicity, type-checking/casting, and exact-field-using of UDFs. I think we could get that by adding Streaming UDFs, for which I've sketched an idea on the wiki: https://cwiki.apache.org/confluence/display/PIG/StreamingUDFs
It's still just a sketch, but I'd love feedback on the direction, or any other ideas if people have thought about it in the past.