Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Long time registering python UDF


+
Duckworth, Will 2012-10-10, 16:11
Copy link to this message
-
Re: Long time registering python UDF
Curious: does the UDF have a lot of includes?

Russell Jurney twitter.com/rjurney
On Oct 10, 2012, at 9:11 AM, "Duckworth, Will" <[EMAIL PROTECTED]> wrote:

> Wanted to see if anyone else is seeing this behavior.
>
> I have a python file with a single 40 line function/UDF that seems to take 20+ minutes to get registered.  I don’t see the same issue when I register a Java UDF.  Is something that I am doing (or not doing)?
>
> 2012-10-10 10:36:33,862 [main] INFO  org.apache.pig.scripting.jython.JythonScriptEngine - Register scripting UDF: udf.explode
> 2012-10-10 10:49:29,018 [main] INFO  org.apache.pig.scripting.jython.JythonFunction - Schema 'R:bag{T:tuple( ... )}' defined for func explode
> 2012-10-10 11:00:45,036 [main] INFO  org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: HASH_JOIN
> ,GROUP_BY,FILTER
>
> Thanks.
>
>
>
> Will Duckworth Senior Vice President, Software Engineering | comScore, Inc. (NASDAQ:SCOR)
>
> o +1 (703) 438-2108 | m +1 (301) 606-2977 | [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
>
> ...........................................................................................................
>
> Introducing Mobile Metrix 2.0 - The next generation of mobile behavioral measurement
> www.comscore.com/MobileMetrix<http://www.comscore.com/Products_Services/Product_Index/Mobile_Metrix_2.0>
>
>
+
Duckworth, Will 2012-10-10, 18:56