Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Long time registering python UDF


Copy link to this message
-
Long time registering python UDF
Wanted to see if anyone else is seeing this behavior.

I have a python file with a single 40 line function/UDF that seems to take 20+ minutes to get registered.  I don’t see the same issue when I register a Java UDF.  Is something that I am doing (or not doing)?

2012-10-10 10:36:33,862 [main] INFO  org.apache.pig.scripting.jython.JythonScriptEngine - Register scripting UDF: udf.explode
2012-10-10 10:49:29,018 [main] INFO  org.apache.pig.scripting.jython.JythonFunction - Schema 'R:bag{T:tuple( ... )}' defined for func explode
2012-10-10 11:00:45,036 [main] INFO  org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: HASH_JOIN
,GROUP_BY,FILTER

Thanks.

Will Duckworth Senior Vice President, Software Engineering | comScore, Inc. (NASDAQ:SCOR)

o +1 (703) 438-2108 | m +1 (301) 606-2977 | [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>

...........................................................................................................

Introducing Mobile Metrix 2.0 - The next generation of mobile behavioral measurement
www.comscore.com/MobileMetrix<http://www.comscore.com/Products_Services/Product_Index/Mobile_Metrix_2.0>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB