Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> python modules


Hi,

What's a proper way to deploy python udfs? I've dropped the latest
version of jython.jar in $PIG_HOME/lib.
Things work in "local" mode, but when I run on a cluster, built-in
python modules cannot be found. E.g., urlparse cannot be located:

ImportError: No module named urlparse

        at org.python.core.PyException.fillInStackTrace(PyException.java:70)
        at java.lang.Throwable.<init>(Throwable.java:181)
        at java.lang.Exception.<init>(Exception.java:29)
        at java.lang.RuntimeException.<init>(RuntimeException.java:32)
        at org.python.core.PyException.<init>(PyException.java:46)
        at org.python.core.PyException.<init>(PyException.java:43)
        at org.python.core.PyException.<init>(PyException.java:61)
        at org.python.core.Py.ImportError(Py.java:290)
        at org.python.core.imp.import_first(imp.java:750)
        at org.python.core.imp.import_name(imp.java:834)
        at org.python.core.imp.importName(imp.java:884)
        at org.python.core.ImportFunction.__call__(__builtin__.java:1220)
        at org.python.core.PyObject.__call__(PyObject.java:357)
        at org.python.core.__builtin__.__import__(__builtin__.java:1173)
        at org.python.core.imp.importFromAs(imp.java:978)
        at org.python.core.imp.importFrom(imp.java:954)
        at org.python.pycode._pyx3.f$0(udfs.py:40)
        at org.python.pycode._pyx3.call_function(udfs.py)
        at org.python.core.PyTableCode.call(PyTableCode.java:165)
        at org.python.core.PyCode.call(PyCode.java:18)
        at org.python.core.Py.runCode(Py.java:1261)
        at org.python.util.PythonInterpreter.execfile(PythonInterpreter.java:235)
        at org.apache.pig.scripting.jython.JythonScriptEngine$Interpreter.execfile(JythonScriptEngine.java:176)
        ... 15 more

Thanks,

stan
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB