Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Python UDF for piglatin script not finding re module


+
Fernando Doglio 2012-04-19, 19:29
Copy link to this message
-
Re: Python UDF for piglatin script not finding re module
Gianmarco De Francisci Mo... 2012-04-19, 20:32
Hi,

welcome to the community!

First question is, how did you install pig?
Source or some other distribution?

Second, Pig wants jython.version=2.5.0
Which version have you installed?

Cheers,
--
Gianmarco

On Thu, Apr 19, 2012 at 21:29, Fernando Doglio <[EMAIL PROTECTED]
> wrote:

> Hello everyone, new pig user here.
>
> I've being toying around with pig for a while now, and I wanted to create
> my first UDF in Python. It all went fine until I wanted to use RegExps.
>
> I tried importing the "re" module on my python script, but when I did that,
> I got an error message:
>
> [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1121: Python
> Error. Traceback (most recent call last):
>
>  File "utils.py", line 3, in <module>
>
>    import re
> ImportError: No module named re
>
>
> Looking around the net, I ended up installing "jython" on my system, that
> didn't do the trick, but doing a bit more research, I found out that the
> sys.path variable was different when I ran the script using the "jython"
> command and when I used the "pig" command:
>
> *Using Jython*
>
> ['', '/usr/share/jython/Lib', '/usr/lib/site-python', '__classpath__']
>
>
> *Using pig*
>
> ['/pig/lib/Lib', '__classpath__', '__pyclasspath__/']1
>
>
> So I went back to my script, and manually changed the "sys.path" variable
> to look like the first example up there.. now when I run the script with
> pig, I get the following message:
>
>  File "utils.py", line 8, in <module>
>
>    import re
>  File "/usr/share/jython/Lib/re.py", line 7, in <module>
>
>    import sre, sys
>  File "/usr/share/jython/Lib/sre.py", line 97, in <module>
>
>    import sre_compile
>  File "/usr/share/jython/Lib/sre_compile.py", line 17, in <module>
>
>    assert _sre.MAGIC == MAGIC, "SRE module mismatch"
> AssertionError: SRE module mismatch
>
>
> According to someone on stackoverflow, this is due to the fact that my
> manually installed "jython" version is different from the one used by
> Pig....
>
> Now, finally, my question is: how do I get my python script to run on pig
> correctly?
>
>
> Thanks so much for reading and if you can, for answering!
>
> Fernando Doglio
>
+
kumar palaniappan 2012-04-19, 20:36
+
kumar palaniappan 2012-04-19, 20:45
+
Fernando Doglio 2012-04-19, 20:58