Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Embedded Pig and MatPlotLib

Copy link to this message
Re: Embedded Pig and MatPlotLib
Eli Finkelshteyn 2012-03-16, 15:33
As a follow-up question, has anyone tried running the Pig part of the
Jython script through Execnet? Is that doable? I was running the whole
thing through pig before, but if I could just feed the part that runs
pig through a Jython interpreter through execnet, I could have the rest
of my code be in regular Python and then freely use MatPlotLib, and
whatever other Python libraries I want that aren't available to Jython.
Annoyingly, It doesn't seem trivial to get this to work. Here's what I
thought should work:

import execnet
gw = execnet.makegateway("popen//python=jython")
channel = gw.remote_exec("""
     from org.apache.pig.scripting import Pig
     P = Pig.compile(
         'a = load "$in"; store a into "$out";')
     input = '/data/dictionaries/Eng-Spa.dic'
     output = 'mehblah'
     result = P.bind({'in':input, 'out':output}).runSingle()
     if result.isSuccessful() :
         print 'weeeee'
         raise 'boooo'

Ideas? Or does anyone know a better way of doing this? All I want to do
is use pig and MatPlotLib in the same framework.
On 3/15/12 3:46 PM, Eli Finkelshteyn wrote:
> Hey folks,
> Maybe this isn't the best place for this question, but I'm thinking
> maybe someone here ran into something similar, so I'll try anyway. I'm
> currently trying to run an embedded pig script and then pass my
> results on to a separate module that uses matplotlib for some
> graphing. Problem is, matplotlib isn't supported by Jython, which is
> necessary to run embedded pig, so I'm winding up needing Jython for
> part of the program and Python for another part. It would be really
> nice if there was a way I could call Python from Jython, or Jython
> from Python to be able to run the whole thing, while keeping its
> modular nature. Any ideas?
> Eli