Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Embedded Pig and MatPlotLib


Copy link to this message
-
Re: Embedded Pig and MatPlotLib
Hi,
As a follow-up question, has anyone tried running the Pig part of the
Jython script through Execnet? Is that doable? I was running the whole
thing through pig before, but if I could just feed the part that runs
pig through a Jython interpreter through execnet, I could have the rest
of my code be in regular Python and then freely use MatPlotLib, and
whatever other Python libraries I want that aren't available to Jython.
Annoyingly, It doesn't seem trivial to get this to work. Here's what I
thought should work:

import execnet
gw = execnet.makegateway("popen//python=jython")
channel = gw.remote_exec("""
     from org.apache.pig.scripting import Pig
     P = Pig.compile(
         'a = load "$in"; store a into "$out";')
     input = '/data/dictionaries/Eng-Spa.dic'
     output = 'mehblah'
     result = P.bind({'in':input, 'out':output}).runSingle()
     if result.isSuccessful() :
         print 'weeeee'
     else:
         raise 'boooo'
""")

Ideas? Or does anyone know a better way of doing this? All I want to do
is use pig and MatPlotLib in the same framework.
On 3/15/12 3:46 PM, Eli Finkelshteyn wrote:
> Hey folks,
> Maybe this isn't the best place for this question, but I'm thinking
> maybe someone here ran into something similar, so I'll try anyway. I'm
> currently trying to run an embedded pig script and then pass my
> results on to a separate module that uses matplotlib for some
> graphing. Problem is, matplotlib isn't supported by Jython, which is
> necessary to run embedded pig, so I'm winding up needing Jython for
> part of the program and Python for another part. It would be really
> nice if there was a way I could call Python from Jython, or Jython
> from Python to be able to run the whole thing, while keeping its
> modular nature. Any ideas?
>
> Eli
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB