-Re: Need example of python code with dependency files
Ryan Compton 2013-11-09, 20:48
Thanks, this worked for some test programs, but then I ran into some
other problems when I had to import things (for some reason pig was
looking for python2.4 stuff even though I explicitly told it 2.7, I'm
assuming this is more of a CentOS/Jython issue).
I ended up using PyInstaller to package up my code and streammed though that.
On Wed, Nov 6, 2013 at 6:23 AM, <[EMAIL PROTECTED]> wrote:
> You said "The .py code takes input from sys.stdin and outputs to sys.stdout" so I infer you are talking about streaming, not a python UDF. In that case, rather than streaming through your python script P.py, instead stream through a shell script S.sh. The shell script can untar shipped or cached files, then run P.py. Maybe not completely elegant, but this approach works for me.
> William F Dowling
> Senior Technologist
> Thomson Reuters
> -----Original Message-----
> From: Ryan Compton [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, November 05, 2013 6:40 PM
> To: [EMAIL PROTECTED]
> Subject: Need example of python code with dependency files
> I have some python code I'd like to deploy with a pig script. The .py
> code takes input from sys.stdin and outputs to sys.stdout. It also
> needs some parameter files to run properly.
> The book "Programming Pig" tells me:
> "The workaround for this is to create a TAR file and ship that, and
> then have a step in your executable that unbundles the TAR file"
> but I'm still real confused. I need an example of how to unpack it (eg
> unpack in a bash script I pass around, unpack in the pig script, ??)