Ryan Compton 2013-11-05, 23:39
-RE: Need example of python code with dependency files
william.dowling@... 2013-11-06, 14:23
You said "The .py code takes input from sys.stdin and outputs to sys.stdout" so I infer you are talking about streaming, not a python UDF. In that case, rather than streaming through your python script P.py, instead stream through a shell script S.sh. The shell script can untar shipped or cached files, then run P.py. Maybe not completely elegant, but this approach works for me.
William F Dowling
From: Ryan Compton [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, November 05, 2013 6:40 PM
To: [EMAIL PROTECTED]
Subject: Need example of python code with dependency files
I have some python code I'd like to deploy with a pig script. The .py
code takes input from sys.stdin and outputs to sys.stdout. It also
needs some parameter files to run properly.
The book "Programming Pig" tells me:
"The workaround for this is to create a TAR file and ship that, and
then have a step in your executable that unbundles the TAR file"
but I'm still real confused. I need an example of how to unpack it (eg
unpack in a bash script I pass around, unpack in the pig script, ??)
Ryan Compton 2013-11-09, 20:48