Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> pig ship tar files


Copy link to this message
-
Re: pig ship tar files
On Thu, Dec 20, 2012 at 01:01:49PM -0500, Danfeng Li wrote:
> I read alot of about pig can ship a tar file and untar it before
> execution. However, I couldn't find any example. Can someone provide
> an example?

The trick is to use the `SH' statement to untar the file.
 
> What I would like to do is to ship a python module, such as nltk,
> for my streaming.

Try something like (untested)

DEFINE my_cmd `relative/path/to/my_cmd/in/tar/file.py`
SHIP('nltk.tar');

SH tar xf nltk.tar

Does this help/work?

Regards,
Thomas.