Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> pig ship tar files


+
Danfeng Li 2012-12-20, 18:01
+
Alan Gates 2012-12-20, 18:57
+
Danfeng Li 2012-12-20, 20:00
Copy link to this message
-
Re: pig ship tar files
On Thu, Dec 20, 2012 at 01:01:49PM -0500, Danfeng Li wrote:
> I read alot of about pig can ship a tar file and untar it before
> execution. However, I couldn't find any example. Can someone provide
> an example?

The trick is to use the `SH' statement to untar the file.
 
> What I would like to do is to ship a python module, such as nltk,
> for my streaming.

Try something like (untested)

DEFINE my_cmd `relative/path/to/my_cmd/in/tar/file.py`
SHIP('nltk.tar');

SH tar xf nltk.tar

Does this help/work?

Regards,
Thomas.
+
Rohini Palaniswamy 2012-12-27, 17:40
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB