Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> running pig on remote cluster


Copy link to this message
-
running pig on remote cluster
Hi,

I am trying to submit a pig job to a remote cluster by setting
mapred.job.tracker and  fs.default.name accordingly.
The job does get executed on the remote cluster, however all
intermediate output is stored on the local cluster from which
pig is run.  From job configuration I can see that that
pig.reduce.output.dirs and pig.streaming.log.dir are referencing the
local cluster.
I am supposed to set these manually or is there an alternative?

pig -version
Apache Pig version 0.10.0 (r1328203)
compiled Apr 19 2012, 22:54:12

Thanks,

stan
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB