Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - running pig on remote cluster


Copy link to this message
-
running pig on remote cluster
Stan Rosenberg 2012-06-08, 21:08
Hi,

I am trying to submit a pig job to a remote cluster by setting
mapred.job.tracker and  fs.default.name accordingly.
The job does get executed on the remote cluster, however all
intermediate output is stored on the local cluster from which
pig is run.  From job configuration I can see that that
pig.reduce.output.dirs and pig.streaming.log.dir are referencing the
local cluster.
I am supposed to set these manually or is there an alternative?

pig -version
Apache Pig version 0.10.0 (r1328203)
compiled Apr 19 2012, 22:54:12

Thanks,

stan
+
rakesh sharma 2012-06-10, 07:23
+
Alex Rovner 2012-06-12, 22:58