Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> /tmp/ dir filling up?


Copy link to this message
-
/tmp/ dir filling up?
I specified -Djava.io.tmpdir=/my/big/partition/path to PIG_OPTS
and I can see that this is indeed set on the JVM args,

but when I ran

pig -x local my_pig_script

it still dumped temp files into /tmp, instead of the custom dir I specified.

I tried adding

SET mapred.child.java.opts '-Djava.io.tmpdir=/my/big/partition/path'

to the pig script too (this is 0.9.2 so the SET works), but it still did
not use the custom dir.
I suspect that for the LocalRunner, hadoop has a bug so it uses the
hard-coded /tmp dir instead of querying the JVM arg
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB