Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> tmp directory for Pig


+
Jakub Glapa 2013-01-25, 15:36
Copy link to this message
-
Re: tmp directory for Pig
java.io.tmpdir should work. If you are running python script, more space is
required as the jars will cached in the java.io.tmp dir. Alternatively, you
can try a different location for it by using -Dpython.cachedir=<location>
or you can skip the caching using -Dpython.cachedir.skip=true.  My guess is
that /foo/tmp also does not have space that is why java.io.tmpdir did not
take effect. Check the space availability using df.

Regards,
Rohini
On Fri, Jan 25, 2013 at 7:36 AM, Jakub Glapa <[EMAIL PROTECTED]> wrote:

> Hi I started running jobs in pig and I'm getting exceptions like that:
> >>Caused by: java.io.IOException: No space left on device
> ...
> I've narrowed down the problem and it looks like the /tmp is to small. I
> wanted to set the java.io.tmpdir to point somewhere else like that:
>
> pig -Djava.io.tmpdir=/foo/tmp/ script.pig
>
>
> but it appears that it's not being picked by pig.
> What property could i modify if I want to avoid the need to increase the
> size of /tmp folder?
>
> Ps.
> pig-0.10.1
> running pig scripts with python embedded if it makes any difference
>
> --
> regards,
> pozdrawiam,
> Jakub Glapa
>