-PigServer memory leak due to calling File.deleteOnExit() for job jars.
Ankit Malpani 2013-03-12, 16:48
Hello Pig Gurus,
I am using PigServer (
schedule jobs on production (~100 per day) and realized that the cleanup of
job jar files on the local filesystem is triggered by calling
java.io.file.deleteOnExit() - indicating that these tmp files get deleted
only when the jvm shuts down gracefully. If my understanding is correct,
even if I write a separate cron to delete the leftover job jars, a long
running process/jvm which relies on using PigServer for executing pig
scripts would eventually run out of memory.
Has anyone faced such issues and if so found a work-around?
Any help/pointers are appreciated.
Other references to similar issues