Pig, mail # user - Duplicate jar are getting into PigContext
Duplicate jar are getting into PigContext
Dotan Patrich 2014-03-05, 15:28
Hi, I've run a pig script the has REGISTER command in it using java code that uses the PigServer.registerScript method. On each execution of the pig script (using the same java process) the registered jar is added to the classpath of the taskjvm.sh file. This is duplicated until we get an error for having a classpath which is too long... Debuging it, I can see that the same jar from the local file system is being added multiple times to the PigContext skipJars member.
Does anyone know of an open issue regarding this? am I using the PigServer wrong? I couldn't find any open issue regarding this, so the current workaround was to remove the register statement from the pig script and call PigServer.registerJar() once when my java process starts.
Thanks for the help, Dotan
Re: Duplicate jar are getting into PigContext
Cheolsoo Park 2014-03-26, 18:51
Very sorry for the late reply.
added multiple times to the PigContext skipJars member.