Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Classpath issues with a custom loadfunc

Niels Basjes 2013-04-16, 20:44
Copy link to this message
Re: Classpath issues with a custom loadfunc
Hi Niels,

Pig supports globbing in REGISTER statements. You could place all the jars
in a certain location and have them all be picked up.

On Tue, Apr 16, 2013 at 1:44 PM, Niels Basjes <[EMAIL PROTECTED]> wrote:

> Hi,
> I'm attempting to build a custom LoadFunc for pig and I'm running into a
> rather silly issue.
> My project has several dependencies and I've been trying to create a single
> jar that contains all of these dependencies (I'm using the maven assembly
> plugin).
> Now if I manually place all dependencies in my pig script using a list of
> "REGISTER foo.jar" statements it all works.
> What I want is to have a single jar with everything in it so that I only
> need a single jar in the pig code.
> After some playing around I found that it I unpack all the dependencies
> inside my jar and unpack them in the root of the jar it all seems to work
> ... and turns into a big mess with warnings about "[INFO]
> META-INF/MANIFEST.MF already added, skipping".
> What is the clean way of doing this and still let PIG find the dependencies
> inside the jar?
> Can anyone point me towards a working example?
> Thanks.
> --
> Best regards / Met vriendelijke groeten,
> Niels Basjes
Ruslan Al-Fakikh 2013-04-16, 21:49