Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Classpath issues with a custom loadfunc


+
Niels Basjes 2013-04-16, 20:44
Copy link to this message
-
Re: Classpath issues with a custom loadfunc
Prashant Kommireddi 2013-04-16, 21:40
Hi Niels,

Pig supports globbing in REGISTER statements. You could place all the jars
in a certain location and have them all be picked up.

REGISTER *.jar
On Tue, Apr 16, 2013 at 1:44 PM, Niels Basjes <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I'm attempting to build a custom LoadFunc for pig and I'm running into a
> rather silly issue.
>
> My project has several dependencies and I've been trying to create a single
> jar that contains all of these dependencies (I'm using the maven assembly
> plugin).
>
> Now if I manually place all dependencies in my pig script using a list of
> "REGISTER foo.jar" statements it all works.
> What I want is to have a single jar with everything in it so that I only
> need a single jar in the pig code.
>
> After some playing around I found that it I unpack all the dependencies
> inside my jar and unpack them in the root of the jar it all seems to work
> ... and turns into a big mess with warnings about "[INFO]
> META-INF/MANIFEST.MF already added, skipping".
>
> What is the clean way of doing this and still let PIG find the dependencies
> inside the jar?
> Can anyone point me towards a working example?
>
> Thanks.
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
>
+
Ruslan Al-Fakikh 2013-04-16, 21:49