Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Classpath issues with a custom loadfunc


Copy link to this message
-
Re: Classpath issues with a custom loadfunc
Hi Niels,

Pig supports globbing in REGISTER statements. You could place all the jars
in a certain location and have them all be picked up.

REGISTER *.jar
On Tue, Apr 16, 2013 at 1:44 PM, Niels Basjes <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I'm attempting to build a custom LoadFunc for pig and I'm running into a
> rather silly issue.
>
> My project has several dependencies and I've been trying to create a single
> jar that contains all of these dependencies (I'm using the maven assembly
> plugin).
>
> Now if I manually place all dependencies in my pig script using a list of
> "REGISTER foo.jar" statements it all works.
> What I want is to have a single jar with everything in it so that I only
> need a single jar in the pig code.
>
> After some playing around I found that it I unpack all the dependencies
> inside my jar and unpack them in the root of the jar it all seems to work
> ... and turns into a big mess with warnings about "[INFO]
> META-INF/MANIFEST.MF already added, skipping".
>
> What is the clean way of doing this and still let PIG find the dependencies
> inside the jar?
> Can anyone point me towards a working example?
>
> Thanks.
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB