Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Is it possible to user hadoop archive to specify third party libs


Copy link to this message
-
Re: Is it possible to user hadoop archive to specify third party libs
Check this article from Cloudera for different options.

http://www.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/

Praveen

On Tue, Jan 3, 2012 at 7:41 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> Samir,
>
> I believe HARs won't work there. But you can use a regular tar instead,
> and that should be unpacked properly.
>
> On 03-Jan-2012, at 5:38 AM, Samir Eljazovic wrote:
>
> > Hi,
> > I need to provide a lot of 3th party libraries (both java and native)
> and doing that using generic option parser (-libjars and -files arguments)
> is a little bit messy. I was wandering if it is possible to wrap all
> libraries into single har archive and use that when submitting the job?
> >
> > Just to mention that I want to avoid putting all libraries into job jar
> for two reasons:
> > 1. does not work for  native libs
> > 2. takes time to upload jar
> >
> > Thanks,
> > Samir
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB