Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Including third party jar files in Map Reduce job


Copy link to this message
-
RE: Including third party jar files in Map Reduce job
Hi Utkarsh,

The usage of the jar command is like this,

Usage: hadoop jar <jar> [mainClass] args...

If you want the commons-math3.jar to be available for all the tasks you can do any one of these 1. Copy the jar file in $HADOOP_HOME/lib dir or 2. Use the generic option -libjars.

Can you give the stack trace of your problem for which class it is giving ClassNotFoundException(i.e for main class or math lib class)?

Thanks
Devaraj
________________________________________
From: Utkarsh Gupta [[EMAIL PROTECTED]]
Sent: Wednesday, April 04, 2012 12:22 PM
To: [EMAIL PROTECTED]
Subject: Including third party jar files in Map Reduce job

Hi All,

I am new to Hadoop and was trying to generate random numbers using apache commons math library.
I used Netbeans to build the jar file and the manifest has path to commons-math jar as lib/commons-math3.jar
I have placed this jar file in HADOOP_HOME/lib folder but still I am getting ClassNotFoundException.
I tried using –libjar option with $HADOOP_HOME/bin/Hadoop jar myprg.jar <inputpath> <outputpath> –libjar <jarpath>
And $HADOOP_HOME/bin/Hadoop jar myprg.jar –libjar <jarpath> <inputpath> <outputpath>
But this is not working. Please help.
Thanks and Regards
Utkarsh Gupta

**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely
for the use of the addressee(s). If you are not the intended recipient, please
notify the sender by e-mail and delete the original message. Further, you are not
to copy, disclose, or distribute this e-mail or its contents to any other person and
any such actions are unlawful. This e-mail may contain viruses. Infosys has taken
every reasonable precaution to minimize this risk, but is not liable for any damage
you may sustain as a result of any virus in this e-mail. You should carry out your
own virus checks before opening the e-mail or attachment. Infosys reserves the
right to monitor and review the content of all messages sent to or from this e-mail
address. Messages sent to or from this e-mail address may be stored on the
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB