Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Issue with third party library


Copy link to this message
-
Issue with third party library
Hi,

I was just trying to setup hadoop and run some sample programs. I ran into
some issues when trying to add a required library. I tried using the
DistributedCache methods to try adding to the classpath.

The basic code is:
  Path mysqlJar = new
Path("target/classes/META-INF/mysql-connector-java-3.1.12-bin.jar");
  fs.copyFromLocalFile(mysqlJar, mysqlJar);
  DistributedCache.addArchiveToClassPath(mysqlJar, jobConf, fs);

I did a copyFromLocal since I wasnt sure where it would get picked from. I
tried both addArchiveToClassPath and addFileToClassPath. But with both, I
get a ClassNotFoundException.

Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
com.mysql.jdbc.Driver
    at
org.apache.hadoop.mapred.lib.db.DBInputFormat.configure(DBInputFormat.java:271)
    ... 16 more

The job seems to have got the file added and also checked in the
tasktracker local dir and the file seems to be present.

<property><name>mapred.job.classpath.archives</name><value>target/classes/META-INF/mysql-connector-java-3.1.12-bin.jar</value></property>

Any clues would be helpful. I dont think I can use -libjars since I am
trying to execute the hadoop jobs from within another application and not
from the command line.

Regards,
Sampath.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB