Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> ClassNotFoundException thrown by DBConfiguration


Copy link to this message
-
Re: ClassNotFoundException thrown by DBConfiguration
Hi,

For option 1: I can not add jars in our cluster as no permission. And we
also want to keep our cluster clean and stable.

For option 2:  It does not work as the jars from "-libjar" are not loaded
in the client process.

However, an optional solution is adding my jar to HADOOP_CLASSPATH.

But here I want to know is why does DBConfiguration not use
"conf.getClassByName" to load the jdbc driver?

2013/4/19 vasanth kumar <[EMAIL PROTECTED]>

> Hi,
>
> No need to pack mysql packages into your jar.
> Option 1: Copy mysql jdbc jar into $HADOOP_HOME/lib. Restart hadoop
> daemons.
>
> Option 2: while executing try : *$hadoop jar your.jar a.b.c.Main -libjar
> /path/mysql.jar*
>
>
> On Fri, Apr 19, 2013 at 12:32 PM, Shixiong Zhu <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> I encountered a ClassNotFoundException when I used DBInputFormat. I
>> packaged my codes with mysql-connector-java-5.1.24.jar to abc.jar. Then I
>> run the following command to run:
>>
>> hadoop jar abc.jar a.b.c.Main
>>
>> But it is crashed by the following exception:
>>
>>   Notes: java.lang.RuntimeException: java.lang.RuntimeException:
>> java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
>>  at
>> org.apache.hadoop.mapreduce.lib.db.DBInputFormat.setConf(DBInputFormat.java:158)
>> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70)
>>  at
>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
>> at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:959)
>>  at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
>> at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
>>  at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:891)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:844)
>>  at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:416)
>>  at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
>> at
>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:844)
>>  at org.apache.hadoop.mapreduce.Job.submit(Job.java:481)
>> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:511)
>>  ...
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at
>> com.microstrategy.etl.alert.feedback.FeedbackDumpTask.main(FeedbackDumpTask.java:17)
>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>  at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:616)
>>  at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> com.mysql.jdbc.Driver
>>  at
>> org.apache.hadoop.mapreduce.lib.db.DBInputFormat.getConnection(DBInputFormat.java:184)
>> at
>> org.apache.hadoop.mapreduce.lib.db.DBInputFormat.setConf(DBInputFormat.java:152)
>>  ... 25 more
>> Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>>  at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>>  at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
>>  at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:186)
>>  at
>> org.apache.hadoop.mapreduce.lib.db.DBConfiguration.getConnection(DBConfiguration.java:143)
>> at
>> org.apache.hadoop.mapreduce.lib.db.DBInputFormat.getConnection(DBInputFormat.java:178)
>>  ... 26 more
>>
>> I read the DBConfiguration.java file, and found it uses "Class.forName"
>> to load "com.mysql.jdbc.Driver":
>>
>>    /** Returns a connection object to the DB.
>>    * @throws ClassNotFoundException
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB