Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> ClassNotFoundException thrown by DBConfiguration


+
Shixiong Zhu 2013-04-19, 07:02
+
vasanth kumar 2013-04-19, 08:39
+
Shixiong Zhu 2013-04-19, 08:48
Copy link to this message
-
Re: ClassNotFoundException thrown by DBConfiguration
Hi Shixiong,
would you mind describing your use case a bit more? Based on the description it seems that you're re-packaging Sqoop with your own code and mysql JDBC driver together.

Jarcec

On Fri, Apr 19, 2013 at 04:48:08PM +0800, Shixiong Zhu wrote:
> Hi,
>
> For option 1: I can not add jars in our cluster as no permission. And we
> also want to keep our cluster clean and stable.
>
> For option 2:  It does not work as the jars from "-libjar" are not loaded
> in the client process.
>
> However, an optional solution is adding my jar to HADOOP_CLASSPATH.
>
> But here I want to know is why does DBConfiguration not use
> "conf.getClassByName" to load the jdbc driver?
>
> 2013/4/19 vasanth kumar <[EMAIL PROTECTED]>
>
> > Hi,
> >
> > No need to pack mysql packages into your jar.
> > Option 1: Copy mysql jdbc jar into $HADOOP_HOME/lib. Restart hadoop
> > daemons.
> >
> > Option 2: while executing try : *$hadoop jar your.jar a.b.c.Main -libjar
> > /path/mysql.jar*
> >
> >
> > On Fri, Apr 19, 2013 at 12:32 PM, Shixiong Zhu <[EMAIL PROTECTED]> wrote:
> >
> >> Hi,
> >>
> >> I encountered a ClassNotFoundException when I used DBInputFormat. I
> >> packaged my codes with mysql-connector-java-5.1.24.jar to abc.jar. Then I
> >> run the following command to run:
> >>
> >> hadoop jar abc.jar a.b.c.Main
> >>
> >> But it is crashed by the following exception:
> >>
> >>   Notes: java.lang.RuntimeException: java.lang.RuntimeException:
> >> java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
> >>  at
> >> org.apache.hadoop.mapreduce.lib.db.DBInputFormat.setConf(DBInputFormat.java:158)
> >> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70)
> >>  at
> >> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
> >> at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:959)
> >>  at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
> >> at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
> >>  at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:891)
> >> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:844)
> >>  at java.security.AccessController.doPrivileged(Native Method)
> >> at javax.security.auth.Subject.doAs(Subject.java:416)
> >>  at
> >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
> >> at
> >> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:844)
> >>  at org.apache.hadoop.mapreduce.Job.submit(Job.java:481)
> >> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:511)
> >>  ...
> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> >> at
> >> com.microstrategy.etl.alert.feedback.FeedbackDumpTask.main(FeedbackDumpTask.java:17)
> >>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at
> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >>  at
> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> at java.lang.reflect.Method.invoke(Method.java:616)
> >>  at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> >> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> >> com.mysql.jdbc.Driver
> >>  at
> >> org.apache.hadoop.mapreduce.lib.db.DBInputFormat.getConnection(DBInputFormat.java:184)
> >> at
> >> org.apache.hadoop.mapreduce.lib.db.DBInputFormat.setConf(DBInputFormat.java:152)
> >>  ... 25 more
> >> Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> >>  at java.security.AccessController.doPrivileged(Native Method)
> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> >>  at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
> >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> >>  at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
> >> at java.lang.Class.forName0(Native Method)
> >> at java.lang.Class.forName(Class.java:186)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB