Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - Why sqoop picks up hsqldb driver jar when connect to mysql db?


Copy link to this message
-
Re: Why sqoop picks up hsqldb driver jar when connect to mysql db?
Jarek Jarcec Cecho 2013-08-11, 21:18
Hi Sam,
Sqoop uses HSQLDB version 1.8, not 2.0. There was period of time when Hadoop was forcing HSQLDB 2.0 into downstream component classpath, but that was resolved. Please check out SQOOP-571 for more background.

I've tried both 1.4.3 and 1.4.3 releases on BigTop 0.6 (that is based on Hadoop 2.0.5-alpha) and both were working for me without any issues, so I'm wondering what Hadoop version are you using?

Jarcec

On Wed, Aug 07, 2013 at 02:56:47PM +0800, sam liu wrote:
> Hi Abe,
>
> I tried with --driver param, but it does not work for me, and returned the
> same exception.
>
> My command is 'bin/sqoop import --driver com.mysql.jdbc.Driver --connect
> jdbc:mysql://hostname:3306/test  --username user --password pwd --table
> testtbl -m 1 --target-dir /tmp/test6'.
>
> Thanks!
>
>
> 2013/8/7 Abraham Elmahrek <[EMAIL PROTECTED]>
>
> > Sam,
> >
> > Did you try the --driver parameter? It allows you to choose the JDBC
> > driver class.
> >
> > -Abe
> >
> >
> > On Tue, Aug 6, 2013 at 7:54 PM, sam liu <[EMAIL PROTECTED]> wrote:
> >
> >> I think this is a defect of Sqoop, and opened a jira for it:
> >> https://issues.apache.org/jira/browse/SQOOP-1164. I set the priority as
> >> Minor, because it has the workaround: only leave the target driver jar file
> >> in the sqoop/lib folder, but remove other driver jar files from the folder.
> >> However, it should be mentioned that when we have many dirver jar files
> >> under sqoop/lib folder for multiple kinds of databases, or a script will
> >> connect to different databases, this workaround can not work well.
> >>
> >> Thanks!
> >>
> >>
> >> 2013/8/6 sam liu <[EMAIL PROTECTED]>
> >>
> >>> Hi Experts,
> >>>
> >>> Under SQOOP/lib, I have both hsqldb driver jar file and mysql driver jar
> >>> file:
> >>> - lib/hsqldb-1.8.0.10.jar
> >>> - lib/mysql-connector-java-5.1.20-bin.jar
> >>>
> >>> But, when I try to connect to mysql db using sqoop import tool, sqoop
> >>> will pick up hsqldb driver jar and returned following exception. Even if I
> >>> add '--connection-manager org.apache.sqoop.manager.MySQLManager' into the
> >>> command, this issue still exists. But, after I removed the hsqldb jar file
> >>> from lib folder, this issue does not exist any more.
> >>>
> >>> Why? What's the correct method to invoke import tool to avoid such
> >>> issue? Thanks!
> >>>
> >>> > bin/sqoop import --connect jdbc:mysql://hostname:3306/test
> >>> --connection-manager org.apache.sqoop.manager.MySQLManager --username user
> >>> --password pwd --table testtbl -m 1 --target-dir /tmp/test5
> >>> 13/08/05 18:44:03 WARN tool.BaseSqoopTool: Setting your password on the
> >>> command-line is insecure. Consider using -P instead.
> >>> 13/08/05 18:44:03 INFO manager.MySQLManager: Preparing to use a MySQL
> >>> streaming resultset.
> >>> 13/08/05 18:44:03 INFO tool.CodeGenTool: Beginning code generation
> >>> Exception in thread "main" java.lang.NoSuchMethodError:
> >>> org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
> >>>         at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
> >>>         at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
> >>>         at java.sql.DriverManager.getConnection(DriverManager.java:402)
> >>>         at java.sql.DriverManager.getConnection(DriverManager.java:450)
> >>>         at
> >>> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:745)
> >>>         at
> >>> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
> >>>         at
> >>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
> >>>         at
> >>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
> >>>         at
> >>> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
> >>>         at
> >>> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
> >>>         at
> >>> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:283)