Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Sqoop import from Oracle DB to Hive


+
Jérôme VERDIER 2013-06-11, 07:36
Copy link to this message
-
Re: Sqoop import from Oracle DB to Hive
Hi Jérôme

The obvious missing parameters are, --target-dir, --num-mappers/-m,
--hive-table

Add all these parameters to the sqoop command and see what you get.

Thanks,
Chalcy
On Tue, Jun 11, 2013 at 3:36 AM, Jérôme VERDIER
<[EMAIL PROTECTED]>wrote:

> Hi everybody,
>
> I'm trying to import data into Hive from an Oracle DB using Sqoop command.
>
> My Sqoop command is this one :
>
> sqoop import --hive-import --connect jdbc:oracle:thin@<IP>:1521/KPI
> --table KPI.ENTITE --username x --password x --verbose
>
> But, i also try this one :
>
> sqoop import --hive-import --driver oracle.jdbc.driver.oracledriver
> --connect jdbc:oracle:thin@<IP>:1521/KPI --table KPI.ENTITE --username x
> --password x --verbose
>
> the two command give me an error, you can see the entire output below :
>
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> 13/06/11 09:06:41 DEBUG tool.BaseSqoopTool: Enabled debug logging.
> 13/06/11 09:06:41 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> 13/06/11 09:06:41 INFO tool.BaseSqoopTool: Using Hive-specific delimiters
> for output. You can override
> 13/06/11 09:06:41 INFO tool.BaseSqoopTool: delimiters with
> --fields-terminated-by, etc.
> 13/06/11 09:06:41 DEBUG sqoop.ConnFactory: Loaded manager factory:
> com.cloudera.sqoop.manager.DefaultManagerFactory
> 13/06/11 09:06:41 WARN sqoop.ConnFactory: Parameter --driver is set to an
> explicit driver however appropriate connection manager is not being set
> (via --connection-manager). Sqoop is going to fall back to
> org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
> which connection manager should be used next time.
> 13/06/11 09:06:41 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/06/11 09:06:41 INFO tool.CodeGenTool: Beginning code generation
> 13/06/11 09:06:41 ERROR sqoop.Sqoop: Got exception running Sqoop:
> java.lang.RuntimeException: Could not load db driver class:
> oracle.jdbc.driver.oracledriver
> java.lang.RuntimeException: Could not load db driver class:
> oracle.jdbc.driver.oracledriver
>         at
> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:716)
>         at
> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>         at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
>         at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
>         at
> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
>         at
> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
>         at
> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:283)
>         at
> org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1255)
>         at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1072)
>         at
> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
>         at
> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
>         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>
> Of course, Oracle JDBC drivers are in SQOOP_HOME/lib, we can see this with
> ls -l command on this directory :
>
> hduser@hadoop:~$ ls -l /usr/local/sqoop/lib
> total 3464
> -rw-rw-r-- 1 hduser hadoop 224277 févr. 26 20:31 ant-contrib-1.0b3.jar
> -rw-rw-r-- 1 hduser hadoop  36455 févr. 26 20:31
> ant-eclipse-1.0-jvm1.2.jar
> -rw-rw-r-- 1 hduser hadoop 263268 févr. 26 20:31 avro-1.5.3.jar
> -rw-rw-r-- 1 hduser hadoop 168042 févr. 26 20:31 avro-ipc-1.5.3.jar
+
Venkat 2013-06-11, 14:53
+
Jérôme VERDIER 2013-06-11, 15:24
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB