Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> import from Oracle to Hive : 2 errors


Copy link to this message
-
Re: import from Oracle to Hive : 2 errors
Hi Jerome,
Hive import in Sqoop is done in two phases. The first phase will transfer the data from your Oracle database to HDFS as would normal non hive import. Subsequently in the second phase Sqoop will invoke Hive to perform LOAD DATA statement to move imported data into Hive. In you first Sqoop invocation the first step obviously finished correctly, however the second phase has failed. This is the reason why the second Sqoop invocation is failing as the intermediate directory between the two phases still exists. You can unblock that by removing the directory using HDFS command, for example:

  hadoop dfs -rmr KPI.ENTITE

The second phase seems to be failing for you on following exception:

> java.lang.RuntimeException: Unable to instantiate

I would therefore suggest to take a look into Hive logs (/tmp/$USER/hive.log if I'm not mistaken) to see if there would be more details about the instantiation failure. Could you also verify that your Hive installation is configured correctly?

Jarcec

On Mon, Jun 17, 2013 at 03:46:28PM +0200, Jérôme Verdier wrote:
> Hi,
>
> I'm try to import various tables from Oracle to Hive using Sqoop, but, i
> have some errors that i don't understand.
>
> Here is my query :
>
> sqoop import --connect jdbc:oracle:thin:@my.db.server:1521/xx --username
> user --password password --create-hive-table --hive-import --table
> schema.table_xx
>
> the first error  is this one :
>
> Please set $HBASE_HOME to the root of your HBase installation.
> 13/06/17 15:36:40 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> 13/06/17 15:36:40 INFO tool.BaseSqoopTool: Using Hive-specific delimiters
> for output. You can override
> 13/06/17 15:36:40 INFO tool.BaseSqoopTool: delimiters with
> --fields-terminated-by, etc.
> 13/06/17 15:36:40 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/06/17 15:36:40 INFO tool.CodeGenTool: Beginning code generation
> 13/06/17 15:36:41 INFO manager.OracleManager: Time zone has been set to GMT
> 13/06/17 15:36:41 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM KPI.ENTITE t WHERE 1=0
> 13/06/17 15:36:41 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /usr/local/hadoop
> Note:
> /tmp/sqoop-hduser/compile/85a6dcface4ca6ca28091ed383edce2e/KPI_ENTITE.java
> uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/06/17 15:36:42 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hduser/compile/85a6dcface4ca6ca28091ed383edce2e/KPI.ENTITE.jar
> 13/06/17 15:36:42 INFO manager.OracleManager: Time zone has been set to GMT
> 13/06/17 15:36:42 WARN manager.OracleManager: The table KPI.ENTITE contains
> a multi-column primary key. Sqoop will default to the column CO_SOCIETE
> only for this job.
> 13/06/17 15:36:42 INFO manager.OracleManager: Time zone has been set to GMT
> 13/06/17 15:36:42 WARN manager.OracleManager: The table KPI.ENTITE contains
> a multi-column primary key. Sqoop will default to the column CO_SOCIETE
> only for this job.
> 13/06/17 15:36:42 INFO mapreduce.ImportJobBase: Beginning import of
> KPI.ENTITE
> 13/06/17 15:36:42 INFO manager.OracleManager: Time zone has been set to GMT
> 13/06/17 15:36:44 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
> SELECT MIN(CO_SOCIETE), MAX(CO_SOCIETE) FROM KPI.ENTITE
> 13/06/17 15:36:44 INFO mapred.JobClient: Running job: job_201306171456_0005
> 13/06/17 15:36:45 INFO mapred.JobClient:  map 0% reduce 0%
> 13/06/17 15:36:56 INFO mapred.JobClient:  map 25% reduce 0%
> 13/06/17 15:37:40 INFO mapred.JobClient:  map 50% reduce 0%
> 13/06/17 15:38:00 INFO mapred.JobClient:  map 75% reduce 0%
> 13/06/17 15:38:08 INFO mapred.JobClient:  map 100% reduce 0%
> 13/06/17 15:38:09 INFO mapred.JobClient: Job complete: job_201306171456_0005
> 13/06/17 15:38:09 INFO mapred.JobClient: Counters: 18
> 13/06/17 15:38:09 INFO mapred.JobClient:   Job Counters
> 13/06/17 15:38:09 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=151932