Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> import from Oracle to Hive : 2 errors


+
Jérôme Verdier 2013-06-17, 13:46
+
Jarek Jarcec Cecho 2013-06-17, 14:58
+
Jérôme Verdier 2013-06-17, 15:31
+
Jarek Jarcec Cecho 2013-06-17, 16:54
+
Jérôme Verdier 2013-06-17, 15:59
+
Jarek Jarcec Cecho 2013-06-17, 16:56
+
Jérôme Verdier 2013-06-18, 10:36
Copy link to this message
-
Re: import from Oracle to Hive : 2 errors
Hi Jerome

You can see the following message in your output.

13/06/18 12:05:21 WARN tool.BaseSqoopTool: It seems that you've specified
at least one of following:
13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --hive-home
13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --hive-overwrite
13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --create-hive-table
13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --hive-table
13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --hive-partition-key
13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --hive-partition-value
13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --map-column-hive
--create-hive-table without --hive-import does not have any effect as the
warning message showed.  So, you may want to add --hive-import to the
command line which is the enabler of hive imports.   These additional
options take effect only if you have enabled hive imports.

THanks

Venkat
On Tue, Jun 18, 2013 at 3:36 AM, Jérôme Verdier
<[EMAIL PROTECTED]>wrote:

> Hi Jarcec,
>
> Thanks for your explanations, it help me understand how Sqoop works.
>
> i'm trying import 1000 Rows for a quite Oracle big table which is divided
> in partitions to keep reasonable query time.
>
> i am using this Sqoop script, with a query to select only the first 1000
> rows :
>
> sqoop import --connect jdbc:oracle:thin:@xx.xx.xx.xx:1521/D_BI --username
> xx --password xx --create-hive-table --query 'SELECT * FROM
> DT_PILOTAGE.DEMARQUE_MAG_JOUR WHERE ROWNUM<1000 AND $CONDITIONS'
> --target-dir /home/hduser --split-by DEMARQUE_MAG_JOUR.CO_SOCIETE
> --hive-table default.DEMARQUE_MAG_JOUR
>
> the M/R job is working quite, but as we can see in the result below, datas
> are not moved to Hive.
>
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> 13/06/18 12:05:21 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> 13/06/18 12:05:21 WARN tool.BaseSqoopTool: It seems that you've specified
> at least one of following:
> 13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --hive-home
> 13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --hive-overwrite
> 13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --create-hive-table
> 13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --hive-table
> 13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --hive-partition-key
> 13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --hive-partition-value
> 13/06/18 12:05:21 WARN tool.BaseSqoopTool:      --map-column-hive
> 13/06/18 12:05:21 WARN tool.BaseSqoopTool: Without specifying parameter
> --hive-import. Please note that
> 13/06/18 12:05:21 WARN tool.BaseSqoopTool: those arguments will not be
> used in this session. Either
> 13/06/18 12:05:21 WARN tool.BaseSqoopTool: specify --hive-import to apply
> them correctly or remove them
> 13/06/18 12:05:21 WARN tool.BaseSqoopTool: from command line to remove
> this warning.
> 13/06/18 12:05:21 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/06/18 12:05:21 INFO tool.CodeGenTool: Beginning code generation
> 13/06/18 12:05:40 INFO manager.OracleManager: Time zone has been set to GMT
> 13/06/18 12:05:40 INFO manager.SqlManager: Executing SQL statement: SELECT
> * FROM DT_PILOTAGE.DEMARQUE_MAG_JOUR WHERE ROWNUM<1000 AND  (1 = 0)
> 13/06/18 12:05:40 INFO manager.SqlManager: Executing SQL statement: SELECT
> * FROM DT_PILOTAGE.DEMARQUE_MAG_JOUR WHERE ROWNUM<1000 AND  (1 = 0)
> 13/06/18 12:05:40 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /usr/local/hadoop
> Note:
> /tmp/sqoop-hduser/compile/b2b0decece541a7abda95580d7b1f0d2/QueryResult.java
> uses or overrides a deprecated API.
>
> Note: Recompile with -Xlint:deprecation for details.
> 13/06/18 12:05:41 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hduser/compile/b2b0decece541a7abda95580d7b1f0d2/QueryResult.jar
> 13/06/18 12:05:41 INFO mapreduce.ImportJobBase: Beginning query import.
> 13/06/18 12:05:42 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
Regards

Venkat
+
Jérôme Verdier 2013-06-18, 14:02
+
Jérôme Verdier 2013-06-18, 14:14