Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Re: Export in free form And Mixed update/insert


Copy link to this message
-
Re: Export in free form And Mixed update/insertj
Hi YouPeng,
I'm glad to hear that the upsert mode has started working for you!

Jarcec

On Fri, May 24, 2013 at 01:06:41PM +0800, YouPeng Yang wrote:
> Hi Jarek Jarcec Cecho
>
>    Thanks for your response.
>
>   I am sorry that the failed job because of the oraoop plugin which I add
> before. Anyways I post the command and the log[1].
>
>   The mixed update/insert job succeeds when I adopt the pure sqoop . Thanks
> a  lot.
>
>
>
>
> [1]---------------------------------------------------------------------------------------------------
> [hadoop@Hadoop01 ~]$  /home/sqoop-1.4.1-cdh4.1.2/bin/sqoop export --connect
> jdbc:oracle:thin:@10.167.14.225:1521:wxoss --username XUJINGYU --password
> 123456  --export-dir sqoop/CMTSIFTABLE/201305131241  --table
> XUJINGYU.CMTSIFTABLE   --fields-terminated-by "|"    --columns
> IFID,DEVICEID,IFDESC,IFINDEX,IFTYPE,IFSPEED,IFALIAS,IFNAME --update-key
> IFID,DEVICEID   --update-key IFID,CMTSID --update-mode  allowinsert
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> 13/05/24 12:58:55 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> 13/05/24 12:58:56 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/05/24 12:58:57 INFO oraoop.OraOopOracleQueries: Current schema is:
> XUJINGYU
> 13/05/24 12:58:57 INFO oraoop.OraOopManagerFactory:
> ***********************************************************************
> *** Using Quest庐 Data Connector for Oracle and Hadoop 1.6.0-cdh4-20 ***
> *** Copyright 2012 Quest Software, Inc.                             ***
> *** ALL RIGHTS RESERVED.                                            ***
> ***********************************************************************
> 13/05/24 12:58:57 WARN oraoop.OraOopManagerFactory:
> *****************************************************************************************************************************
> The table "XUJINGYU"."CMTSIFTABLE" does not have a valid index on the
> column(s) IFID,DEVICEID.
> As a consequence, this export may take a long time to complete.
> If performance is unacceptable, consider reattempting this job after
> creating an index on this table via the SQL...
> create index <index_name> on "XUJINGYU"."CMTSIFTABLE"(IFID,DEVICEID);
> *****************************************************************************************************************************
> 13/05/24 12:58:57 INFO oraoop.OraOopManagerFactory: Oracle Database
> version: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bi
> 13/05/24 12:58:57 INFO oraoop.OraOopManagerFactory: This Oracle database is
> not a RAC.
> 13/05/24 12:58:57 WARN conf.Configuration: mapred.map.max.attempts is
> deprecated. Instead, use mapreduce.map.maxattempts
> 13/05/24 12:58:57 INFO manager.SqlManager: Executing SQL statement: SELECT
> IFID,DEVICEID,IFDESC,IFINDEX,IFTYPE,IFSPEED,IFNAME,IFALIAS FROM
> XUJINGYU.CMTSIFTABLE WHERE 1=0
> 13/05/24 12:58:57 INFO tool.CodeGenTool: Beginning code generation
> 13/05/24 12:58:57 INFO manager.SqlManager: Executing SQL statement: SELECT
> "IFID","DEVICEID","IFDESC","IFINDEX","IFTYPE","IFSPEED","IFNAME","IFALIAS"
> FROM XUJINGYU.CMTSIFTABLE WHERE 0=1
> 13/05/24 12:58:57 INFO orm.CompilationManager: HADOOP_HOME is
> /usr/local/hadoop
> Note:
> /tmp/sqoop-hadoop/compile/7f64b56b53b34f3fe3d6eab03a5255ae/XUJINGYU_CMTSIFTABLE.java
> uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/05/24 12:58:59 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hadoop/compile/7f64b56b53b34f3fe3d6eab03a5255ae/XUJINGYU.CMTSIFTABLE.jar
> 13/05/24 12:58:59 ERROR tool.ExportTool: Error during export: Mixed
> update/insert is not supported against the target database yet
>
>
>
>
>
>
>
> 2013/5/23 Jarek Jarcec Cecho <[EMAIL PROTECTED]>
>
> > Hi YouPeng,
> > Sqoop's Oracle connector do supports upsert mode [1]. Would you mind