Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Re: Export in free form And Mixed update/insert


Copy link to this message
-
Re: Export in free form And Mixed update/insert
Hi Jarek Jarcec Cecho

   Thanks for your response.

  I am sorry that the failed job because of the oraoop plugin which I add
before. Anyways I post the command and the log[1].

  The mixed update/insert job succeeds when I adopt the pure sqoop . Thanks
a  lot.
[1]---------------------------------------------------------------------------------------------------
[hadoop@Hadoop01 ~]$  /home/sqoop-1.4.1-cdh4.1.2/bin/sqoop export --connect
jdbc:oracle:thin:@10.167.14.225:1521:wxoss --username XUJINGYU --password
123456  --export-dir sqoop/CMTSIFTABLE/201305131241  --table
XUJINGYU.CMTSIFTABLE   --fields-terminated-by "|"    --columns
IFID,DEVICEID,IFDESC,IFINDEX,IFTYPE,IFSPEED,IFALIAS,IFNAME --update-key
IFID,DEVICEID   --update-key IFID,CMTSID --update-mode  allowinsert
Warning: /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
13/05/24 12:58:55 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
13/05/24 12:58:56 INFO manager.SqlManager: Using default fetchSize of 1000
13/05/24 12:58:57 INFO oraoop.OraOopOracleQueries: Current schema is:
XUJINGYU
13/05/24 12:58:57 INFO oraoop.OraOopManagerFactory:
***********************************************************************
*** Using Quest庐 Data Connector for Oracle and Hadoop 1.6.0-cdh4-20 ***
*** Copyright 2012 Quest Software, Inc.                             ***
*** ALL RIGHTS RESERVED.                                            ***
***********************************************************************
13/05/24 12:58:57 WARN oraoop.OraOopManagerFactory:
*****************************************************************************************************************************
The table "XUJINGYU"."CMTSIFTABLE" does not have a valid index on the
column(s) IFID,DEVICEID.
As a consequence, this export may take a long time to complete.
If performance is unacceptable, consider reattempting this job after
creating an index on this table via the SQL...
create index <index_name> on "XUJINGYU"."CMTSIFTABLE"(IFID,DEVICEID);
*****************************************************************************************************************************
13/05/24 12:58:57 INFO oraoop.OraOopManagerFactory: Oracle Database
version: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bi
13/05/24 12:58:57 INFO oraoop.OraOopManagerFactory: This Oracle database is
not a RAC.
13/05/24 12:58:57 WARN conf.Configuration: mapred.map.max.attempts is
deprecated. Instead, use mapreduce.map.maxattempts
13/05/24 12:58:57 INFO manager.SqlManager: Executing SQL statement: SELECT
IFID,DEVICEID,IFDESC,IFINDEX,IFTYPE,IFSPEED,IFNAME,IFALIAS FROM
XUJINGYU.CMTSIFTABLE WHERE 1=0
13/05/24 12:58:57 INFO tool.CodeGenTool: Beginning code generation
13/05/24 12:58:57 INFO manager.SqlManager: Executing SQL statement: SELECT
"IFID","DEVICEID","IFDESC","IFINDEX","IFTYPE","IFSPEED","IFNAME","IFALIAS"
FROM XUJINGYU.CMTSIFTABLE WHERE 0=1
13/05/24 12:58:57 INFO orm.CompilationManager: HADOOP_HOME is
/usr/local/hadoop
Note:
/tmp/sqoop-hadoop/compile/7f64b56b53b34f3fe3d6eab03a5255ae/XUJINGYU_CMTSIFTABLE.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
13/05/24 12:58:59 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-hadoop/compile/7f64b56b53b34f3fe3d6eab03a5255ae/XUJINGYU.CMTSIFTABLE.jar
13/05/24 12:58:59 ERROR tool.ExportTool: Error during export: Mixed
update/insert is not supported against the target database yet

2013/5/23 Jarek Jarcec Cecho <[EMAIL PROTECTED]>

> Hi YouPeng,
> Sqoop's Oracle connector do supports upsert mode [1]. Would you mind
> sharing with us entire Sqoop command line and log generated with parameter
> --verbose?
>
> Jarcec
>
> Links:
> 1:
> https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/manager/OracleManager.java#L401
>
> On Wed, May 22, 2013 at 06:20:06PM +0800, YouPeng Yang wrote:
> > Hi Jarek Jarcec Cecho
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB