Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Re: Export in free form And Mixed update/insert


+
YouPeng Yang 2013-05-22, 10:20
+
Jarek Jarcec Cecho 2013-05-22, 17:34
+
YouPeng Yang 2013-05-24, 05:06
Copy link to this message
-
Re: Export in free form And Mixed update/insertj
Hi YouPeng,
I'm glad to hear that the upsert mode has started working for you!

Jarcec

On Fri, May 24, 2013 at 01:06:41PM +0800, YouPeng Yang wrote:
> Hi Jarek Jarcec Cecho
>
>    Thanks for your response.
>
>   I am sorry that the failed job because of the oraoop plugin which I add
> before. Anyways I post the command and the log[1].
>
>   The mixed update/insert job succeeds when I adopt the pure sqoop . Thanks
> a  lot.
>
>
>
>
> [1]---------------------------------------------------------------------------------------------------
> [hadoop@Hadoop01 ~]$  /home/sqoop-1.4.1-cdh4.1.2/bin/sqoop export --connect
> jdbc:oracle:thin:@10.167.14.225:1521:wxoss --username XUJINGYU --password
> 123456  --export-dir sqoop/CMTSIFTABLE/201305131241  --table
> XUJINGYU.CMTSIFTABLE   --fields-terminated-by "|"    --columns
> IFID,DEVICEID,IFDESC,IFINDEX,IFTYPE,IFSPEED,IFALIAS,IFNAME --update-key
> IFID,DEVICEID   --update-key IFID,CMTSID --update-mode  allowinsert
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> 13/05/24 12:58:55 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> 13/05/24 12:58:56 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/05/24 12:58:57 INFO oraoop.OraOopOracleQueries: Current schema is:
> XUJINGYU
> 13/05/24 12:58:57 INFO oraoop.OraOopManagerFactory:
> ***********************************************************************
> *** Using Quest庐 Data Connector for Oracle and Hadoop 1.6.0-cdh4-20 ***
> *** Copyright 2012 Quest Software, Inc.                             ***
> *** ALL RIGHTS RESERVED.                                            ***
> ***********************************************************************
> 13/05/24 12:58:57 WARN oraoop.OraOopManagerFactory:
> *****************************************************************************************************************************
> The table "XUJINGYU"."CMTSIFTABLE" does not have a valid index on the
> column(s) IFID,DEVICEID.
> As a consequence, this export may take a long time to complete.
> If performance is unacceptable, consider reattempting this job after
> creating an index on this table via the SQL...
> create index <index_name> on "XUJINGYU"."CMTSIFTABLE"(IFID,DEVICEID);
> *****************************************************************************************************************************
> 13/05/24 12:58:57 INFO oraoop.OraOopManagerFactory: Oracle Database
> version: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bi
> 13/05/24 12:58:57 INFO oraoop.OraOopManagerFactory: This Oracle database is
> not a RAC.
> 13/05/24 12:58:57 WARN conf.Configuration: mapred.map.max.attempts is
> deprecated. Instead, use mapreduce.map.maxattempts
> 13/05/24 12:58:57 INFO manager.SqlManager: Executing SQL statement: SELECT
> IFID,DEVICEID,IFDESC,IFINDEX,IFTYPE,IFSPEED,IFNAME,IFALIAS FROM
> XUJINGYU.CMTSIFTABLE WHERE 1=0
> 13/05/24 12:58:57 INFO tool.CodeGenTool: Beginning code generation
> 13/05/24 12:58:57 INFO manager.SqlManager: Executing SQL statement: SELECT
> "IFID","DEVICEID","IFDESC","IFINDEX","IFTYPE","IFSPEED","IFNAME","IFALIAS"
> FROM XUJINGYU.CMTSIFTABLE WHERE 0=1
> 13/05/24 12:58:57 INFO orm.CompilationManager: HADOOP_HOME is
> /usr/local/hadoop
> Note:
> /tmp/sqoop-hadoop/compile/7f64b56b53b34f3fe3d6eab03a5255ae/XUJINGYU_CMTSIFTABLE.java
> uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/05/24 12:58:59 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hadoop/compile/7f64b56b53b34f3fe3d6eab03a5255ae/XUJINGYU.CMTSIFTABLE.jar
> 13/05/24 12:58:59 ERROR tool.ExportTool: Error during export: Mixed
> update/insert is not supported against the target database yet
>
>
>
>
>
>
>
> 2013/5/23 Jarek Jarcec Cecho <[EMAIL PROTECTED]>
>
> > Hi YouPeng,
> > Sqoop's Oracle connector do supports upsert mode [1]. Would you mind
+
Felix GV 2013-05-22, 16:26
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB