Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Incremental import from Oracle - last-value not getting updated in metastore


Copy link to this message
-
Incremental import from Oracle - last-value not getting updated in metastore
Need help on an error encountered during incremental import. The sqoop job
completes and it does a full import rather than doing it incremental
resulting in duplicate records. From what I can deduce, the metastore is
not getting updated with the latest last-value.

Probably, this has been answered before, but I couldn't find it anywhere.
The error stack is below:

 INFO tool.ImportTool: Saving incremental import state to the metastore
 ERROR tool.ImportTool: Encountered IOException running import job:
java.io.IOException: Error communicating with database
        at
org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:426)
        at
org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.update(HsqldbJobStorage.java:445)
        at
org.apache.sqoop.tool.ImportTool.saveIncrementalState(ImportTool.java:131)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:428)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
        at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:228)
        at org.apache.sqoop.tool.JobTool.run(JobTool.java:283)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
Caused by: java.sql.SQLException: Violation of unique constraint
SQOOP_SESSIONS_UNQ: duplicate value(s) for column(s)
JOB_NAME,PROPNAME,PROPCLASS in statement [INSERT INTO SQOOP_SESSIONS
(propval, job_name, propclass, propname) VALUES (?, ?, ?, ?)]
        at org.hsqldb.jdbc.Util.throwError(Unknown Source)
        at org.hsqldb.jdbc.jdbcPreparedStatement.executeUpdate(Unknown
Source)
        at
org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.setV0Property(HsqldbJobStorage.java:707)
        at
org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:415)
        ... 12 more

Sqoop version:
Sqoop 1.4.4-mapr
git commit id 16d0124c5b5f7bc68b8f67fbe77f0c91d46d64c1

Appreciate if someone can throw some light on this.

Thanks.
Gautham
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB