Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Incremental import from Oracle - last-value not getting updated in metastore


Copy link to this message
-
Re: Incremental import from Oracle - last-value not getting updated in metastore
Gautum,

Would you mind sharing your sqoop command that you are executing as well as
the sqoop logs generated with the --verbose option?

-Abe
On Wed, Dec 11, 2013 at 7:30 PM, Gautham V <[EMAIL PROTECTED]> wrote:

> Need help on an error encountered during incremental import. The sqoop job
> completes and it does a full import rather than doing it incremental
> resulting in duplicate records. From what I can deduce, the metastore is
> not getting updated with the latest last-value.
>
> Probably, this has been answered before, but I couldn't find it anywhere.
> The error stack is below:
>
>  INFO tool.ImportTool: Saving incremental import state to the metastore
>  ERROR tool.ImportTool: Encountered IOException running import job:
> java.io.IOException: Error communicating with database
>         at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:426)
>         at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.update(HsqldbJobStorage.java:445)
>          at
> org.apache.sqoop.tool.ImportTool.saveIncrementalState(ImportTool.java:131)
>         at
> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:428)
>         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
>         at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:228)
>         at org.apache.sqoop.tool.JobTool.run(JobTool.java:283)
>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> Caused by: java.sql.SQLException: Violation of unique constraint
> SQOOP_SESSIONS_UNQ: duplicate value(s) for column(s)
> JOB_NAME,PROPNAME,PROPCLASS in statement [INSERT INTO SQOOP_SESSIONS
> (propval, job_name, propclass, propname) VALUES (?, ?, ?, ?)]
>         at org.hsqldb.jdbc.Util.throwError(Unknown Source)
>         at org.hsqldb.jdbc.jdbcPreparedStatement.executeUpdate(Unknown
> Source)
>         at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.setV0Property(HsqldbJobStorage.java:707)
>         at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:415)
>         ... 12 more
>
> Sqoop version:
> Sqoop 1.4.4-mapr
> git commit id 16d0124c5b5f7bc68b8f67fbe77f0c91d46d64c1
>
> Appreciate if someone can throw some light on this.
>
> Thanks.
> Gautham
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB