Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # dev >> Review Request: SQOOP-788 Sqoop2: Import sometimes duplicate some data


Copy link to this message
-
Review Request: SQOOP-788 Sqoop2: Import sometimes duplicate some data

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8677/
-----------------------------------------------------------

Review request for Sqoop.
Description
-------

I've submitted patch with very simple but not optimal solution. I'm recreating the object each time so that it can't happen that we're changing object that is being referenced from somewhere else. This is just temporary solution that will be further improved in SQOOP-791.
This addresses bug SQOOP-788.
    https://issues.apache.org/jira/browse/SQOOP-788
Diffs
-----

  connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java b856ce6040572125bcec926de0db37ab679d7463
  execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsSequenceExportExtractor.java 16afcdb6acac6d50bcc04a029fbf487a1b902204
  execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsTextExportExtractor.java 805514065e47401ee947f9a4abb0606a4a5d70e3

Diff: https://reviews.apache.org/r/8677/diff/
Testing
-------

Unit tests are passing and I've also tested it on real cluster.
Thanks,

Jarek Cecho