Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # dev >> Review Request 13445: Fix for SQOOP-1167


Copy link to this message
-
Review Request 13445: Fix for SQOOP-1167

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13445/
-----------------------------------------------------------

Review request for Sqoop.
Bugs: SQOOP-1167
    https://issues.apache.org/jira/browse/SQOOP-1167
Repository: sqoop-trunk
Description
-------

This patch implements the following

Enhance HCat support to allow direct mode connectors by
   Creating helpers for export and import hcat mappers and refactor hcat mappers to use that
   Adding additional method for connection managers to declare their ability to exploit this feature
   Make the detection of compatibility between hcat and direct mode managers after connection managers are created
   As an example usecase, fix the netezza implementation to
       Abstract the Netezza direct mode mappers and add hcat support
       Fix Netezza connector implementation issues etc
   Add documentation
   Add Netezza tests to third party test suite
   Move Netezza tests to org.apache.namespace to be consistent with requirements for newly added tests
Diffs
-----

  src/docs/user/hcatalog.txt b8e495e
  src/java/org/apache/sqoop/manager/ConnManager.java f4b22f9
  src/java/org/apache/sqoop/manager/DirectNetezzaManager.java 4f36bf6
  src/java/org/apache/sqoop/mapreduce/ExportJobBase.java d0be570
  src/java/org/apache/sqoop/mapreduce/ImportJobBase.java ab7f21e
  src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableExportMapper.java 22b7af5
  src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatExportMapper.java PRE-CREATION
  src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatImportMapper.java PRE-CREATION
  src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableImportMapper.java bcdc9e1
  src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableTextImportMapper.java PRE-CREATION
  src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaJDBCStatementRunner.java 3a5df40
  src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportHelper.java PRE-CREATION
  src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportMapper.java 539cedf
  src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportHelper.java PRE-CREATION
  src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportMapper.java 4f0ff1b
  src/java/org/apache/sqoop/mapreduce/netezza/NetezzaExternalTableExportJob.java 7caf9be
  src/java/org/apache/sqoop/mapreduce/netezza/NetezzaExternalTableImportJob.java 0f7c1b0
  src/java/org/apache/sqoop/tool/BaseSqoopTool.java 0eca991
  src/test/com/cloudera/sqoop/ThirdPartyTests.java 0a080b6
  src/test/com/cloudera/sqoop/manager/DirectNetezzaExportManualTest.java aace924
  src/test/com/cloudera/sqoop/manager/NetezzaExportManualTest.java 43dd254
  src/test/com/cloudera/sqoop/manager/NetezzaImportManualTest.java 86f5bdd
  src/test/com/cloudera/sqoop/manager/NetezzaTestUtils.java 4bf05b8
  src/test/org/apache/sqoop/hcat/HCatalogExportTest.java 77bafcc
  src/test/org/apache/sqoop/hcat/HCatalogImportTest.java 293015e
  src/test/org/apache/sqoop/hcat/HCatalogTestUtils.java ddae5f5
  src/test/org/apache/sqoop/hcat/TestHCatalogBasic.java da803d0
  src/test/org/apache/sqoop/manager/netezza/DirectNetezzaExportManualTest.java PRE-CREATION
  src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatExportManualTest.java PRE-CREATION
  src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatImportManualTest.java PRE-CREATION
  src/test/org/apache/sqoop/manager/netezza/NetezzaExportManualTest.java PRE-CREATION
  src/test/org/apache/sqoop/manager/netezza/NetezzaImportManualTest.java PRE-CREATION
  src/test/org/apache/sqoop/manager/netezza/NetezzaTestUtils.java PRE-CREATION

Diff: https://reviews.apache.org/r/13445/diff/
Testing
-------

Add additional tests for testing this feature.   Ran all tests and all of them passed
Thanks,

Venkat Ranganathan

+
Jarek Cecho 2013-09-03, 09:18
+
Venkat Ranganathan 2013-09-04, 06:09
+
Venkat Ranganathan 2013-09-07, 06:08
+
Jarek Cecho 2013-09-09, 15:19