Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # dev - Review Request: SQOOP-749: Exports Using Stored Procedures (Functions)


Copy link to this message
-
Review Request: SQOOP-749: Exports Using Stored Procedures (Functions)
Nick White 2013-01-03, 00:36

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8806/
-----------------------------------------------------------

Review request for Sqoop and Jarek Cecho.
Description
-------

It'd be useful if you could use stored procedures (or functions) to insert data - currently you can only use insert or update statements (or upsert / merges, depending on the SqlManager you're using). This would help sqoop adoption / migration into environments which have existing, SQL-based data import workflows.

The attached patch adds a --call argument to the export tool that can be used in place of --table.

Note that this patch depends on HSQLDB 2.x, whereas trunk currently depends on 1.8, so it can't be committed until that situation's resolved (see SQOOP-519 and linked tickets).
This addresses bug SQOOP-749.
    https://issues.apache.org/jira/browse/SQOOP-749
Diffs
-----

  src/java/org/apache/sqoop/SqoopOptions.java 613f797
  src/java/org/apache/sqoop/manager/ConnManager.java 21eea93
  src/java/org/apache/sqoop/manager/SqlManager.java 3a52c6d
  src/java/org/apache/sqoop/mapreduce/ExportCallOutputFormat.java PRE-CREATION
  src/java/org/apache/sqoop/mapreduce/JdbcCallExportJob.java PRE-CREATION
  src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 7c52110
  src/java/org/apache/sqoop/orm/ClassWriter.java b73711e
  src/java/org/apache/sqoop/tool/BaseSqoopTool.java d795646
  src/java/org/apache/sqoop/tool/ExportTool.java acd296d
  src/test/com/cloudera/sqoop/SmokeTests.java 76df6cf
  src/test/com/cloudera/sqoop/TestConnFactory.java 893b388
  src/test/com/cloudera/sqoop/TestExport.java eba10aa
  src/test/com/cloudera/sqoop/testutil/ExportJobTestCase.java 4f6fd37
  src/test/org/apache/sqoop/TestExportUsingProcedure.java PRE-CREATION

Diff: https://reviews.apache.org/r/8806/diff/
Testing
-------

I've added a subclass of TestExport to make sure all the usual export functionality works. I've also used it on live data exporting to postgres.
Thanks,

Nick White