Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # dev >> Review Request: SQOOP-749: Exports Using Stored Procedures (Functions)

Copy link to this message
Re: Review Request: SQOOP-749: Exports Using Stored Procedures (Functions)

This is an automatically generated e-mail. To reply, visit:

(Updated Jan. 7, 2013, 7:47 p.m.)
Review request for Sqoop and Jarek Cecho.

I've revised the patch so that it includes H2 as a test dependency, and uses that in the unit tests (I'm more familiar with H2 than Derby). I've also fixed the checkstyle issues and added some documentation.

It'd be useful if you could use stored procedures (or functions) to insert data - currently you can only use insert or update statements (or upsert / merges, depending on the SqlManager you're using). This would help sqoop adoption / migration into environments which have existing, SQL-based data import workflows.

The attached patch adds a --call argument to the export tool that can be used in place of --table.

Note that this patch depends on HSQLDB 2.x, whereas trunk currently depends on 1.8, so it can't be committed until that situation's resolved (see SQOOP-519 and linked tickets).
This addresses bug SQOOP-749.
Diffs (updated)

  ivy.xml 1ee60df
  ivy/libraries.properties 4c9e37d
  src/docs/user/export-purpose.txt c26eaa7
  src/docs/user/export.txt 9f600fe
  src/java/org/apache/sqoop/SqoopOptions.java 3e0ec3e
  src/java/org/apache/sqoop/manager/ConnManager.java 21eea93
  src/java/org/apache/sqoop/manager/SqlManager.java 3a52c6d
  src/java/org/apache/sqoop/mapreduce/ExportCallOutputFormat.java PRE-CREATION
  src/java/org/apache/sqoop/mapreduce/JdbcCallExportJob.java PRE-CREATION
  src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 7c52110
  src/java/org/apache/sqoop/orm/ClassWriter.java b73711e
  src/java/org/apache/sqoop/tool/BaseSqoopTool.java c0221c9
  src/java/org/apache/sqoop/tool/ExportTool.java acd296d
  src/test/com/cloudera/sqoop/SmokeTests.java 76df6cf
  src/test/com/cloudera/sqoop/TestConnFactory.java 893b388
  src/test/com/cloudera/sqoop/TestExport.java eba10aa
  src/test/com/cloudera/sqoop/testutil/ExportJobTestCase.java 4f6fd37
  src/test/org/apache/sqoop/TestExportUsingProcedure.java PRE-CREATION

Diff: https://reviews.apache.org/r/8806/diff/

I've added a subclass of TestExport to make sure all the usual export functionality works. I've also used it on live data exporting to postgres.

Nick White