Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # dev >> Review Request: SQOOP-740 Provide export submission engine implementation for Sqoop 2


+
Jarek Cecho 2012-12-05, 20:57
+
Bilung Lee 2012-12-07, 19:54
Copy link to this message
-
Re: Review Request: SQOOP-740 Provide export submission engine implementation for Sqoop 2

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8363/
-----------------------------------------------------------

(Updated Dec. 7, 2012, 9:54 p.m.)
Review request for Sqoop.
Changes
-------

Fixed Bilung's excellent point.
Description
-------

I've provided basic implementation for export path in submission/execution engine. Current implementation have hard coded which extractor will be used. I would suggest to solve other extractors in follow up jira.
This addresses bug SQOOP-740.
    https://issues.apache.org/jira/browse/SQOOP-740
Diffs (updated)
-----

  client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java 734276dfdd88b121c64a608cb79db2f10900ae4b
  client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java 0b685bf1ab4e783a654cfc4b5737efb8fc23a566
  connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java 13574b2d8c9c534fc43b21fe3987d751c8064947
  connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestExportLoader.java c97693dc05a6e190a94e62ca6f76df6147319151
  core/src/main/java/org/apache/sqoop/framework/ExecutionEngine.java ae14d9a8af0b2abb161c610cc8795e745fd3e110
  core/src/main/java/org/apache/sqoop/framework/FrameworkManager.java 66746437166aa05f6f3d9a454d0aeca77322cc49
  core/src/main/java/org/apache/sqoop/framework/configuration/ExportJobConfiguration.java 330aff0dfcb6aa84d2d7f504e5a3a7528ff5217d
  core/src/main/java/org/apache/sqoop/framework/configuration/ImportJobConfiguration.java c674fc2615d9f35343ebc73387a79361ba68888a
  core/src/main/java/org/apache/sqoop/framework/configuration/InputForm.java PRE-CREATION
  core/src/main/resources/framework-resources.properties db40946242e204850066e2aa63325d895dd1e221
  execution/mapreduce/src/main/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngine.java e2163ff0c74cb0d11037a2ff114f6e748350a372
  execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsExportPartitioner.java 9e7ea4e1231f3f26ad98adc5620169a037284cc7
  execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsSequenceImportLoader.java a706ea84d9605dacfc57272328dcbafcc3501979
  execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsTextImportLoader.java 55eb389084209d17fcb29072d5b7a393c6ea774b
  execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java 3bd1e1b38aa57a9fd49f9960acd58492f49db9e3
  execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsExtract.java 585fac7c82ed48f37704a951cf09b20ac547bdb0
  execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMapReduce.java c8caecd4abc4d8ec4b7d0c5055a1cecd05339de3
  spi/src/main/java/org/apache/sqoop/job/etl/Loader.java 046b939b6ca535571f20cfb42273aef2ca05f868

Diff: https://reviews.apache.org/r/8363/diff/
Testing
-------

* Tests seems to be passing
* I've tested export on real cluster
Thanks,

Jarek Cecho

+
Bilung Lee 2012-12-07, 23:59