Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # dev >> Review Request: SQOOP-601: Support custom schemas in PostgreSQL Connector

Jarek Cecho 2012-09-12, 12:09
Copy link to this message
Re: Review Request: SQOOP-601: Support custom schemas in PostgreSQL Connector

This is an automatically generated e-mail. To reply, visit:

(Updated Sept. 14, 2012, 8:36 a.m.)
Review request for Sqoop.

I've incorporated Cheolsoo's changes.

This patch is adding support for custom schemas into PostgreSQL manager. Changes for import job were very simple and rather straightforward. On the other side, changes for export job were little tricky, but I believe that I've succeeded without breaking backward compatibility.
Diffs (updated)

  src/docs/user/connectors.txt a93f14e
  src/java/com/cloudera/sqoop/manager/PostgresqlManager.java 16adeb2
  src/java/org/apache/sqoop/manager/CatalogQueryManager.java 5f2f89f
  src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java a557aa1
  src/java/org/apache/sqoop/manager/PGBulkloadManager.java 92174f8
  src/java/org/apache/sqoop/manager/PostgresqlManager.java d18321c
  src/java/org/apache/sqoop/manager/SqlManager.java ea961cd
  src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java b574f82
  src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java 7be5ed9
  src/java/org/apache/sqoop/mapreduce/JdbcUpsertExportJob.java f299f98
  src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 38e9fb9
  src/test/com/cloudera/sqoop/ThirdPartyTests.java eeab7f3
  src/test/com/cloudera/sqoop/manager/PostgresqlExportTest.java PRE-CREATION
  src/test/com/cloudera/sqoop/manager/PostgresqlImportTest.java PRE-CREATION
  src/test/com/cloudera/sqoop/manager/PostgresqlTest.java 0dfd1fc

Diff: https://reviews.apache.org/r/7055/diff/

I've added a lot of third party tests (e.g. they won't be run automatically during "ant test") to properly test various combinations of parameters. You can invoke those tests issuing one of following commands:

ant test -Dtestcase=PostgresqlImportTest
ant test -Dtestcase=PostgresqlExportTest

I've tested compatibility by running ant test -Dhadoopversion={20,100,23,200}

Jarek Cecho

Jarek Cecho 2012-09-14, 08:36
Jarek Cecho 2012-09-14, 18:15
Cheolsoo Park 2012-09-14, 18:25
Cheolsoo Park 2012-09-14, 01:26
Cheolsoo Park 2012-09-14, 16:21
Jarek Cecho 2012-09-14, 18:14
Jarek Cecho 2012-09-14, 08:33