Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop, mail # dev - Review Request: SQOOP-476 If table name is a qualified name, Sqoop import fails in DB2


+
Cheolsoo Park 2012-04-07, 00:30
+
Bilung Lee 2012-05-15, 22:28
Copy link to this message
-
Re: Review Request: SQOOP-476 If table name is a qualified name, Sqoop import fails in DB2
Bilung Lee 2012-06-01, 17:22


> On 2012-05-15 22:28:27, Bilung Lee wrote:
> > Thanks for the patch!  It looks good overall.  One comment though.  It seems that two DB2 record readers are
> > provided, but only one of them is actually used by the Db2Manager and is tested by the test case.  Unless the
> > generic JDBC manager also works, there is probably no need for the other record reader.
>
> Cheolsoo Park wrote:
>     Hello Bilung, thank you very much for pointing that out.
>    
>     In fact, I've verified that GenericJdbcManager also works with DB2, so I decided to extend the unit tests to test GenericJdbManager as well as Db2Manager. Now I am testing:
>    
>     1) Non-qualified table name
>        a) GenericJdbcManager
>        b) Db2Manager
>    
>     2) Qualified table name
>        a) GenericJdbcManager
>        b) Db2Manager

Thanks for verifying this and extending the test cases.
- Bilung
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4674/#review7917
-----------------------------------------------------------
On 2012-05-27 19:15:12, Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4674/
> -----------------------------------------------------------
>
> (Updated 2012-05-27 19:15:12)
>
>
> Review request for Sqoop.
>
>
> Summary
> -------
>
> During the import job, Sqoop uses the following select query:
>
> SELECT <column names> FROM <table name> AS <table name>
>
> This causes a SQL syntax error for DB2 if table name has a dot "." in it.
>
>
> The changes include:
>
> 1) Create DB2-specific DBInputFormat/DBRecordReader + DataDrivenDBInputFormat. Note that DataDrivenDBRecordReader is not subclassed since DB2-specific logic is implemented directly in its getSelectQuery() method.
>
> 2) Add a unit test that imports data from a table that has a qualified name.
>
>
> This addresses bug SQOOP-476.
>     https://issues.apache.org/jira/browse/SQOOP-476
>
>
> Diffs
> -----
>
>   /src/java/org/apache/sqoop/manager/Db2Manager.java 1343070
>   /src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java 1343070
>   /src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 1343070
>   /src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java PRE-CREATION
>   /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java PRE-CREATION
>   /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java PRE-CREATION
>   /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java 1343070
>
> Diff: https://reviews.apache.org/r/4674/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Cheolsoo
>
>

+
Cheolsoo Park 2012-05-27, 19:31
+
Cheolsoo Park 2012-05-27, 19:15
+
Bilung Lee 2012-06-01, 17:22