Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # dev >> Review Request: SQOOP-476 If table name is a qualified name, Sqoop import fails in DB2


Copy link to this message
-
Re: Review Request: SQOOP-476 If table name is a qualified name, Sqoop import fails in DB2


> On 2012-05-15 22:28:27, Bilung Lee wrote:
> > Thanks for the patch!  It looks good overall.  One comment though.  It seems that two DB2 record readers are
> > provided, but only one of them is actually used by the Db2Manager and is tested by the test case.  Unless the
> > generic JDBC manager also works, there is probably no need for the other record reader.

Hello Bilung, thank you very much for pointing that out.

In fact, I've verified that GenericJdbcManager also works with DB2, so I decided to extend the unit tests to test GenericJdbManager as well as Db2Manager. Now I am testing:

1) Non-qualified table name
   a) GenericJdbcManager
   b) Db2Manager

2) Qualified table name
   a) GenericJdbcManager
   b) Db2Manager
- Cheolsoo
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4674/#review7917
-----------------------------------------------------------
On 2012-05-27 19:15:12, Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4674/
> -----------------------------------------------------------
>
> (Updated 2012-05-27 19:15:12)
>
>
> Review request for Sqoop.
>
>
> Summary
> -------
>
> During the import job, Sqoop uses the following select query:
>
> SELECT <column names> FROM <table name> AS <table name>
>
> This causes a SQL syntax error for DB2 if table name has a dot "." in it.
>
>
> The changes include:
>
> 1) Create DB2-specific DBInputFormat/DBRecordReader + DataDrivenDBInputFormat. Note that DataDrivenDBRecordReader is not subclassed since DB2-specific logic is implemented directly in its getSelectQuery() method.
>
> 2) Add a unit test that imports data from a table that has a qualified name.
>
>
> This addresses bug SQOOP-476.
>     https://issues.apache.org/jira/browse/SQOOP-476
>
>
> Diffs
> -----
>
>   /src/java/org/apache/sqoop/manager/Db2Manager.java 1343070
>   /src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java 1343070
>   /src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 1343070
>   /src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java PRE-CREATION
>   /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java PRE-CREATION
>   /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java PRE-CREATION
>   /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java 1343070
>
> Diff: https://reviews.apache.org/r/4674/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Cheolsoo
>
>