Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # dev - Review Request: Fix for SQOOP-1029


Copy link to this message
-
Re: Review Request: Fix for SQOOP-1013
Jarek Cecho 2013-06-10, 15:36

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11537/#review21652
-----------------------------------------------------------
Hi Abe and Venkat,
thank you very much for working on this patch and supplying very valid comments! I definitely agree that current splitters have a lot room for improvement, especially for skewed values. However as this topic itself is very big, I would suggest to get this rather simpler implementation in and tweak it later. What do you think?
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java
<https://reviews.apache.org/r/11537/#comment44741>

    Whereas splitting on text based column is not the most optimal way how to use Sqoop for importing data I would argue that in some situations it might be necessary. The SQOOP-976 is more about using text based column as a base for incremental import than as a general split by column. The main issue why we've decided not to support text based columns for incremental import is that it's very easy to insert value "in the middle" and thus make entire incremental import invalid. However this issue is not the case when doing normal one time import.

connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java
<https://reviews.apache.org/r/11537/#comment44742>

    Nit: This method seems to be unused.

connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportPartitioner.java
<https://reviews.apache.org/r/11537/#comment44743>

    Extreme Nit: Would you mind removing this unused import?
Jarcec

- Jarek Cecho
On May 30, 2013, 6:25 p.m., Venkat Ranganathan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11537/
> -----------------------------------------------------------
>
> (Updated May 30, 2013, 6:25 p.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> This addresses Boolean, date, time, and timestamp splitters.
>
> THis also disallows char type splitters as discussed in SQOOP-976
>
>
> Diffs
> -----
>
>   connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java f80f30d
>   connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportPartitioner.java ee314d0
>
> Diff: https://reviews.apache.org/r/11537/diff/
>
>
> Testing
> -------
>
> Introduced new unit tests to test new functionality
> All tests pass
>
>
> Thanks,
>
> Venkat Ranganathan
>
>