Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # dev - Review Request: Sqoop2: HdfsExportPartitioner is not always respecting maximal number of partitions


Copy link to this message
-
Re: Review Request: Sqoop2: HdfsExportPartitioner is not always respecting maximal number of partitions
Jarek Cecho 2013-04-13, 18:40

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10143/#review19124
-----------------------------------------------------------

Ship it!
Ship It!

- Jarek Cecho
On April 2, 2013, 11:54 a.m., vasanthkumar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10143/
> -----------------------------------------------------------
>
> (Updated April 2, 2013, 11:54 a.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> HdfsExportPartitioner is not always respecting maximal number of partitions.
> Modified partition logic.
>
>
> This addresses bug sqoop-844.
>     https://issues.apache.org/jira/browse/sqoop-844
>
>
> Diffs
> -----
>
>   execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsExportPartitioner.java 115ca54
>   execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsExtract.java 62f3a03
>
> Diff: https://reviews.apache.org/r/10143/diff/
>
>
> Testing
> -------
>
> Before using this patch, for simulating the failure in base code.
> In TestHdfsExtract.java, change NUMBER_OF_ROWS_PER_FILE = 1. Then try running particular test case given in this patch. Where returning partitions size is greater than required.
>
> Fix will create partitions less than or equal to required. Fixed.
>
>
> Thanks,
>
> vasanthkumar
>
>