Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # dev >> Review Request: Sqoop2: HdfsExportPartitioner is not always respecting maximal number of partitions


+
rj.vasanthkumar@... 2013-03-26, 21:42
+
Jarek Cecho 2013-03-31, 03:28
Copy link to this message
-
Re: Review Request: Sqoop2: HdfsExportPartitioner is not always respecting maximal number of partitions

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10143/
-----------------------------------------------------------

(Updated April 2, 2013, 11:54 a.m.)
Review request for Sqoop.
Changes
-------

Implemented Jarek's suggestion and added test case.
Description
-------

HdfsExportPartitioner is not always respecting maximal number of partitions.
Modified partition logic.
This addresses bug sqoop-844.
    https://issues.apache.org/jira/browse/sqoop-844
Diffs (updated)
-----

  execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsExportPartitioner.java 115ca54
  execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsExtract.java 62f3a03

Diff: https://reviews.apache.org/r/10143/diff/
Testing (updated)
-------

Before using this patch, for simulating the failure in base code.
In TestHdfsExtract.java, change NUMBER_OF_ROWS_PER_FILE = 1. Then try running particular test case given in this patch. Where returning partitions size is greater than required.

Fix will create partitions less than or equal to required. Fixed.
Thanks,

vasanthkumar

+
Jarek Cecho 2013-04-13, 18:40