Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # dev >> Re: Review Request 13052: Add the --bulk-load-dir option to support the HBase doBulkLoad function


Copy link to this message
-
Re: Review Request 13052: Add the --bulk-load-dir option to support the HBase doBulkLoad function


> On Sept. 2, 2013, 2:30 p.m., Jarek Cecho wrote:
> > Hi Zhancheng,
> > thank you for working on this JIRA, appreciated! Would you mind adding test cases to ensure that this functionality works as expected?

If I'm not mistaken, integration testing bulk loading is still very hard/impossible because part of the configureIncrementalLoad needs to read the partition list which fails with the combination of a mini cluster + LocalJobRunner. I'm not advocating for no tests but maybe the compromise here would be to extract the bulk loading operations and test the rest of the logic of the HBaseBulkImportJob.
- Alexandre
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13052/#review25824
-----------------------------------------------------------
On July 30, 2013, 5:32 a.m., Zhancheng Deng wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13052/
> -----------------------------------------------------------
>
> (Updated July 30, 2013, 5:32 a.m.)
>
>
> Review request for Sqoop, Jarek Cecho and vasanthkumar.
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> SQOOP-1032: Add the --bulk-load-dir option to support the HBase doBulkLoad function
>
>
> Diffs
> -----
>
>   src/java/com/cloudera/sqoop/hbase/HBasePutProcessor.java 425b0f4
>   src/java/org/apache/sqoop/SqoopOptions.java 01805f9
>   src/java/org/apache/sqoop/hbase/HBasePutProcessor.java 9ceb5bd
>   src/java/org/apache/sqoop/hbase/ToStringPutTransformer.java 5ccf311
>   src/java/org/apache/sqoop/manager/SqlManager.java 2a4992d
>   src/java/org/apache/sqoop/mapreduce/HBaseBulkImportJob.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/HBaseBulkImportMapper.java PRE-CREATION
>   src/java/org/apache/sqoop/tool/BaseSqoopTool.java 0eca991
>
> Diff: https://reviews.apache.org/r/13052/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Zhancheng Deng
>
>