Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # dev >> Review Request 14240: SQOOP-1032: Add the --bulk-load-dir option to support the HBase doBulkLoad function


+
Alexandre Normand 2013-09-19, 23:21
+
Alexandre Normand 2013-09-19, 23:35
+
Alexandre Normand 2013-09-27, 23:01
+
Alexandre Normand 2013-09-28, 00:35
+
Alexandre Normand 2013-10-01, 20:30
+
Alexandre Normand 2013-10-02, 21:21
+
Alexandre Normand 2013-10-04, 17:45
+
Alexandre Normand 2013-10-09, 23:53
+
Alexandre Normand 2013-10-10, 16:46
+
Jarek Cecho 2013-10-10, 23:25
+
Jarek Cecho 2013-10-10, 16:36
+
Jarek Cecho 2013-10-10, 23:17
+
Jarek Cecho 2013-10-09, 22:31
+
rj.vasanthkumar@... 2013-10-04, 09:45
+
Jarek Cecho 2013-10-01, 19:49
+
Alexandre Normand 2013-10-02, 20:40
+
Alexandre Normand 2013-10-03, 16:54
+
Alexandre Normand 2013-10-09, 00:09
+
Jarek Cecho 2013-10-08, 23:40
+
Jarek Cecho 2013-10-09, 21:01
Copy link to this message
-
Re: Review Request 14240: SQOOP-1032: Add the --bulk-load-dir option to support the HBase doBulkLoad function


> On Oct. 1, 2013, 7:49 p.m., Jarek Cecho wrote:
> > Thank you Alexandre for taking up this ticket!
> >
> > Would you mind adding automated tests to ensure that the functionality is indeed working?
>
> Alexandre Normand wrote:
>     I'd like to be able to integration test bulk load but this is still not a possibility in non-distributed mode. We currently have the same problems in some of our map/reduce jobs that use bulk loading. The way we've approached it with those is that we have two modes for each job: bulk load enabled/bulk load disabled. The integration tests always run with bulk load disabled. The "bulk load mode" is tested on a real cluster with automated tests that run from jenkins.
>    
>     For what it's worth, I've done the manual testing on our cluster for this patch.
>
> Jarek Cecho wrote:
>     Thank you for the feedback Alexandre! Do you happen to know if there is already a JIRA for adding the bulk load mode into the HBase MiniCluster? I tried to search for now, but didn't found it.
>
> Alexandre Normand wrote:
>     Yes, I think this captures the current blocker (although it's possible that other things in local mode would still need work once that's fixed): https://issues.apache.org/jira/browse/HBASE-7744.

You've probably seen the activity on HBASE-7744. I'm trying to get this fixed but do you think this needs to block this sqoop patch?
- Alexandre
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14240/#review26576
-----------------------------------------------------------
On Oct. 4, 2013, 5:45 p.m., Alexandre Normand wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14240/
> -----------------------------------------------------------
>
> (Updated Oct. 4, 2013, 5:45 p.m.)
>
>
> Review request for Sqoop, Jarek Cecho, Jean-Marc Spaggiari, and Vasanth kumar RJ.
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> This is the patch to address some of the comments on Zhancheng Deng's review request (https://reviews.apache.org/r/13052/) which was basically the change to add the --bulk-load-dir option to support the HBase doBulkLoad function.
>
>
> Diffs
> -----
>
>   src/java/org/apache/sqoop/SqoopOptions.java 01805f9
>   src/java/org/apache/sqoop/hbase/HBasePutProcessor.java 9ceb5bd
>   src/java/org/apache/sqoop/hbase/ToStringPutTransformer.java 5ccf311
>   src/java/org/apache/sqoop/manager/SqlManager.java 2a4992d
>   src/java/org/apache/sqoop/mapreduce/HBaseBulkImportJob.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/HBaseBulkImportMapper.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/ImportJobBase.java ab7f21e
>   src/java/org/apache/sqoop/tool/BaseSqoopTool.java 0eca991
>   src/test/com/cloudera/sqoop/TestSqoopOptions.java 03e2504
>
> Diff: https://reviews.apache.org/r/14240/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Alexandre Normand
>
>