Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # dev - Review Request: SQOOP-863 Sqoop2: Introduce ProgressThread into Extractor and Loader


Copy link to this message
-
Re: Review Request: SQOOP-863 Sqoop2: Introduce ProgressThread into Extractor and Loader
Hari Shreedharan 2013-03-11, 06:13

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9845/#review17681
-----------------------------------------------------------
Generally looks good, but it'd be better to use the more modern thread scheduling capabilities in Java.
execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/ProgressThread.java
<https://reviews.apache.org/r/9845/#comment37551>

    Why don't you simply use a ScheduledSingleThreadedExecutor to do this? You can get rid of all of the time related stuff and the infinite loop. In addition to making the code cleaner, we can avoid one additional thread
- Hari Shreedharan
On March 11, 2013, 2:53 a.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9845/
> -----------------------------------------------------------
>
> (Updated March 11, 2013, 2:53 a.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> I've ported ProgressThread from Sqoop1 and plug it into SqoopMapper and SqoopReducer.
>
>
> This addresses bug SQOOP-863.
>     https://issues.apache.org/jira/browse/SQOOP-863
>
>
> Diffs
> -----
>
>   execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/ProgressThread.java PRE-CREATION
>   execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java 2a823032cfd91fb8fd1a8aee3ffd9d80c2e3bae6
>   execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopReducer.java d2361482771eea1a34b14c767952c5592f89c45a
>
> Diff: https://reviews.apache.org/r/9845/diff/
>
>
> Testing
> -------
>
> Unit and integration tests seems to be passing. I've also verified the functionality on real cluster.
>
>
> Thanks,
>
> Jarek Cecho
>
>