Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # dev >> Review Request: Review request for SQOOP-1033 "CombineFileInputFormat does not work with paths not on default FS like ASV"


+
Shuaishuai Nie 2013-05-08, 00:38
+
Venkat Ranganathan 2013-05-08, 16:00
+
Shuaishuai Nie 2013-05-08, 17:10
+
Shuaishuai Nie 2013-05-08, 17:11
+
Shuaishuai Nie 2013-05-08, 20:12
+
Shuaishuai Nie 2013-05-09, 22:16
+
Venkat Ranganathan 2013-05-10, 04:49
+
Shuaishuai Nie 2013-05-10, 14:51
+
Jarek Cecho 2013-05-20, 14:27
+
Venkat Ranganathan 2013-05-09, 21:57
+
Shuaishuai Nie 2013-05-08, 00:30
+
Venkat Ranganathan 2013-05-12, 15:48
+
Shuaishuai Nie 2013-05-13, 20:54
+
Shuaishuai Nie 2013-05-14, 00:31
+
Shuaishuai Nie 2013-05-16, 18:48
+
Venkat Ranganathan 2013-05-21, 20:03
+
Shuaishuai Nie 2013-05-23, 17:55
+
Venkat Ranganathan 2013-05-25, 00:37
+
Abhijeet Gaikwad 2013-06-03, 10:36
+
Jarek Cecho 2013-06-09, 16:02
+
Shuaishuai Nie 2013-06-12, 17:30
+
Jarek Cecho 2013-06-09, 16:23
+
Shuaishuai Nie 2013-06-12, 17:30
+
Shuaishuai Nie 2013-06-12, 17:30
+
Venkat Ranganathan 2013-06-12, 17:51
+
Jarek Cecho 2013-06-15, 17:41
+
Shuaishuai Nie 2013-06-18, 20:57
+
Shuaishuai Nie 2013-06-18, 20:57
+
Jarek Cecho 2013-06-18, 22:09
+
Shuaishuai Nie 2013-06-18, 22:24
+
Shuaishuai Nie 2013-05-21, 21:51
+
Shuaishuai Nie 2013-05-21, 21:10
+
Shuaishuai Nie 2013-05-22, 16:59
+
Shuaishuai Nie 2013-05-30, 17:11
Copy link to this message
-
Re: Review Request: Review request for SQOOP-1056 "Implement connection resiliency in Sqoop using pluggable failure handlers"

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11320/#review21197
-----------------------------------------------------------

Ship it!
Thanks for fixing the name.

Thanks for your contribution Shuaishuai Nie

- Venkat Ranganathan
On May 30, 2013, 5:11 p.m., Shuaishuai Nie wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11320/
> -----------------------------------------------------------
>
> (Updated May 30, 2013, 5:11 p.m.)
>
>
> Review request for Sqoop and Venkat Ranganathan.
>
>
> Description
> -------
>
> Implement a pluggable way for handling connection failures, and/or intermittent errors in Sqoop. This is especially crucial in environments where the probability of connections getting reset or throttled is high.
> In case of intermittent failures in Sqoop, due to connection losses or server throttling, Sqoop does not recover from those failures. As a result, the running Sqoop task would eventually fail, and a new task is started. In those cases, Sqoop does not always guarantee that tasks can safely be restarted. For example, if part of the records is already committed to the database, then restarting the task would result in some failures like primary key violations. Even for Sqoop jobs which commit the records only at the end of the task, any failures towards the end of the task would involve reprocessing the whole range of split owned by the task, and any progress is lost. (More detail please refer to the document attached at SQOOP-1056)
>
>
> This addresses bug SQOOP-1056.
>     https://issues.apache.org/jira/browse/SQOOP-1056
>
>
> Diffs
> -----
>
>   src/java/org/apache/sqoop/manager/SQLServerManager.java eea5a1d
>   src/java/org/apache/sqoop/mapreduce/SQLServerAsyncDBExecThread.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/SQLServerExportDBExecThread.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/SQLServerResilientExportOutputFormat.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/SQLServerResilientUpdateOutputFormat.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/SQLServerUpdateDBExecThread.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/db/BasicRetrySQLFailureHandler.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java 7b9e4d2
>   src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 4b44244
>   src/java/org/apache/sqoop/mapreduce/db/SQLFailureHandler.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/db/SQLServerConnectionFailureHandler.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/db/SQLServerDBInputFormat.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/db/SQLServerDBRecordReader.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/sqlserver/SqlServerRecordReader.java 2c08f12
>
> Diff: https://reviews.apache.org/r/11320/diff/
>
>
> Testing
> -------
>
> Tested with fault injection on SQL server integration tests from SQOOP-1035
>
>
> Thanks,
>
> Shuaishuai Nie
>
>

+
Venkat Ranganathan 2013-05-30, 06:55
+
Venkat Ranganathan 2013-05-22, 06:48
+
Raghav Gautam 2013-05-31, 20:37
+
Jarek Cecho 2013-06-02, 14:07
+
Raghav Gautam 2013-06-06, 03:56
+
Raghav Gautam 2013-06-06, 03:56
+
Jarek Cecho 2013-06-09, 15:17
+
Raghav Gautam 2013-06-10, 21:29
+
Venkat Ranganathan 2013-06-10, 23:32
+
Jarek Cecho 2013-06-11, 18:54