Sqoop, mail # dev - Review Request: SQOOP-746 Sqoop2: Mapreduce job should have running with speculative execution disabled

Review Request: SQOOP-746 Sqoop2: Mapreduce job should have running with speculative execution disabled

Jarek Cecho 2012-12-09, 22:28

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8436/
-----------------------------------------------------------

Review request for Sqoop.
Description
-------

I've disabled speculative execution in mapreduce submission engine.
This addresses bug SQOOP-746.
https://issues.apache.org/jira/browse/SQOOP-746
Diffs
-----

submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java 8f7864eba448774b7313a283ea3f617f58ad749f

Diff: https://reviews.apache.org/r/8436/diff/
Testing
-------
Thanks,

Jarek Cecho

Re: Review Request: SQOOP-746 Sqoop2: Mapreduce job should have running with speculative execution disabled

Cheolsoo Park 2012-12-12, 21:27

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8436/#review14384
-----------------------------------------------------------

submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java
<https://reviews.apache.org/r/8436/#comment30606>

   Hi Jarcec,

   Do we have to worry about Hadoop-2.0.x?

   According to Hadoop docs, "mapred.map.tasks.speculative.execution" is depreciated:
   http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
- Cheolsoo Park
On Dec. 9, 2012, 10:28 p.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8436/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2012, 10:28 p.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> I've disabled speculative execution in mapreduce submission engine.
>
>
> This addresses bug SQOOP-746.
> https://issues.apache.org/jira/browse/SQOOP-746
>
>
> Diffs
> -----
>
> submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java 8f7864eba448774b7313a283ea3f617f58ad749f
>
> Diff: https://reviews.apache.org/r/8436/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jarek Cecho
>
>

Re: Review Request: SQOOP-746 Sqoop2: Mapreduce job should have running with speculative execution disabled

Jarek Cecho 2012-12-12, 21:31

> On Dec. 12, 2012, 9:27 p.m., Cheolsoo Park wrote:
> > submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java, line 178
> > <https://reviews.apache.org/r/8436/diff/1/?file=236169#file236169line178>
> >
> > Hi Jarcec,
> >
> > Do we have to worry about Hadoop-2.0.x?
> >
> > According to Hadoop docs, "mapred.map.tasks.speculative.execution" is depreciated:
> > http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/DeprecatedProperties.html

That is excellent question Cheolsoo. Thanks for asking. This property is indeed deprecated on Hadoop 2.0, however is still properly working :-) I've tested it on real Hadoop 2 cluster (based on BigTop 0.4.0).
- Jarek
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8436/#review14384
-----------------------------------------------------------
On Dec. 9, 2012, 10:28 p.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8436/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2012, 10:28 p.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> I've disabled speculative execution in mapreduce submission engine.
>
>
> This addresses bug SQOOP-746.
> https://issues.apache.org/jira/browse/SQOOP-746
>
>
> Diffs
> -----
>
> submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java 8f7864eba448774b7313a283ea3f617f58ad749f
>
> Diff: https://reviews.apache.org/r/8436/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jarek Cecho
>
>

Re: Review Request: SQOOP-746 Sqoop2: Mapreduce job should have running with speculative execution disabled

Cheolsoo Park 2012-12-12, 21:43

> On Dec. 12, 2012, 9:27 p.m., Cheolsoo Park wrote:
> > submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java, line 178
> > <https://reviews.apache.org/r/8436/diff/1/?file=236169#file236169line178>
> >
> > Hi Jarcec,
> >
> > Do we have to worry about Hadoop-2.0.x?
> >
> > According to Hadoop docs, "mapred.map.tasks.speculative.execution" is depreciated:
> > http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
>
> Jarek Cecho wrote:
> That is excellent question Cheolsoo. Thanks for asking. This property is indeed deprecated on Hadoop 2.0, however is still properly working :-) I've tested it on real Hadoop 2 cluster (based on BigTop 0.4.0).

Thanks for the clarification!
- Cheolsoo
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8436/#review14384
-----------------------------------------------------------
On Dec. 9, 2012, 10:28 p.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8436/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2012, 10:28 p.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> I've disabled speculative execution in mapreduce submission engine.
>
>
> This addresses bug SQOOP-746.
> https://issues.apache.org/jira/browse/SQOOP-746
>
>
> Diffs
> -----
>
> submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java 8f7864eba448774b7313a283ea3f617f58ad749f
>
> Diff: https://reviews.apache.org/r/8436/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jarek Cecho
>
>

Re: Review Request: SQOOP-746 Sqoop2: Mapreduce job should have running with speculative execution disabled

Cheolsoo Park 2012-12-12, 21:43

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8436/#review14390
-----------------------------------------------------------

Ship it!
Ship It!

- Cheolsoo Park
On Dec. 9, 2012, 10:28 p.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8436/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2012, 10:28 p.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> I've disabled speculative execution in mapreduce submission engine.
>
>
> This addresses bug SQOOP-746.
> https://issues.apache.org/jira/browse/SQOOP-746
>
>
> Diffs
> -----
>
> submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java 8f7864eba448774b7313a283ea3f617f58ad749f
>
> Diff: https://reviews.apache.org/r/8436/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jarek Cecho
>
>