Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # dev >> Review Request: SQOOP-746 Sqoop2: Mapreduce job should have running with speculative execution disabled


Copy link to this message
-
Re: Review Request: SQOOP-746 Sqoop2: Mapreduce job should have running with speculative execution disabled


> On Dec. 12, 2012, 9:27 p.m., Cheolsoo Park wrote:
> > submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java, line 178
> > <https://reviews.apache.org/r/8436/diff/1/?file=236169#file236169line178>
> >
> >     Hi Jarcec,
> >    
> >     Do we have to worry about Hadoop-2.0.x?
> >    
> >     According to Hadoop docs, "mapred.map.tasks.speculative.execution" is depreciated:
> >     http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/DeprecatedProperties.html

That is excellent question Cheolsoo. Thanks for asking. This property is indeed deprecated on Hadoop 2.0, however is still properly working :-) I've tested it on real Hadoop 2 cluster (based on BigTop 0.4.0).
- Jarek
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8436/#review14384
-----------------------------------------------------------
On Dec. 9, 2012, 10:28 p.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8436/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2012, 10:28 p.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> I've disabled speculative execution in mapreduce submission engine.
>
>
> This addresses bug SQOOP-746.
>     https://issues.apache.org/jira/browse/SQOOP-746
>
>
> Diffs
> -----
>
>   submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java 8f7864eba448774b7313a283ea3f617f58ad749f
>
> Diff: https://reviews.apache.org/r/8436/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jarek Cecho
>
>