-Re: Disable retries
Harsh J 2012-08-03, 06:15
You may use the APIs directly:
to avoid config strings pain.
On Fri, Aug 3, 2012 at 5:59 AM, Marco Gallotta <[EMAIL PROTECTED]> wrote:
> Great, thanks!
> Marco Gallotta | Mountain View, California
> Software Engineer, Infrastructure | Loki Studios
> fb.me/marco.gallotta | twitter.com/marcog
> [EMAIL PROTECTED] | +1 (650) 417-3313
> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> On Thursday 02 August 2012 at 5:02 PM, Bejoy KS wrote:
>> Hi Marco
>> You can disable retries by setting
>> mapred.map.max.attempts and mapred.reduce.max.attempts to 1.
>> Also if you need to disable speculative execution. You can disable it by setting
>> mapred.map.tasks.speculative.execution and mapred.reduce.tasks.speculative.execution to false.
>> With these two steps you can ensure that a task is attempted only once.
>> These properties to be set in mapred-site.xml or at job level.
>> Bejoy KS
>> Sent from handheld, please excuse typos.
>> -----Original Message-----
>> From: Marco Gallotta <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])>
>> Date: Thu, 2 Aug 2012 16:52:00
>> To: <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])>
>> Reply-To: [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])
>> Subject: Disable retries
>> Hi there
>> Is there a way to disable retries when a mapper/reducer fails? I'm writing data in my mapper and I'd rather catch the failure, recover from a backup (fairly lightweight in this case, as the output tables aren't big) and restart.
>> Marco Gallotta | Mountain View, California
>> Software Engineer, Infrastructure | Loki Studios
>> fb.me/marco.gallotta (http://fb.me/marco.gallotta) | twitter.com/marcog (http://twitter.com/marcog)
>> [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED]) | +1 (650) 417-3313
>> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)