Marco Gallotta 2012-08-02, 23:52
syed kather 2012-08-03, 00:23
You can disable retries by setting
mapred.map.max.attempts and mapred.reduce.max.attempts to 1.
Also if you need to disable speculative execution. You can disable it by setting
mapred.map.tasks.speculative.execution and mapred.reduce.tasks.speculative.execution to false.
With these two steps you can ensure that a task is attempted only once.
These properties to be set in mapred-site.xml or at job level.
Sent from handheld, please excuse typos.
From: Marco Gallotta <[EMAIL PROTECTED]>
Date: Thu, 2 Aug 2012 16:52:00
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Disable retries
Is there a way to disable retries when a mapper/reducer fails? I'm writing data in my mapper and I'd rather catch the failure, recover from a backup (fairly lightweight in this case, as the output tables aren't big) and restart.
Marco Gallotta | Mountain View, California
Software Engineer, Infrastructure | Loki Studios
fb.me/marco.gallotta | twitter.com/marcog
[EMAIL PROTECTED] | +1 (650) 417-3313
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
Marco Gallotta 2012-08-03, 00:29
Harsh J 2012-08-03, 06:15