Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Multiple reduce task retries running at same time


Copy link to this message
-
Re: Multiple reduce task retries running at same time
Harsh J 2013-01-29, 16:48
Hi Ben,

Take a look at the Speculative Execution feature of MR which should
answer your question. See its section under 'Fault Tolerance' here:
http://developer.yahoo.com/hadoop/tutorial/module4.html#tolerence.

On Tue, Jan 29, 2013 at 1:08 PM, Ben Kim <[EMAIL PROTECTED]> wrote:
> Attached a screenshot showing the retries
>
>
> On Tue, Jan 29, 2013 at 4:35 PM, Ben Kim <[EMAIL PROTECTED]> wrote:
>>
>> Hi!
>>
>> I have come across the situation where i found a single reducer task
>> executing with multiple retries simultaneously.
>> Which is potent for slowing down the whole reduce process for large data
>> sets.
>>
>> Is this pretty normal to yall for hadoop 1.0.3?
>>
>> --
>>
>> Benjamin Kim
>> benkimkimben at gmail
>
>
>
>
> --
>
> Benjamin Kim
> benkimkimben at gmail

--
Harsh J