Hi!

I have come across the situation where i found a single reducer task
executing with multiple retries simultaneously.
Which is potent for slowing down the whole reduce process for large data
sets.

Is this pretty normal to yall for hadoop 1.0.3?

--

*Benjamin Kim*
*benkimkimben at gmail*

Attached a screenshot showing the retries

On Tue, Jan 29, 2013 at 4:35 PM, Ben Kim <[EMAIL PROTECTED]> wrote:

> Hi!
>
> I have come across the situation where i found a single reducer task
> executing with multiple retries simultaneously.
> Which is potent for slowing down the whole reduce process for large data
> sets.
>
> Is this pretty normal to yall for hadoop 1.0.3?
>
> --
>
> *Benjamin Kim*
> *benkimkimben at gmail*
>

--

*Benjamin Kim*
*benkimkimben at gmail*

Hi Ben,

Take a look at the Speculative Execution feature of MR which should
answer your question. See its section under 'Fault Tolerance' here:
http://developer.yahoo.com/hadoop/tutorial/module4.html#tolerence.

On Tue, Jan 29, 2013 at 1:08 PM, Ben Kim <[EMAIL PROTECTED]> wrote:
> Attached a screenshot showing the retries
>
>
> On Tue, Jan 29, 2013 at 4:35 PM, Ben Kim <[EMAIL PROTECTED]> wrote:
>>
>> Hi!
>>
>> I have come across the situation where i found a single reducer task
>> executing with multiple retries simultaneously.
>> Which is potent for slowing down the whole reduce process for large data
>> sets.
>>
>> Is this pretty normal to yall for hadoop 1.0.3?
>>
>> --
>>
>> Benjamin Kim
>> benkimkimben at gmail
>
>
>
>
> --
>
> Benjamin Kim
> benkimkimben at gmail

--
Harsh J