Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: "attempt*" directories in user logs

Copy link to this message
Re: "attempt*" directories in user logs
Tsuyoshi OZAWA 2012-12-10, 08:18
Hi Oleg,

Speculative tasks can be launched as TaskAttempt in MR jobs.
And, if no reducer class is set, MR launches default Reducer

On Sun, Dec 9, 2012 at 11:53 PM, Oleg Zhurakousky <

> I studying user logs on the two node cluster that I have setup and I was
> wondering if anyone can shed some light on these "attempt*' directories
> >$ ls
> attempt_201212051224_0021_m_000000_0  attempt_201212051224_0021_m_000003_0
>  job-acls.xml
> attempt_201212051224_0021_m_000002_0  attempt_201212051224_0021_r_000000_0
> I mean its obvious that its talking about 3 attempts for Map task and 1
> attempt for reduce task. However my current MR job only results in some
> output written to "attempt_201212051224_0021_m_000000_0". Nothing is the
> reduce part (understandably since I don't even have a reducer, so my
> question is:
> 1. The two more M attempts. . . what are they?
> 2. Why was there an attempt to do a Reduce when no reducer was
> provided.implemented
> 3. Why my master node only had 1 attempt for M task but the slave had all
> that's displayed and questioned above (the 'ls' output above is from the
> slave node)
> Thanks
> Oleg
OZAWA Tsuyoshi