Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Unexpected Hadoop behavior: map task re-running after reducer has been running


Copy link to this message
-
Unexpected Hadoop behavior: map task re-running after reducer has been running
David Parks 2013-03-11, 08:23
I can't explain this behavior, can someone help me here:

 

  Kind  % Complete Num Tasks Pending Running Complete Killed Failed/Killed
Task Attempts

    map      100.00%    23547       0       1    23546      0       247 / 0

   reduce     62.40%    10000    3738      30     6232      0       336 / 0

 

This job has been running for 48 hours and the reducer is quite a ways
through its processing. But we've hit a snag (I'm not sure what snag
exactly).

 

The map tasks were 100% complete, none running, but now I see 1 map task
running. In a few minutes that map task will finish and I'll see "Running
map tasks" change to 0, the # of failed map tasks will increment by 1, and
the map task will run again a short time thereafter. This seems perpetual as
we had 1 map task failed at the end of the map processing when the reducer
started running, now we have 247 failed map tasks and this is the pattern
I've been watching for hours now.

 

Anyone want to venture some guesses here?

 

Thanks,

David

 

 

 

 

 

+
David Parks 2013-03-11, 08:30
+
Ravi Prakash 2013-03-11, 17:00
+
Bejoy Ks 2013-03-11, 13:16