Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Delays in worker node jobs

Copy link to this message
Delays in worker node jobs
Running 1.0.2, in this case on Linux.

I was watching the processes / loads on one TaskTracker instance and
noticed that it completed it's first 8 map tasks and reported 8 free
slots (the max for this system). It then waited doing nothing for more
than 30 seconds before the next "batch" of work came in and started running.

Likewise it also has relatively long periods with all 8 cores running at
or near idle. There are no jobs failing or obvious errors in the
TaskTracker log.

What could be causing this?

Should I increase the number of map jobs to greater than number of cores
to try and keep it busier?

Harsh J 2012-08-29, 16:01
Terry Healy 2012-08-30, 01:20
Steve Loughran 2012-08-30, 11:49