Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Pending reducers


Copy link to this message
-
Pending reducers
Evert Lammerts 2012-08-14, 10:39
Hi list,

I have a cluster running Hadoop 0.20.205 with Kerberos enabled, exposing 528 map slots and 528 reduce slots. Currently somebody is running a NORMAL priority job with 7 mappers and 400 reducers. The mappers have finished and the system is processing the reducers. Another user is running a NORMAL priority job with 1 mapper and 26 reducers. The mapper has finished, but the reducers won't come out of "pending" state. There are no other jobs running right now. We've not yet installed a different scheduler, so right now the system is using the default scheduler. How can this behavior be explained? I see mappers of multiple jobs run concurrently, and I *thought* I've seen reducers of multiple jobs run concurrently, but I'm not completely sure. Any idea?

Evert
+
Nitin Pawar 2012-08-14, 10:48
+
Evert Lammerts 2012-08-14, 11:50
+
Nitin Pawar 2012-08-14, 11:56
+
Evert Lammerts 2012-08-14, 12:26
+
Harsh J 2012-08-14, 13:12
+
Evert Lammerts 2012-08-14, 20:45