Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Pending reducers


Copy link to this message
-
Pending reducers
Hi list,

I have a cluster running Hadoop 0.20.205 with Kerberos enabled, exposing 528 map slots and 528 reduce slots. Currently somebody is running a NORMAL priority job with 7 mappers and 400 reducers. The mappers have finished and the system is processing the reducers. Another user is running a NORMAL priority job with 1 mapper and 26 reducers. The mapper has finished, but the reducers won't come out of "pending" state. There are no other jobs running right now. We've not yet installed a different scheduler, so right now the system is using the default scheduler. How can this behavior be explained? I see mappers of multiple jobs run concurrently, and I *thought* I've seen reducers of multiple jobs run concurrently, but I'm not completely sure. Any idea?

Evert
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB