Evert Lammerts 2012-08-14, 10:39
I have a cluster running Hadoop 0.20.205 with Kerberos enabled, exposing 528 map slots and 528 reduce slots. Currently somebody is running a NORMAL priority job with 7 mappers and 400 reducers. The mappers have finished and the system is processing the reducers. Another user is running a NORMAL priority job with 1 mapper and 26 reducers. The mapper has finished, but the reducers won't come out of "pending" state. There are no other jobs running right now. We've not yet installed a different scheduler, so right now the system is using the default scheduler. How can this behavior be explained? I see mappers of multiple jobs run concurrently, and I *thought* I've seen reducers of multiple jobs run concurrently, but I'm not completely sure. Any idea?
Nitin Pawar 2012-08-14, 10:48
Evert Lammerts 2012-08-14, 11:50
Nitin Pawar 2012-08-14, 11:56
Evert Lammerts 2012-08-14, 12:26
Harsh J 2012-08-14, 13:12
Evert Lammerts 2012-08-14, 20:45