-Re: Pending reducers
Nitin Pawar 2012-08-14, 11:56
whats the memory/cpu stats on the machines ? are they exhausted
On Tue, Aug 14, 2012 at 5:20 PM, Evert Lammerts <[EMAIL PROTECTED]> wrote:
>> reducers of multiple jobs do run con-currently as long as they have the
>> resources available.
> Yep, and that's what's not happening in my situation. 528 reduce slots, 400 taken by one job, 26 of another job remain in pending state. What could explain this behavior?
>> If you want to limit someone overtaking the cluster, then you can
>> create different job queues and assign quota to each queue. You also
>> have the flexibility of allocating max quota per user in a queue as
>> On Tue, Aug 14, 2012 at 4:09 PM, Evert Lammerts
>> <[EMAIL PROTECTED]> wrote:
>> > Hi list,
>> > I have a cluster running Hadoop 0.20.205 with Kerberos enabled,
>> exposing 528 map slots and 528 reduce slots. Currently somebody is
>> running a NORMAL priority job with 7 mappers and 400 reducers. The
>> mappers have finished and the system is processing the reducers.
>> Another user is running a NORMAL priority job with 1 mapper and 26
>> reducers. The mapper has finished, but the reducers won't come out of
>> "pending" state. There are no other jobs running right now. We've not
>> yet installed a different scheduler, so right now the system is using
>> the default scheduler. How can this behavior be explained? I see
>> mappers of multiple jobs run concurrently, and I *thought* I've seen
>> reducers of multiple jobs run concurrently, but I'm not completely
>> sure. Any idea?
>> > Evert
>> Nitin Pawar