Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: Hadoop schedulers!


Copy link to this message
-
Re: Hadoop schedulers!
Sandy Ryza 2013-05-13, 18:55
Hi Rahul,

You're right that the schedulers have evolved to support many of the same
features.  To your second question, I haven't looked in detail at the FIFO
or capacity schedulers, but for the FIFO mode in the fair scheduler, jobs
next in the queue will get slots if the first one isn't using them all.  As
Harsh says, the fair scheduler queues can also work in the way that
capacity scheduler queues do.

-Sandy
On Mon, May 13, 2013 at 11:32 AM, Rahul Bhattacharjee <
[EMAIL PROTECTED]> wrote:

> Any pointer to my question.
>
> There is another question , kind-of dumb , but just wanted to clarify.
>
> Say in a FIFO scheduler or a capacity scheduler , if there are slots
> available and the first job doesn't need all of the available slots , then
> the job next in the queue is scheduled for execution or that still waits
> for the first job to finish?
>
> Thanks,
> Rahul
>
>
> On Sat, May 11, 2013 at 8:31 PM, Rahul Bhattacharjee <
> [EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> I was going through the job schedulers of Hadoop and could not see any
>> major operational difference between the capacity scheduler and the fair
>> share scheduler apart from the fact that fair share scheduler supports
>> preemption and capacity scheduler doesn't.
>>
>> Another thing is the former creates logical pools based on certain
>> attribute like username , user group etc and the later has a notion of job
>> queues. Can someone point me to any other major differences between these
>> two types of schedulers.
>>
>> Another question in this regard is the capacity scheduler uses a FIFO
>> queue.So its still possible that a high priority long running job using all
>> the capacity allocated to the queue to block all the other jobs after it in
>> the queue.I think this is the expected behavior , but wanted to confirm.
>>
>> Thanks,
>> Rahul
>>
>>
>>
>