Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Hadoop schedulers!


Copy link to this message
-
Hadoop schedulers!
Hi,

I was going through the job schedulers of Hadoop and could not see any
major operational difference between the capacity scheduler and the fair
share scheduler apart from the fact that fair share scheduler supports
preemption and capacity scheduler doesn't.

Another thing is the former creates logical pools based on certain
attribute like username , user group etc and the later has a notion of job
queues. Can someone point me to any other major differences between these
two types of schedulers.

Another question in this regard is the capacity scheduler uses a FIFO
queue.So its still possible that a high priority long running job using all
the capacity allocated to the queue to block all the other jobs after it in
the queue.I think this is the expected behavior , but wanted to confirm.

Thanks,
Rahul
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB