Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> hadoop queue

Copy link to this message
hadoop queue
Hi guys,

It is mentioned below for Hadoop queue (
http://blog.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/), for the
comments "FIFO scheduling with priorities is used", does it mean for
resources (mapper/reducer slots) allocation inside each queue, two factors
((1) job submit order, i.e. FIFO, and (2) individual job priorities are
also considered)? So it means even if inside one queue, the resource
allocation is not fair between jobs?

*However, within each queue, FIFO scheduling with priorities is used,
except for one aspect – you can place a limit on percent of running tasks
per user, so that users share a cluster equally. In other words, the
capacity scheduler tries to simulate a separate FIFO/priority cluster for
each user and each organization, rather than performing fair sharing
between all jobs. *