Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: change yarn application priority


Copy link to this message
-
Re: change yarn application priority
WRT capacity scheduler, its not so much changing the priority of a job, but allowing for pre-emption.  Note that I guess you could raise the one job's priority, and then the other job's priority so that when a task finishes the other job gets the next slot. However, you're still stuck waiting and that can take time.)

Here's the rub though... pre-empting a task could have a negative impact on the long running job.
If you have large jobs and small jobs, you could create multiple queues and leave some capacity to the smaller job queue.
So you need to specify the queue up front.  (Its been a while, but can you move jobs across queues? )

I would suggest that you look at fair scheduler, but also consider multiple queues under capacity scheduler. You may have admin jobs you want to run in the background while other tasks are running.

 
On Jun 2, 2014, at 8:10 PM, Henry Hung <[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB