Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Limit on total jobs running using fair scheduler

Copy link to this message
Re: Limit on total jobs running using fair scheduler
Not sure about the fair scheduler but in capacity scheduler you can achieve
this by controlling number of jobs/applications per queue.

Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>
On Tue, Nov 19, 2013 at 3:26 AM, Ivan Tretyakov <[EMAIL PROTECTED]
> wrote:

> Hello!
> We are using CDH 4.1.1 (Version: 2.0.0-mr1-cdh4.1.1) and fair-scheduler.
> We need to limit total number of jobs which can run at the same time on
> cluster.
> I can see maxRunningJobs options but it sets limit for pool or user.
> We wouldn't like to limit each pool or user we just need to set limit on
> total number of jobs running.
> Is it possible to do it using fair scheduler?
> Can capacity scheduler help here?
> Maybe there are other options to achieve the goal?
> Thanks in advance!
> --
> Best Regards
> Ivan Tretyakov

NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.