Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - Re: Question regarding job execution


Copy link to this message
-
Re: Question regarding job execution
Harsh J 2013-03-19, 05:54
I am assuming you refer to the YARN's CapacityScheduler.

The CS in YARN does support parallel job (the right term is application, or
'app', not 'job' anymore, when speaking in YARN's context) execution. If
looking at code of CapacityScheduler.java and LeafQueue.java, you can
notice it iterate over all applications for node-update time assignments
(see nodeUpdate(…) call (via NODE_UPDATE event) in former and
assignContainers(…) call in the latter).

On Sun, Mar 3, 2013 at 1:28 PM, Jagmohan Chauhan <[EMAIL PROTECTED]
> wrote:

> Hi
>
> I am using Capacity scheduler on a  cluster of 5 nodes. I submit 3 jobs to
> the system with a single queue for same user. I am observing that they are
> executed on FIFO basis even if the map slots are empty.  According to my
> observation , when the first job for the user ends then its 2nd job starts.
>
> Is it true that when using Capacity Scheduler, for any user, its next job
> will be executed when its prior job is finished?
>
>
>
> --
> Thanks and Regards
> Jagmohan Chauhan
> MSc student,CS
> Univ. of Saskatchewan
> IEEE Graduate Student Member
>
> http://homepage.usask.ca/~jac735/
>
>
--
Harsh J