Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # dev - How to Free-up a Map Slot without Killing the Entire Job?


Copy link to this message
-
Re: How to Free-up a Map Slot without Killing the Entire Job?
Sandy Ryza 2013-06-29, 20:46
The fair scheduler does preemption as well.

-Sandy
On Sat, Jun 29, 2013 at 12:05 PM, Steve Loughran <[EMAIL PROTECTED]>wrote:

> there's a scheduler in contrib/ that does pre-emption. look here
>
>
> https://github.com/apache/hadoop-common/tree/branch-0.22/mapreduce/src/contrib/dynamic-scheduler
>
> On 28 June 2013 22:13, Sreejith Ramakrishnan <[EMAIL PROTECTED]
> >wrote:
>
> > I'm trying to implement a scheduler (EDF). The scheduler should be able
> to
> > kill or free-up a running map slot so that it can be assigned as a map
> slot
> > to another job.
> >
> > I did some looking around and found a kill() method in
> > org.apache.hadoop.mapred.JobInProgress. But, this kills the entire job. I
> > want the job to still be working after a map slot has been removed from
> it.
> >
> > Can you guys tell me the right method/class to use?
> >
> > Thanks
> >
>