Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> How to Free-up a Map Slot without Killing the Entire Job?


Copy link to this message
-
Re: How to Free-up a Map Slot without Killing the Entire Job?
The fair scheduler does preemption as well.

-Sandy
On Sat, Jun 29, 2013 at 12:05 PM, Steve Loughran <[EMAIL PROTECTED]>wrote:

> there's a scheduler in contrib/ that does pre-emption. look here
>
>
> https://github.com/apache/hadoop-common/tree/branch-0.22/mapreduce/src/contrib/dynamic-scheduler
>
> On 28 June 2013 22:13, Sreejith Ramakrishnan <[EMAIL PROTECTED]
> >wrote:
>
> > I'm trying to implement a scheduler (EDF). The scheduler should be able
> to
> > kill or free-up a running map slot so that it can be assigned as a map
> slot
> > to another job.
> >
> > I did some looking around and found a kill() method in
> > org.apache.hadoop.mapred.JobInProgress. But, this kills the entire job. I
> > want the job to still be working after a map slot has been removed from
> it.
> >
> > Can you guys tell me the right method/class to use?
> >
> > Thanks
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB