Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Do we shoot ourselves by using all task slots?

Copy link to this message
Re: Do we shoot ourselves by using all task slots?
Allen Wittenauer 2010-05-28, 18:54

On May 28, 2010, at 11:43 AM, Todd Lipcon wrote:

> Hi Allen,
> Recent versions of the fair scheduler have configurations for "delay scheduling" - essentially, it will wait for a few seconds when a slot opens up to try to find a local task before assigning a non-local one. This is specifically to avoid the issue you're describing.
> Check out Matei's Eurosys 2010 paper here:
> http://www.cs.berkeley.edu/~matei/papers/2010/eurosys_delay_scheduling.pdf
> I believe this got lumped in with MAPREDUCE-706.
Thanks.  I'll take a look at this and see how applicable this is to Capacity.