Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Do we shoot ourselves by using all task slots?


Copy link to this message
-
Re: Do we shoot ourselves by using all task slots?

On May 28, 2010, at 11:43 AM, Todd Lipcon wrote:

> Hi Allen,
>
> Recent versions of the fair scheduler have configurations for "delay scheduling" - essentially, it will wait for a few seconds when a slot opens up to try to find a local task before assigning a non-local one. This is specifically to avoid the issue you're describing.
>
> Check out Matei's Eurosys 2010 paper here:
>
> http://www.cs.berkeley.edu/~matei/papers/2010/eurosys_delay_scheduling.pdf
>
> I believe this got lumped in with MAPREDUCE-706.
Thanks.  I'll take a look at this and see how applicable this is to Capacity.