-Re: Scheduler functionality in MRv2
Praveen Sripati 2011-11-27, 01:03
Thanks for the response.
As of now there as less articles/blogs on MRv2, so anxiously waiting for
On Sun, Nov 27, 2011 at 3:55 AM, Arun Murthy <[EMAIL PROTECTED]> wrote:
> There are many ways to prevent what you described...
> I'm in the process of adding more docs, for now pls take a look at the
> following older blog post for more details:
> Sent from my iPhone
> On Nov 25, 2011, at 11:55 PM, Praveen Sripati <[EMAIL PROTECTED]>
> > Hi,
> > Let's consider the following scenario
> > -> The MR Job has an InputSplit on host h1 and h2
> > -> AM makes a request to the Scheduler for a container on h1 and h2
> > -> The scheduler responds with containers c1 and c2 on h1 and h2
> > -> But the AM uses c1 and releases c2 after 15 minutes
> > In this scenario c2 cannot be allocated to any of the AM by the scheduler
> > for that span of 15 minutes and the c2 remains idle. And the released
> > container is also not accounted into the Queues quota. How can such
> > scenario be avoided?
> > Also, it would be nice to specify a more complex requests to the
> > like a request for (h1 or h2), (r1 or r2). Add some dependencies between
> > the requests.
> > Thanks,
> > Praveen