Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> Scheduler functionality in MRv2


Copy link to this message
-
Re: Scheduler functionality in MRv2
Arun,

Thanks for the response.

As of now there as less articles/blogs on MRv2, so anxiously waiting for
the documents.

Praveen

On Sun, Nov 27, 2011 at 3:55 AM, Arun Murthy <[EMAIL PROTECTED]> wrote:

> Praveen,
>
> There are many ways to prevent what you described...
>
> I'm in the process of adding more docs, for now pls take a look at the
> following older blog post for more details:
>
>
> http://developer.yahoo.com/blogs/hadoop/posts/2011/03/mapreduce-nextgen-scheduler/
>
> Arun
>
> Sent from my iPhone
>
> On Nov 25, 2011, at 11:55 PM, Praveen Sripati <[EMAIL PROTECTED]>
> wrote:
>
> > Hi,
> >
> > Let's consider the following scenario
> >
> > -> The MR Job has an InputSplit on host h1 and h2
> > -> AM makes a request to the Scheduler for a container on h1 and h2
> > -> The scheduler responds with containers c1 and c2 on h1 and h2
> > -> But the AM uses c1 and releases c2 after 15 minutes
> >
> > In this scenario c2 cannot be allocated to any of the AM by the scheduler
> > for that span of 15 minutes and the c2 remains idle. And the released
> > container is also not accounted into the Queues quota. How can such
> > scenario be avoided?
> >
> > Also, it would be nice to specify a more complex requests to the
> scheduler,
> > like a request for (h1 or h2), (r1 or r2). Add some dependencies between
> > the requests.
> >
> > Thanks,
> > Praveen
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB