Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> data locality


Copy link to this message
-
Re: data locality
On 26/10/11 05:22, Eugene Kirpichov wrote:
> But I guess it isn't always possible to achieve optimal scheduling, right?
> What's done then; any account for network topology perhaps?

I'd recommend this paper if you are curious, it explains the Fair Scheduler

http://www.cs.berkeley.edu/~matei/papers/2010/eurosys_delay_scheduling.pdf

You can plug in different schedulers with different policies if you want
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB