Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> How to Influence Reduce Task Location.


Copy link to this message
-
Re: How to Influence Reduce Task Location.

On Dec 19, 2010, at 10:23 AM, Jane Chen wrote:

> Suppose that the output is written to a database, that only runs on certain nodes.  It will be desirable to schedule the reducer tasks to run on the nodes local or close to the database nodes.
a) That's a side-effect--pretty much "against the rules".  Very little support is provided for such things.

b) At a minimum, you'll need to write your own scheduler.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB