Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> How to Influence Reduce Task Location.

Copy link to this message
Re: How to Influence Reduce Task Location.
On 12/18/2010 12:43 PM, Jane Chen wrote:
> Hi All,
> Is there anyway to influence where a reduce task is run?  We have a case where we'd like to choose the host to run the reduce task based on the task's input key.
> Any suggestion is greatly appreciated.
> Thanks,
> Jane

We don't do exactly that, but we do something similar.

We don't make specific reducers run on specific hosts.  But we do
specifically shard our data - e.g., into 1024 shards - and we then run
1024 reducers, each of which runs on its correspondingly numbered shard
of the data.