Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> How to Influence Reduce Task Location.


Copy link to this message
-
Re: How to Influence Reduce Task Location.
It doesn't.  But you really can't do what you're asking.  Nor, I think,
would you really want to.  The whole idea behind Hadoop is that it's a
distributed system whereby nodes are pretty much interchangeable.
There's nothing to be gained by trying to pin a particular reduce task
to a particular node - and much to be lost:  e.g., redundancy,
speculative execution, etc.

DR

On 12/18/2010 01:24 PM, Jane Chen wrote:
> But how does this help me request which host to schedule the reduce task to?
>
> Thanks,
> Jane
>
> --- On Sat, 12/18/10, Hari Sreekumar<[EMAIL PROTECTED]>  wrote:
>
>
> From: Hari Sreekumar<[EMAIL PROTECTED]>
> Subject: Re: How to Influence Reduce Task Location.
> To: [EMAIL PROTECTED]
> Date: Saturday, December 18, 2010, 10:16 AM
>
>
> Hi Jane,
>
>
>           The partitioner class can be used to achieve this. (http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/Partitioner.html).
>
>
> Thanks,
> Hari
>
>
> On Sat, Dec 18, 2010 at 11:13 PM, Jane Chen<[EMAIL PROTECTED]>  wrote:
>
> Hi All,
>
> Is there anyway to influence where a reduce task is run?  We have a case where we'd like to choose the host to run the reduce task based on the task's input key.
>
> Any suggestion is greatly appreciated.
>
> Thanks,
> Jane
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB