Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> How to Influence Reduce Task Location.


Copy link to this message
-
Re: How to Influence Reduce Task Location.
It doesn't.  But you really can't do what you're asking.  Nor, I think,
would you really want to.  The whole idea behind Hadoop is that it's a
distributed system whereby nodes are pretty much interchangeable.
There's nothing to be gained by trying to pin a particular reduce task
to a particular node - and much to be lost:  e.g., redundancy,
speculative execution, etc.

DR

On 12/18/2010 01:24 PM, Jane Chen wrote:
> But how does this help me request which host to schedule the reduce task to?
>
> Thanks,
> Jane
>
> --- On Sat, 12/18/10, Hari Sreekumar<[EMAIL PROTECTED]>  wrote:
>
>
> From: Hari Sreekumar<[EMAIL PROTECTED]>
> Subject: Re: How to Influence Reduce Task Location.
> To: [EMAIL PROTECTED]
> Date: Saturday, December 18, 2010, 10:16 AM
>
>
> Hi Jane,
>
>
>           The partitioner class can be used to achieve this. (http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/Partitioner.html).
>
>
> Thanks,
> Hari
>
>
> On Sat, Dec 18, 2010 at 11:13 PM, Jane Chen<[EMAIL PROTECTED]>  wrote:
>
> Hi All,
>
> Is there anyway to influence where a reduce task is run?  We have a case where we'd like to choose the host to run the reduce task based on the task's input key.
>
> Any suggestion is greatly appreciated.
>
> Thanks,
> Jane