Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> How to Influence Reduce Task Location.


Copy link to this message
-
Re: How to Influence Reduce Task Location.
And, as a follow-up, yes, we use the partitioner class to achieve this.
  Our partioner runs a hashing algorithm which ensures that a given user
key will always map to a specific shard #.

DR

On 12/18/2010 01:16 PM, Hari Sreekumar wrote:
> Hi Jane,
>
>           The partitioner class can be used to achieve this. (
> http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/Partitioner.html
> ).
>
> Thanks,
> Hari
>
> On Sat, Dec 18, 2010 at 11:13 PM, Jane Chen<[EMAIL PROTECTED]>wrote:
>
>> Hi All,
>>
>> Is there anyway to influence where a reduce task is run?  We have a case
>> where we'd like to choose the host to run the reduce task based on the
>> task's input key.
>>
>> Any suggestion is greatly appreciated.
>>
>> Thanks,
>> Jane
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB