Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # dev >> How to modify the Map-Reduce execution order?


+
Anh Pham 2013-10-16, 08:16
Copy link to this message
-
Re: How to modify the Map-Reduce execution order?
Hi Anh,

How about using ChainMapper? Is the helpful for you?
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapred/lib/ChainMapper.html

Thanks, Tsuyoshi

On Wed, Oct 16, 2013 at 1:16 AM, Anh Pham <[EMAIL PROTECTED]> wrote:
> (Please correct me if I am wrong) So the original chain is:
> InputSplits-->Mapper--> [Sorting/Shuffling, etc]-->Reducer-->...
>
> Now I don't want the input splits to get to the Mappers first, but to go to
> some other new stage instead (we can call it Pre-Mapper for example, this
> class will be created by myself).
>
> So the new order will be: InputSplits -> Pre-Mapper->Mapper ->...
>
> I'm currently reading the source code. However, I still cannot find any
> clue (what classes I should touch). Any suggestion is welcome. Thank you
> very much :)

--
- Tsuyoshi
+
Arun C Murthy 2013-10-18, 21:06