Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # dev - How to modify the Map-Reduce execution order?


Copy link to this message
-
How to modify the Map-Reduce execution order?
Anh Pham 2013-10-16, 08:16
(Please correct me if I am wrong) So the original chain is:
InputSplits-->Mapper--> [Sorting/Shuffling, etc]-->Reducer-->...

Now I don't want the input splits to get to the Mappers first, but to go to
some other new stage instead (we can call it Pre-Mapper for example, this
class will be created by myself).

So the new order will be: InputSplits -> Pre-Mapper->Mapper ->...

I'm currently reading the source code. However, I still cannot find any
clue (what classes I should touch). Any suggestion is welcome. Thank you
very much :)