Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> How to modify the Map-Reduce execution order?


Copy link to this message
-
Re: How to modify the Map-Reduce execution order?
Hi Anh,

How about using ChainMapper? Is the helpful for you?
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapred/lib/ChainMapper.html

Thanks, Tsuyoshi

On Wed, Oct 16, 2013 at 1:16 AM, Anh Pham <[EMAIL PROTECTED]> wrote:
> (Please correct me if I am wrong) So the original chain is:
> InputSplits-->Mapper--> [Sorting/Shuffling, etc]-->Reducer-->...
>
> Now I don't want the input splits to get to the Mappers first, but to go to
> some other new stage instead (we can call it Pre-Mapper for example, this
> class will be created by myself).
>
> So the new order will be: InputSplits -> Pre-Mapper->Mapper ->...
>
> I'm currently reading the source code. However, I still cannot find any
> clue (what classes I should touch). Any suggestion is welcome. Thank you
> very much :)

--
- Tsuyoshi
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB