Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Can I have multiple reducers?

Copy link to this message
Re: Can I have multiple reducers?
If you need another shuffle after your first reduce pass, then you need a
second MapReduce job to run after the first one. Just use an IdentityMapper.

This is a reasonably common situation.
- Aaron

On Thu, Oct 22, 2009 at 4:17 PM, Forhadoop <[EMAIL PROTECTED]> wrote:

> Hello,
> In my application I need to reduce the original reducer output keys
> further.
> I was reading about Chainreducer and Chainmappers but looks like it is for
> :
> one or more mapper -> reducer -> 0 or more mappers
> I need something like:
> one or more mapper -> reducer -> reducer
> Please help me figure out the best way to achieve it. Currently, the only
> options seems like I write another map reduce application and run it
> separately after the first map-reduce application. In this second
> application, the mapper will be dummy and won't do anything. The reducer
> will further club the first run outputs.
> Any other comments such as this is not a good programming practice are
> welcome, so that I know I am in the wrong direction..
> --
> View this message in context:
> http://www.nabble.com/Can-I-have-multiple-reducers--tp26018722p26018722.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.