-Re: Can I have multiple reducers?
Aaron Kimball 2009-10-23, 02:54
If you need another shuffle after your first reduce pass, then you need a
second MapReduce job to run after the first one. Just use an IdentityMapper.
This is a reasonably common situation.
On Thu, Oct 22, 2009 at 4:17 PM, Forhadoop <[EMAIL PROTECTED]> wrote:
> In my application I need to reduce the original reducer output keys
> I was reading about Chainreducer and Chainmappers but looks like it is for
> one or more mapper -> reducer -> 0 or more mappers
> I need something like:
> one or more mapper -> reducer -> reducer
> Please help me figure out the best way to achieve it. Currently, the only
> options seems like I write another map reduce application and run it
> separately after the first map-reduce application. In this second
> application, the mapper will be dummy and won't do anything. The reducer
> will further club the first run outputs.
> Any other comments such as this is not a good programming practice are
> welcome, so that I know I am in the wrong direction..
> View this message in context:
> Sent from the Hadoop core-user mailing list archive at Nabble.com.