Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Is the mapper output type must the same as reducer if combiner is used ?


+
Jeff Zhang 2009-11-22, 08:27
+
Y G 2009-11-22, 09:16
Copy link to this message
-
Re: Is the mapper output type must the same as reducer if combiner is used ?
I have never tried it, but the following must also be possible.

map: (k1,v1)  -> list(k2,v2)
combine: (k2 ,list(v2)) -> list(k3,v3)
reduce: (k3 ,list(v3)) -> list(k4,v4)

Karthik Kambatla
On Sun, Nov 22, 2009 at 4:16 AM, Y G <[EMAIL PROTECTED]> wrote:

> if your combiner is the same as reducer,the output type of mapper must the
> same as
> the input type of reducer.
>
> map: (k1,v1)  -> list(k2,v2)
> combine: (k2 ,list(v2)) -> list(k2,v2)
> reduce: (k2 ,list(v2)) -> list(k3,v3)
> -----
> 天天开心
> 身体健康
> Sent from Guangzhou, Guangdong, China
> Stephen Leacock<
> http://www.brainyquote.com/quotes/authors/s/stephen_leacock.html>
> - "I detest life-insurance agents: they always argue that I shall some
> day
> die, which is not so."
>
> 2009/11/22 Jeff Zhang <[EMAIL PROTECTED]>
>
> > Hi all,
> >
> > As I know, Combiner is used in the mapper task, and most of the time,
> > combiner is the same as reducer.
> >
> > So if combiner is used, the output type of mapper task must the same as
> > reducer task, is it right ?
> >
> >
> > Thank you
> >
> > Jeff Zhang
> >
>
+
Jeff Zhang 2009-11-22, 15:41
+
Jason Venner 2009-11-22, 18:28
+
Owen OMalley 2009-11-22, 19:05