Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Is the mapper output type must the same as reducer if combiner is used ?


+
Jeff Zhang 2009-11-22, 08:27
+
Y G 2009-11-22, 09:16
Copy link to this message
-
Re: Is the mapper output type must the same as reducer if combiner is used ?
I have never tried it, but the following must also be possible.

map: (k1,v1)  -> list(k2,v2)
combine: (k2 ,list(v2)) -> list(k3,v3)
reduce: (k3 ,list(v3)) -> list(k4,v4)

Karthik Kambatla
On Sun, Nov 22, 2009 at 4:16 AM, Y G <[EMAIL PROTECTED]> wrote:

> if your combiner is the same as reducer,the output type of mapper must the
> same as
> the input type of reducer.
>
> map: (k1,v1)  -> list(k2,v2)
> combine: (k2 ,list(v2)) -> list(k2,v2)
> reduce: (k2 ,list(v2)) -> list(k3,v3)
> -----
> 天天开心
> 身体健康
> Sent from Guangzhou, Guangdong, China
> Stephen Leacock<
> http://www.brainyquote.com/quotes/authors/s/stephen_leacock.html>
> - "I detest life-insurance agents: they always argue that I shall some
> day
> die, which is not so."
>
> 2009/11/22 Jeff Zhang <[EMAIL PROTECTED]>
>
> > Hi all,
> >
> > As I know, Combiner is used in the mapper task, and most of the time,
> > combiner is the same as reducer.
> >
> > So if combiner is used, the output type of mapper task must the same as
> > reducer task, is it right ?
> >
> >
> > Thank you
> >
> > Jeff Zhang
> >
>
+
Jeff Zhang 2009-11-22, 15:41
+
Jason Venner 2009-11-22, 18:28
+
Owen OMalley 2009-11-22, 19:05
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB