Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Relation between Mapper and Combiner


Copy link to this message
-
RE: Relation between Mapper and Combiner
Devaraj K 2011-07-01, 11:47
Hi Lucian,

 

            For every map task, combiner will be executed multiple times
before writing the map output. Combine step is not a separate task and it is
part of map task execution. Reducer will copy the output of the map task
which is reduced by the combiner.

 

>For example:
>If I have 2 map tasks ran on the same machine, will I have 1 combine task
on that machine to combine the maps outputs, or 2 combine tasks?

 

In this case, combiner will be executed for each map task independent to
each other. This combiner step will execute multiple times till it gets same
output for 1 or more runs of combiner.

 

You can go through Combiner section here for more info :
http://wiki.apache.org/hadoop/HadoopMapReduce

 

Devaraj K

----------------------------------------------------------------------------
---------------------------------------------------------
This e-mail and its attachments contain confidential information from
HUAWEI, which
is intended only for the person or entity whose address is listed above. Any
use of the
information contained herein in any way (including, but not limited to,
total or partial
disclosure, reproduction, or dissemination) by persons other than the
intended
recipient(s) is prohibited. If you receive this e-mail in error, please
notify the sender by
phone or email immediately and delete it!ss

 

  _____  

From: Lucian Iordache [mailto:[EMAIL PROTECTED]]
Sent: Friday, July 01, 2011 2:25 PM
To: [EMAIL PROTECTED]
Subject: Relation between Mapper and Combiner

 

Hello guys,

Can anybody tell me which is the relation between map task and combine
tasks?
I would like to know if there is a 1:1 relation between them, or is a *:1
(many to one).

For example:
If I have 2 map tasks ran on the same machine, will I have 1 combine task on
that machine to combine the maps outputs, or 2 combine tasks?

Best Regards,
--
Lucian