Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - About the combiner execution


+
Florin P 2011-07-10, 11:42
Copy link to this message
-
Re: About the combiner execution
Jeff Bean 2011-07-10, 15:29
Yes this is true. Combiner may never run if intermediate values don't need to shuffle out to disk before the final output is done. Also, combiner cant be substituted as a reducer.

Sent from my iPad

On Jul 10, 2011, at 4:42, Florin P <[EMAIL PROTECTED]> wrote:

> Hello!
>  I've read on http://www.fromdev.com/2010/12/interview-questions-hadoop-mapreduce.html (cite):
> "The execution of combiner is not guaranteed, Hadoop may or may not execute a combiner. Also, if required it may execute it more then 1 times. Therefore your MapReduce jobs should not depend on the combiners execution. "
>
> Is it true? Also is it possible to use the Combiner without the Reducer? The framework will take into the consideration the Combiner in this case?
>
>  Thank you,
>
> Florin