Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> About the combiner execution


Copy link to this message
-
Re: About the combiner execution
Yes this is true. Combiner may never run if intermediate values don't need to shuffle out to disk before the final output is done. Also, combiner cant be substituted as a reducer.

Sent from my iPad

On Jul 10, 2011, at 4:42, Florin P <[EMAIL PROTECTED]> wrote:

> Hello!
>  I've read on http://www.fromdev.com/2010/12/interview-questions-hadoop-mapreduce.html (cite):
> "The execution of combiner is not guaranteed, Hadoop may or may not execute a combiner. Also, if required it may execute it more then 1 times. Therefore your MapReduce jobs should not depend on the combiners execution. "
>
> Is it true? Also is it possible to use the Combiner without the Reducer? The framework will take into the consideration the Combiner in this case?
>
>  Thank you,
>
> Florin
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB