Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Custom counters in combiner


Copy link to this message
-
Custom counters in combiner
Amit Sela 2014-01-13, 18:17
Hi all,

I'm running a mapreduce job that has custom counters incremented in the
combiner's reduce function.
Looking at the mapreduce web UI I see that, like all counters, its has
three columns: Map, Reduce and Total.
>From what I know, the combiner is executed on the map output, hence runs in
Mapper (?).
The strange thing is that I see the counter incrementing in both the Map
and Reduce columns...

If the combiner runs on the Mapper output, locally on the Mapper, why do I
see reduce counts as well ?

Thanks,
Amit.