Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: EOF when Combiner works


Copy link to this message
-
Re: Re: EOF when Combiner works
Hi, Harsh. Sorry, I failed to write to [EMAIL PROTECTED] (http://mail-archives.apache.org/mod_mbox/hadoop-user/201301.mbox/%3CCAOcnVr2A%2BmDQ%2BVY3iovxuP13VTXk2Wog7ui_o6x%3DW0et5igUSg%40mail.gmail.com%3E)
The logic of combiner is the same as reducer's one - summation of some statistical parameters:

        @Override
  protected void reduce(Text hostname, Iterable hostsCounters, Context context) throws IOException, InterruptedException {
    HostCounters totals = new HostCounters();
    totals.calculate(hostsCounters);    context.write(hostname, totals);}

where HostCounters - Writable with public fields of types int, long and MapWritable; and function clculate() is:

public void calculate(Iterable hostsCounters) {    for (HostCounters hostCounters : hostsCounters) {        a += hostCounters.a;        b += hostCounters.b;        //...    }}

Thank you! Sergey.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB