Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Re: EOF when Combiner works

Harsh J 2013-01-23, 14:43
Copy link to this message
Re: Re: EOF when Combiner works
s2323 2013-01-25, 09:12
Hi, Harsh. Sorry, I failed to write to [EMAIL PROTECTED] (http://mail-archives.apache.org/mod_mbox/hadoop-user/201301.mbox/%3CCAOcnVr2A%2BmDQ%2BVY3iovxuP13VTXk2Wog7ui_o6x%3DW0et5igUSg%40mail.gmail.com%3E)
The logic of combiner is the same as reducer's one - summation of some statistical parameters:

  protected void reduce(Text hostname, Iterable hostsCounters, Context context) throws IOException, InterruptedException {
    HostCounters totals = new HostCounters();
    totals.calculate(hostsCounters);    context.write(hostname, totals);}

where HostCounters - Writable with public fields of types int, long and MapWritable; and function clculate() is:

public void calculate(Iterable hostsCounters) {    for (HostCounters hostCounters : hostsCounters) {        a += hostCounters.a;        b += hostCounters.b;        //...    }}

Thank you! Sergey.