-
Re: Re: EOF when Combiner workss2323 2013-01-25, 09:12
Hi, Harsh. Sorry, I failed to write to [EMAIL PROTECTED] (http://mail-archives.apache.org/mod_mbox/hadoop-user/201301.mbox/%3CCAOcnVr2A%2BmDQ%2BVY3iovxuP13VTXk2Wog7ui_o6x%3DW0et5igUSg%40mail.gmail.com%3E)
The logic of combiner is the same as reducer's one - summation of some statistical parameters:
@Override
protected void reduce(Text hostname, Iterable hostsCounters, Context context) throws IOException, InterruptedException {
HostCounters totals = new HostCounters();
totals.calculate(hostsCounters); context.write(hostname, totals);}
where HostCounters - Writable with public fields of types int, long and MapWritable; and function clculate() is:
public void calculate(Iterable hostsCounters) { for (HostCounters hostCounters : hostsCounters) { a += hostCounters.a; b += hostCounters.b; //... }}
Thank you! Sergey.
The logic of combiner is the same as reducer's one - summation of some statistical parameters:
@Override
protected void reduce(Text hostname, Iterable hostsCounters, Context context) throws IOException, InterruptedException {
HostCounters totals = new HostCounters();
totals.calculate(hostsCounters); context.write(hostname, totals);}
where HostCounters - Writable with public fields of types int, long and MapWritable; and function clculate() is:
public void calculate(Iterable hostsCounters) { for (HostCounters hostCounters : hostsCounters) { a += hostCounters.a; b += hostCounters.b; //... }}
Thank you! Sergey.
