Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Cumulative value using mapreduce


Copy link to this message
-
Re: Cumulative value using mapreduce
Bertrand is almost right.

The only difference is that the original poster asked about cumulative sum.

This can be done in reducer exactly as Bertrand described except for two
points that make it different from word count:

a) you can't use a combiner

b) the output of the program is as large as the input so it will have
different performance characteristics than aggregation programs like
wordcount.

Bertrand's key recommendation to go read a book is the most important
advice.

On Thu, Oct 4, 2012 at 5:20 PM, Bertrand Dechoux <[EMAIL PROTECTED]> wrote:

> Hi,
>
> It sounds like a
> 1) group information by account
> 2) compute sum per account
>
> If that not the case, you should precise a bit more about your context.
>
> This computing looks like a small variant of wordcount. If you do not know
> how to do it, you should read books about Hadoop MapReduce and/or online
> tutorial. Yahoo's is old but still a nice read to begin with :
> http://developer.yahoo.com/hadoop/tutorial/
>
> Regards,
>
> Bertrand
>
>
> On Thu, Oct 4, 2012 at 3:58 PM, Sarath <
> [EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> I have a file which has some financial transaction data. Each transaction
>> will have amount and a credit/debit indicator.
>> I want to write a mapreduce program which computes cumulative credit &
>> debit amounts at each record
>> and append these values to the record before dumping into the output file.
>>
>> Is this possible? How can I achieve this? Where should i put the logic of
>> computing the cumulative values?
>>
>> Regards,
>> Sarath.
>>
>
>
>
> --
> Bertrand Dechoux
>