iwannaplay games 2012-10-05, 07:02
-Re: Multiple Aggregate functions in map reduce program
Bejoy KS 2012-10-05, 07:18
It is definitely possible. In your map make the dept name as the output key and salary as the value.
In the reducer for every key you can initialize a counter and a sum. Add on to the sum for all values and increment the counter by 1 for each value. Output the dept key and the new aggregated sum and count for each key.
Sent from handheld, please excuse typos.
From: iwannaplay games <[EMAIL PROTECTED]>
Date: Fri, 5 Oct 2012 12:32:28
To: user<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>; hdfs-user<[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Multiple Aggregate functions in map reduce program
I have to get the count and sum of data
for eg if my table is
*employeename salary department*
A 1000 testing
B 2000 testing
C 3000 development
D 4000 testing
E 1000 development
F 5000 management
I want result like
Department TotalSalary count(employees)
testing 7000 3
development 4000 2
management 5000 1
Please let me know whether it is possible to write a java map reduce for
this.I tried this on hive.It takes time for big data.I heard map reduce
java code will b faster.IS it true???Or i should go for pig programming??
Bertrand Dechoux 2012-10-05, 08:13