Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Multiple Aggregate functions in map reduce program


+
iwannaplay games 2012-10-05, 07:02
Copy link to this message
-
Re: Multiple Aggregate functions in map reduce program
Bejoy KS 2012-10-05, 07:18
Hi

It is definitely possible. In your map make the dept name as the output key and salary as the value.

In the reducer for every key you can initialize a counter and a sum. Add on to the sum for all values and increment the counter by 1 for each value. Output the dept key and the new aggregated sum and count for each key.
Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: iwannaplay games <[EMAIL PROTECTED]>
Date: Fri, 5 Oct 2012 12:32:28
To: user<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>; hdfs-user<[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Multiple Aggregate functions in map reduce program

Hi All,

I have to get the count and sum of data
for eg if my  table is
*employeename   salary   department*
A                       1000     testing
B                       2000     testing
C                       3000     development
D                       4000     testing
E                       1000     development
F                       5000     management

I want result like

Department       TotalSalary      count(employees)

testing                    7000                 3
development           4000                  2
management           5000                  1
Please let me know whether it is possible to write a java map reduce for
this.I tried this on hive.It takes time for big data.I heard map reduce
java code will b faster.IS it true???Or i should go for pig programming??

Please guide..
Regards
Prabhjot

+
Bertrand Dechoux 2012-10-05, 08:13