Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Multiple Aggregate functions in map reduce program


Copy link to this message
-
Re: Multiple Aggregate functions in map reduce program
Bertrand Dechoux 2012-10-05, 08:13
>
> .It takes time for big data.I heard map reduce
> java code will b faster.IS it true???Or i should go for pig programming??
>

I guess one important question is what do you mean by 'it takes time'. And
what goal do you want to reach.
It may be that your current implementation is naive and can be improved
(which begs for the question : what is your current implementation?).
Or it may be simply that given your data volume and cluster capacity, you
can not reduce greatly the time.

Anyway, please do not post across multiple mailing lists at the same time,
especially when not related to your problem. I have a hard time figuring
why [EMAIL PROTECTED] is targeted when there is no mention of HBase in
your message. Anyone answering this message, please answer only to
[EMAIL PROTECTED].

Regards

Bertrand
On Fri, Oct 5, 2012 at 9:18 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:

> **
> Hi
>
> It is definitely possible. In your map make the dept name as the output
> key and salary as the value.
>
> In the reducer for every key you can initialize a counter and a sum. Add
> on to the sum for all values and increment the counter by 1 for each value.
> Output the dept key and the new aggregated sum and count for each key.
>
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * iwannaplay games <[EMAIL PROTECTED]>
> *Date: *Fri, 5 Oct 2012 12:32:28 +0530
> *To: *user<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>; hdfs-user<
> [EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Multiple Aggregate functions in map reduce program
>
> Hi All,
>
> I have to get the count and sum of data
> for eg if my  table is
>
>
> *employeename   salary   department*
>
> A                       1000     testing
> B                       2000     testing
> C                       3000     development
> D                       4000     testing
> E                       1000     development
> F                       5000     management
>
>
>
> I want result like
>
> Department       TotalSalary      count(employees)
>
> testing                    7000                 3
> development           4000                  2
> management           5000                  1
>
>
> Please let me know whether it is possible to write a java map reduce for
> this.I tried this on hive.It takes time for big data.I heard map reduce
> java code will b faster.IS it true???Or i should go for pig programming??
>
> Please guide..
>
>
> Regards
> Prabhjot
>

--
Bertrand Dechoux