Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Accessing global Counters


Copy link to this message
-
Re: Accessing global Counters
Robert Evans 2012-04-20, 18:24
There was a discussion about this several months ago

http://mail-archives.apache.org/mod_mbox/hadoop-mapreduce-user/201112.mbox/%[EMAIL PROTECTED]%3E

The conclusion is that if you want to read them from the reducer you are going to have to do something special until someone finds time to implement it as part of.

https://issues.apache.org/jira/browse/MAPREDUCE-3520

--Bobby Evans
On 4/20/12 11:36 AM, "Amith D K" <[EMAIL PROTECTED]> wrote:

Yes U can use user defined counter as Jagat suggeted.

Counter can be enum as Jagat described or any string which are called dynamic counters.

It is easier to use Enum counter than dynamic counters, finally it depends on your use case :)

Amith
________________________________________
From: Jagat [[EMAIL PROTECTED]]
Sent: Saturday, April 21, 2012 12:25 AM
To: [EMAIL PROTECTED]
Subject: Re: Accessing global Counters

Hi

You can create your own counters like

enum CountFruits {
Apple,
Mango,
Banana
}
And in your mapper class when you see condition to increment , you can use
Reporter incrCounter method to do the same.

http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/Reporter.html#incrCounter(java.lang.Enum,%20long)

e.g
// I saw Apple increment it by one
reporter.incrCounter(CountFruits.Apple,1);

Now you can access them using job.getCounters

http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html#getCounters()

Hope this helps

Regards,

Jagat Singh
On Fri, Apr 20, 2012 at 9:43 PM, Gayatri Rao <[EMAIL PROTECTED]> wrote:

> Hi All,
>
> Is there a way for me to set global counters in Mapper and access them from
> reducer?
> Could you suggest how I can acheve this?
>
> Thanks
> Gayatri
>