Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> What is the cost of using a counter

Copy link to this message
What is the cost of using a counter
I have several counters that I maintain to allow me to keep statistics on
critical operations. I have my code incrementing the counters in an inner
partly to make sure my job is not killed for not making progress. It would
be very easy to keep an internal counter and increment the Hadoop value
less frequently.  Assuming
I am currently incrementing a counter several million times in a reduce
task- is this costing me performance and would I be better off incrementing
less frequently

Steven M. Lewis PhD
4221 105th Ave NE
Kirkland, WA 98033
206-384-1340 (cell)
Skype lordjoe_com