Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> What is the cost of using a counter


Copy link to this message
-
What is the cost of using a counter
I have several counters that I maintain to allow me to keep statistics on
critical operations. I have my code incrementing the counters in an inner
loop
partly to make sure my job is not killed for not making progress. It would
be very easy to keep an internal counter and increment the Hadoop value
less frequently.  Assuming
I am currently incrementing a counter several million times in a reduce
task- is this costing me performance and would I be better off incrementing
less frequently

--
Steven M. Lewis PhD
4221 105th Ave NE
Kirkland, WA 98033
206-384-1340 (cell)
Skype lordjoe_com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB