Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Can I number output results with a Counter?


Copy link to this message
-
Can I number output results with a Counter?
Mark Kerzner 2011-05-20, 16:55
Hi, can I use a Counter to give each record in all reducers a consecutive
number? Currently I am using a single Reducer, but it is an anti-pattern.
But I need to assign consecutive numbers to all output records in all
reducers, and it does not matter how, as long as each gets its own number.

If it IS possible, then how are multiple processes accessing those counters
without creating race conditions.

Thank you,

Mark