Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - how to increment counters inside of InputFormat/RecordReader in mapreduce api?


+
Jim Donofrio 2012-07-30, 06:11
Copy link to this message
-
Re: how to increment counters inside of InputFormat/RecordReader in mapreduce api?
Harsh J 2012-07-30, 07:38
Jim,

This is fixed in 2.x releases already via
https://issues.apache.org/jira/browse/MAPREDUCE-1905 (incompat
change). In 1.x this is a known limitation. Perhaps we can come up
with a different non-breaking fix for 1.x?

On Mon, Jul 30, 2012 at 11:41 AM, Jim Donofrio <[EMAIL PROTECTED]> wrote:
> In the mapred api getRecordReader was passed a Reporter which could then get
> passed to the RecordReader to allow a RecordReader to increment counters for
> different types of records, bad records, etc. In the new mapreduce api
> createRecordReader only gets the InputSplit and TaskAttemptContext, both
> have no access to counters.
>
> Is there really no way to increment counters inside of a RecordReader or
> InputFormat in the mapreduce api?

--
Harsh J