Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - help with rewriting hadoop java code for new API: getPos() and getCounter()


Copy link to this message
-
Re: help with rewriting hadoop java code for new API: getPos() and getCounter()
Hemanth Yamijala 2010-10-30, 09:39
Hi,

On Wed, Oct 27, 2010 at 2:19 AM, Bibek Paudel <[EMAIL PROTECTED]> wrote:
> [Apologies for cross-posting]
>
> HI all,
> I am rewriting a hadoop java code for the new (0.20.2) API- the code
> was originally written for versions <= 0.19.
>
> 1. What is the equivalent of the getCounter() method ? For example,
> the old code is as following:
> //import org.apache.hadoop.mapred.RunningJob;
> RunningJob job = JobClient.runJob(conf);
> Counters c = job.getCounters();
> long cnt = c.getCounter(EnumType.A);
>
> I tried to rewrite it as:
> //import org.apache.hadoop.mapreduce.Job;
> Configuration conf = new Configuration();
> Job job = new Job(conf);
> Counters c = job.getCounters();
> long cnt = c.getCounter(EnumType.A);
>

Per documentation here:
http://hadoop.apache.org/common/docs/r0.20.2/api/index.html, there
seems to be only a findCounter API, not a getCounter. Can you see if
that works ?

> I get the error: The method getCounter(MyClassName.EnumType) is
> undefined for the type Counters
>
> 2. What is the equivalent of getPos() method [0] of RecordReader ?
>
> I read that in 0.20, the getPos() methos is no longer there, but what
> am I supposed to replace it with?
>
> For example, in a map function, I have (RecordReader<IntWritable,
> CustomInput> input, Context context) and in the function body, I have:
> long pos = -1;
> pos = input.getPos();
> while(input.next(key, value)){
> //some code here
> }
>
> and I get the error: The method getPos() is undefined for the type
> RecordReader<IntWritable,CustomInput>

It doesn't seem like there's an equivalent in 0.20.2. How do you use
the position ?

Thanks
hemanth
>
> Any pointers or help will be highly appreciated.
>
> Thanks,
> Bibek
>
> [0] http://hadoop.apache.org/common/docs/r0.20.1/api/org/apache/hadoop/mapred/RecordReader.html#getPos%28%29
>