Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Access Hadoop Counters


+
Nan Zhu 2012-10-30, 03:49
+
Harsh J 2012-10-30, 04:51
Copy link to this message
-
Re: Access Hadoop Counters
Hi, Harsh

thanks for your reply,

yes, I'm modifying the source code of MR,

I have dig it out, just as jip.getMapCounters().getCounter(FileInputFormat.Counter.BYTES_READ);

Thx

--
Nan Zhu
School of Computer Science,
McGill University

On Tuesday, October 30, 2012 at 12:51 AM, Harsh J wrote:

> Hi,
>
> Are you attempting to modify MR sources? JobInProgress is a system
> class, not to be used by users directly. If you seek MR project-side
> development help, its better to reach the mapreduce-dev@ lists
> instead.
>
> That said, if you want the file/hdfs bytes counters, you can access
> them via the RunningJob or Job APIs directly (just give the group and
> counter names).
>
> On Tue, Oct 30, 2012 at 9:19 AM, Nan Zhu <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > Hi, all
> >
> > I noticed that in FileInputFormat and FileOutputFormat classes, there are
> > counters(Counter's name?) for input and output data size,
> >
> > my question is that given a JobInProgress object, e.g. jip, after I get the
> > corresponding object by
> >
> > FileInputFormat fif = (FileInputFormat) jip.getConf().getInputFormat();
> > FileOutputFormat fof = (FileOutputFormat)
> > jip.getConf().getOutputputFormat();
> >
> > How can I access the values of the counters?
> >
> > Best,
> >
> > --
> > Nan Zhu
> > School of Computer Science,
> > McGill University
> >
>
>
>
>
> --
> Harsh J
>
>