Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Access Hadoop Counters


+
Nan Zhu 2012-10-30, 03:49
+
Harsh J 2012-10-30, 04:51
Copy link to this message
-
Re: Access Hadoop Counters
Hi, Harsh

thanks for your reply,

yes, I'm modifying the source code of MR,

I have dig it out, just as jip.getMapCounters().getCounter(FileInputFormat.Counter.BYTES_READ);

Thx

--
Nan Zhu
School of Computer Science,
McGill University

On Tuesday, October 30, 2012 at 12:51 AM, Harsh J wrote:

> Hi,
>
> Are you attempting to modify MR sources? JobInProgress is a system
> class, not to be used by users directly. If you seek MR project-side
> development help, its better to reach the mapreduce-dev@ lists
> instead.
>
> That said, if you want the file/hdfs bytes counters, you can access
> them via the RunningJob or Job APIs directly (just give the group and
> counter names).
>
> On Tue, Oct 30, 2012 at 9:19 AM, Nan Zhu <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > Hi, all
> >
> > I noticed that in FileInputFormat and FileOutputFormat classes, there are
> > counters(Counter's name?) for input and output data size,
> >
> > my question is that given a JobInProgress object, e.g. jip, after I get the
> > corresponding object by
> >
> > FileInputFormat fif = (FileInputFormat) jip.getConf().getInputFormat();
> > FileOutputFormat fof = (FileOutputFormat)
> > jip.getConf().getOutputputFormat();
> >
> > How can I access the values of the counters?
> >
> > Best,
> >
> > --
> > Nan Zhu
> > School of Computer Science,
> > McGill University
> >
>
>
>
>
> --
> Harsh J
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB