Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # dev - response time measurement


+
Samaneh Shokuhi 2012-11-12, 23:14
+
Nan Zhu 2012-11-13, 00:41
+
Samaneh Shokuhi 2012-11-17, 14:12
Copy link to this message
-
Re: response time measurement
Nan Zhu 2012-11-17, 14:19
yes, you can do that

and actually there some counters implementations,

  public static enum Counter {  
    NUM_FAILED_MAPS,
    NUM_FAILED_REDUCES,
    TOTAL_LAUNCHED_MAPS,
    TOTAL_LAUNCHED_REDUCES,
    OTHER_LOCAL_MAPS,
    DATA_LOCAL_MAPS,
    RACK_LOCAL_MAPS,
    SLOTS_MILLIS_MAPS,
    SLOTS_MILLIS_REDUCES,
    FALLOW_SLOTS_MILLIS_MAPS,
    FALLOW_SLOTS_MILLIS_REDUCES,
    MAPS_RESPONSE_TIME,
    REDUCE_START_TIME,
    REDUCE_RESPONSE_TIME
  }

Best,

--
Nan Zhu
School of Computer Science,
McGill University

On Saturday, 17 November, 2012 at 9:12 AM, Samaneh Shokuhi wrote:

> Hi,
> I checked the JobInProgress.java class, i see as you said there is no
> counter as "Total Job Time" there, but there are two different variables
> ,one called "finishTime" another called "startTime" .I guess if i add
> (finishTime - startTime) to logJobSummary method that may give us the
> "Total Job Time" .Am i right ?
>
> Best,
> Samaneh
>
>
>
>
> On Tue, Nov 13, 2012 at 1:41 AM, Nan Zhu <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
>
> > There is no such a counter in default Hadoop implementation
> >
> > you can instrument your own code in JobInProgress.java to calculate it,
> >
> > there is also an implemented counter in JobInProgress, MAP_SLOTS_MILLS,
> > considering both the map tasks occupying multiple slots, but this value
> > also increase when the task is failed,
> >
> > Best,
> >
> > --
> > Nan Zhu
> > School of Computer Science,
> > McGill University
> >
> >
> > On Monday, 12 November, 2012 at 6:14 PM, Samaneh Shokuhi wrote:
> > > Hi,
> > > I need to do some experience on hadoop source code.
> > > i have modified part of it in MapTask.java class and need to compare the
> > > response time for mapper /job before and after modification .Is it
> > > reported somewhere to see that?
> > >
> > >
> > > Cheers,
> > > Samaneh
> > >
> >
> >
>
>
>