Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Kafka stats description


+
Hanish Bansal 2013-06-14, 15:23
+
Jun Rao 2013-06-15, 15:51
+
Hanish Bansal 2013-06-15, 16:44
+
Jun Rao 2013-06-16, 03:55
+
Hanish Bansal 2013-06-17, 08:30
+
Jun Rao 2013-06-18, 04:25
Copy link to this message
-
Re: Kafka stats description
Hi

If i look at these numbers(getting from kafka) then according to that Ms
should be microseconds.

NumFetchRequests=5350287

TotalFetchRequestMs=56605349834

FetchRequestsPerSecond=92.76876273667219

*Calculation of  **FetchRequestsPerSecond manually:*

FetchRequestsPerSecond = NumFetchRequests/TotalFetchRequestTime in seconds
=
5350287/56605 = 94.5 ~ 92.76 .

But if i look at these 2 parameters values then it seems to be in
milliseconds:

AvgFetchRequestMs=0.008763

MaxFetchRequestMs=3.610314

I am still not able to finalize that reported time is in milliseconds or
microseconds.
On Tue, Jun 18, 2013 at 9:55 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Correct, except that they are millisecs.
>
> Thanks,
>
> Jun
>
>
> On Mon, Jun 17, 2013 at 1:29 AM, Hanish Bansal <
> [EMAIL PROTECTED]> wrote:
>
> > Hi Jun
> >
> > As per my understanding:
> >
> > TotalFetchRequestMs - Total time of fetch requests in *microseconds*
> >
> > AvgFetchRequestMs - Average time of all fetch requests in *microseconds*
> >
> > MaxFetchRequestMs - Maximum time of a fetch request in *microseconds*
> >
> > **
> > Here I am taking Ms as Microsecond.
> >
> > Am i right or missing something ?
> >
> >
> >
> > On Sun, Jun 16, 2013 at 9:25 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> > > The important metrics are byte rate and message rate in both the
> clients
> > > and the broker.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Sat, Jun 15, 2013 at 9:44 AM, Hanish Bansal <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > > > Thanks for reply Jun !
> > > >
> > > > But still not clear with significance of these parameters.
> > > >
> > > >
> > > > On Sat, Jun 15, 2013 at 9:20 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > AvgProduceRequestMs: avg time of a produce RPC request
> > > > >
> > > > > For mbeans with Num and Total, they represent the cumulated value
> > since
> > > > the
> > > > > service is up.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > > On Fri, Jun 14, 2013 at 8:22 AM, Hanish Bansal <
> > > > > [EMAIL PROTECTED]> wrote:
> > > > >
> > > > > > Hi
> > > > > >
> > > > > > I'm trying to fetch Kafka stats using JMX and able to fetch the
> > > > > parameters
> > > > > > mentioned below.
> > > > > >
> > > > > > Can somebody please elaborate on what each of these parameters
> > > signify.
> > > > > > Although the name is pretty indicative but I still want to be
> sure.
> > > As
> > > > > per
> > > > > > my understanding:
> > > > > >
> > > > > > ProduceRequestsPerSecond - number of requests produced into Kafka
> > per
> > > > > > second
> > > > > > FetchRequestsPerSecond - number of requests fetched from kafka
> per
> > > > second
> > > > > > AvgProduceRequestMs - ?
> > > > > > MaxProduceRequestMs- ?
> > > > > > AvgFetchRequestMs- ?
> > > > > > MaxFetchRequestMs -?
> > > > > > BytesReadPerSecond - Bytes read from kafka per second. So when
> the
> > > > > consumer
> > > > > > reads from the kafka broker this parameter is affected.
> > > > > > BytesWrittenPerSecond - Bytes written into kafka per second. So
> > when
> > > > the
> > > > > > producer produces into kafka broker this parameter is affected.
> > > > > > NumFetchRequests - Total number of fetch requests from Kafka
> > > > > > NumProduceRequests - Total number of produced requests into kafka
> > > > > >
> > > > > > TotalBytesRead - Total number of bytes read from Kafka. Is this
> > from
> > > > the
> > > > > > beginning of the time when the kafka starts and never resets???
> > > > > > TotalBytesWritten - Total number of bytes written into Kafka. Is
> > this
> > > > > also
> > > > > > from
> > > > > > the beginning of the time when the kafka starts and never
> resets???
> > > > > > TotalFetchRequestMs - ?
> > > > > > TotalProduceRequestMs - ?
> > > > > >
> > > > > > Here are the values I'm getting but I'm not able to properly
> > > interpret
> > > > > > them.
> > > > > >
> > > > > > NumFetchRequests=5350287
> > > > > >

*Thanks & Regards*
*Hanish Bansal*

 
+
Jun Rao 2013-06-20, 03:59
+
Hanish Bansal 2013-06-20, 04:19