Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Kafka stats description


+
Hanish Bansal 2013-06-14, 15:23
+
Jun Rao 2013-06-15, 15:51
+
Hanish Bansal 2013-06-15, 16:44
+
Jun Rao 2013-06-16, 03:55
+
Hanish Bansal 2013-06-17, 08:30
+
Jun Rao 2013-06-18, 04:25
Copy link to this message
-
Re: Kafka stats description
Hi

If i look at these numbers(getting from kafka) then according to that Ms
should be microseconds.

NumFetchRequests=5350287

TotalFetchRequestMs=56605349834

FetchRequestsPerSecond=92.76876273667219

*Calculation of  **FetchRequestsPerSecond manually:*

FetchRequestsPerSecond = NumFetchRequests/TotalFetchRequestTime in seconds
=
5350287/56605 = 94.5 ~ 92.76 .

But if i look at these 2 parameters values then it seems to be in
milliseconds:

AvgFetchRequestMs=0.008763

MaxFetchRequestMs=3.610314

I am still not able to finalize that reported time is in milliseconds or
microseconds.
On Tue, Jun 18, 2013 at 9:55 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Correct, except that they are millisecs.
>
> Thanks,
>
> Jun
>
>
> On Mon, Jun 17, 2013 at 1:29 AM, Hanish Bansal <
> [EMAIL PROTECTED]> wrote:
>
> > Hi Jun
> >
> > As per my understanding:
> >
> > TotalFetchRequestMs - Total time of fetch requests in *microseconds*
> >
> > AvgFetchRequestMs - Average time of all fetch requests in *microseconds*
> >
> > MaxFetchRequestMs - Maximum time of a fetch request in *microseconds*
> >
> > **
> > Here I am taking Ms as Microsecond.
> >
> > Am i right or missing something ?
> >
> >
> >
> > On Sun, Jun 16, 2013 at 9:25 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> > > The important metrics are byte rate and message rate in both the
> clients
> > > and the broker.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Sat, Jun 15, 2013 at 9:44 AM, Hanish Bansal <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > > > Thanks for reply Jun !
> > > >
> > > > But still not clear with significance of these parameters.
> > > >
> > > >
> > > > On Sat, Jun 15, 2013 at 9:20 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > AvgProduceRequestMs: avg time of a produce RPC request
> > > > >
> > > > > For mbeans with Num and Total, they represent the cumulated value
> > since
> > > > the
> > > > > service is up.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > > On Fri, Jun 14, 2013 at 8:22 AM, Hanish Bansal <
> > > > > [EMAIL PROTECTED]> wrote:
> > > > >
> > > > > > Hi
> > > > > >
> > > > > > I'm trying to fetch Kafka stats using JMX and able to fetch the
> > > > > parameters
> > > > > > mentioned below.
> > > > > >
> > > > > > Can somebody please elaborate on what each of these parameters
> > > signify.
> > > > > > Although the name is pretty indicative but I still want to be
> sure.
> > > As
> > > > > per
> > > > > > my understanding:
> > > > > >
> > > > > > ProduceRequestsPerSecond - number of requests produced into Kafka
> > per
> > > > > > second
> > > > > > FetchRequestsPerSecond - number of requests fetched from kafka
> per
> > > > second
> > > > > > AvgProduceRequestMs - ?
> > > > > > MaxProduceRequestMs- ?
> > > > > > AvgFetchRequestMs- ?
> > > > > > MaxFetchRequestMs -?
> > > > > > BytesReadPerSecond - Bytes read from kafka per second. So when
> the
> > > > > consumer
> > > > > > reads from the kafka broker this parameter is affected.
> > > > > > BytesWrittenPerSecond - Bytes written into kafka per second. So
> > when
> > > > the
> > > > > > producer produces into kafka broker this parameter is affected.
> > > > > > NumFetchRequests - Total number of fetch requests from Kafka
> > > > > > NumProduceRequests - Total number of produced requests into kafka
> > > > > >
> > > > > > TotalBytesRead - Total number of bytes read from Kafka. Is this
> > from
> > > > the
> > > > > > beginning of the time when the kafka starts and never resets???
> > > > > > TotalBytesWritten - Total number of bytes written into Kafka. Is
> > this
> > > > > also
> > > > > > from
> > > > > > the beginning of the time when the kafka starts and never
> resets???
> > > > > > TotalFetchRequestMs - ?
> > > > > > TotalProduceRequestMs - ?
> > > > > >
> > > > > > Here are the values I'm getting but I'm not able to properly
> > > interpret
> > > > > > them.
> > > > > >
> > > > > > NumFetchRequests=5350287
> > > > > >

*Thanks & Regards*
*Hanish Bansal*

 
+
Jun Rao 2013-06-20, 03:59
+
Hanish Bansal 2013-06-20, 04:19
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB