Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX


+
Monika Garg 2013-10-21, 12:43
+
Neha Narkhede 2013-10-21, 15:05
+
Monika Garg 2013-10-21, 16:43
+
Jun Rao 2013-10-22, 03:31
+
Monika Garg 2013-10-22, 07:55
+
Jun Rao 2013-10-22, 15:21
+
Monika Garg 2013-10-22, 17:31
+
Monika Garg 2013-10-22, 17:41
+
Joel Koshy 2013-10-22, 17:58
Copy link to this message
-
Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX
Thanks a lot Joel.

On Tue, Oct 22, 2013 at 11:27 PM, Joel Koshy <[EMAIL PROTECTED]> wrote:

> >
> (1)"kafka.network":type="RequestChannel",name="Processor-0-ResponseQueueSize",
> >
> "kafka.network":type="RequestChannel",name="Processor-1-ResponseQueueSize",
> >
> "kafka.network":type="RequestChannel",name="Processor-2-ResponseQueueSize"
> > Im not getting what  these Processor in (1) are actually for?
>
> Kafka's request handling utilizes a number of processor "threads" and
> responses are added to the above queue before sending to the client.
> This diagram may help:
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Internals
>
>
> >
> (2)"kafka.server":type="FetcherLagMetrics",name="ReplicaFetcherThread-0-1-host_moni-192-168-145-119-port_9092-demo-0-ConsumerLag"
>
> This reports how far behind that replica is. The format is
>
> ReplicaFetcherThread-<fetcherid>-<leader-broker-id>-<leaderbrokerinfo>-<topic>-<partition>-ConsumerLag
>
> (Fetcher id is typically zero (since the default number of replica
> fetchers is one).)
>
> >
> >
> (3)"kafka.server":type="FetcherStats",name="ReplicaFetcherThread-0-1-host_moni-192-168-145-119-port_9092-RequestsPerSec"
> >
> (4)"kafka.server":type="FetcherStats",name="ReplicaFetcherThread-0-2-host_moni-192-168-145-176-port_9092-BytesPerSec"
> > The Mbeans (2),(3) and (4) are getting generated on my 3rd broker -
> > host_moni-192.168.145.67,kafka running on same port-9092.Any reason for
> > these.
>
> Requests per sec/byte rate for replica fetchers, but for all fetches
> to the indicated leader brokers (1 and 2).
>
>
> > I am getting some more Mbeans which are not mentioned here-
> > http://kafka.apache.org/documentation.html#monitoring
> >
> Kafka exposes a *lot* of mbeans - not all of them are listed on that
> page that you are referring to. I think the goal of the table was to
> provide a minimal set of critical metrics that people may want to set
> up monitoring/alerting for.
>
> Thanks,
>
> Joel
>
> >
> > On Tue, Oct 22, 2013 at 11:01 PM, Monika Garg <[EMAIL PROTECTED]>
> wrote:
> >
> >> Thanks Jun.
> >>
> >>
> >>
> >>
> >> On Tue, Oct 22, 2013 at 8:51 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >>
> >>> When a broker goes down, we have an optimization in the controller that
> >>> proactively shrinks ISR. The ISR shrink rate only measures the rate
> that
> >>> the leader shrinks ISR and doesn't reflect the shrinking by the
> >>> controller.
> >>> In general, having ISR shrink rate stay at 0 is a good thing since it
> >>> means
> >>> that the followers can always keep up.
> >>>
> >>> Thanks,
> >>>
> >>> Jun
> >>>
> >>>
> >>> On Tue, Oct 22, 2013 at 12:55 AM, Monika Garg <[EMAIL PROTECTED]>
> >>> wrote:
> >>>
> >>> > Thanks a lot Jun.
> >>> >
> >>> > Now im getting the values in ISR expansion rate,whenever broker goes
> >>> down
> >>> > and then up.
> >>> >
> >>> > But still ISR shrink rate is always having value 0.
> >>> >
> >>> > what I have done is:
> >>> >
> >>> > I made 3 node cluster.Than created 5 partitions with replica=3.
> >>> > partitions got distributed on 1,2,3 brokers with isr= 1,2,3, and
> >>> replica =
> >>> > 1,2,3
> >>> > broker -3 is leader for 2 partitions.
> >>> >
> >>> > then i stopped one broker(say 3),then isr = 1,2 and replica = 1,2,3.
> >>> > In this case I waited for some time to get the ISR shrink value to
> get
> >>> > updated in my jconsole,but value remained 0 for all attributes of ISR
> >>> > shrink rate.
> >>> >
> >>> > Then I started broker(say 3) that i stopped earlier,what I saw is:
> >>> >
> >>> > ISR expansion rate has the values as:
> >>> >
> >>> > count -2 (on both other brokers 1,2)
> >>> > oneminuterate - 0.012 etc.
> >>> >
> >>> > then I again stopped broker-1,still isr shrink rate not getting any
> >>> > value.Can you please check?
> >>> >
> >>> > One more thing if you can please have a look:
> >>> > How the other attributes like FifteenMinuteRate,OneMinuteRate etc are
> >>> > continuosly changing for any Mbean.
> >>> >
> >>> >
> >
*Moniii*

 
+
Monika Garg 2013-10-22, 08:01