Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Understanding how to monitor using JMX attributes


+
Mike Heffner 2012-11-20, 03:10
+
Jun Rao 2012-11-20, 05:34
+
Mike Heffner 2012-11-20, 16:03
+
Evan Chan 2012-11-20, 17:11
+
Mike Heffner 2012-11-20, 21:07
+
Jun Rao 2012-11-20, 17:12
+
Mike Heffner 2012-11-20, 21:12
Copy link to this message
-
Re: Understanding how to monitor using JMX attributes
The attribute getCurrentOffset gives the log end offset. It's not
necessarily the log size though since older segments could be deleted.

Thanks,

Jun

On Tue, Nov 20, 2012 at 1:12 PM, Mike Heffner <[EMAIL PROTECTED]> wrote:

> Jun,
>
> Do you have any idea on what the JMX attribute values on the beans "
> kafka:type=kafka.logs.{topic name}-{partition idx}" represent then? It
> seems like these should correctly represent the current offsets of the
> producer logs? They appeared to track correctly for a while, but once the
> log size grew, they seemed to no longer be correct. Is there potentially a
> bug in these values are large log sizes?
>
> I can try the other interface, but it would be nice to know what's wrong
> with the current JMX values.
>
> Thanks,
>
> Mike
>
>
> On Tue, Nov 20, 2012 at 12:12 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > The tool gets the end offset of the log using getOffsetBefore and the
> > consumer offset from ZK. It then calculates the lag.
> >
> > We do have a JMX for lag in ZookeeperConsumerConnector. The api is the
> > following, but you need to provide topic/brokerid/partitionid.
> >
> > /**
> >  *  JMX interface for monitoring consumer
> >  */
> > trait ZookeeperConsumerConnectorMBean {
> >   def getPartOwnerStats: String
> >   def getConsumerGroup: String
> >   def getOffsetLag(topic: String, brokerId: Int, partitionId: Int): Long
> >   def getConsumedOffset(topic: String, brokerId: Int, partitionId: Int):
> > Long
> >   def getLatestOffset(topic: String, brokerId: Int, partitionId: Int):
> Long
> > }
> >
> > Thanks
> >
> > Jun
> >
> > On Tue, Nov 20, 2012 at 8:03 AM, Mike Heffner <[EMAIL PROTECTED]> wrote:
> >
> > > I have not tried that yet, I was hoping to use an existing Ruby
> > monitoring
> > > process that we use to monitor several other existing resources.  I
> also
> > > don't want to make changes to the Kafka consumer code, as it's part of
> a
> > > bundled package (Storm).
> > >
> > > Where does ConsumerOffsetChecker pull its information from? Shouldn't
> the
> > > values from JMX match? Guess I might need to look at its source code to
> > > figure out what it's doing.
> > >
> > >
> > >
> > >
> > > On Tue, Nov 20, 2012 at 12:34 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > >
> > > > Instead of using jmx, have you tried using ConsumerOffsetChecker to
> > > figure
> > > > out the consumer lag?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Mon, Nov 19, 2012 at 7:10 PM, Mike Heffner <[EMAIL PROTECTED]>
> > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I am trying to write a custom monitoring script for our Kafka setup
> > and
> > > > > would like some help understanding how to interpret the JMX
> > attributes.
> > > > >
> > > > > In our setup, the consumers are writing their current offset to a
> > path
> > > in
> > > > > ZK. This is the value they are getting back from a call
> > > > > to SimpleConsumer.getOffsetsBefore(). A snapshot of this value
> looks
> > > > like:
> > > > >
> > > > > {"offset"=>5338008447, "partition"=>2,
> > "broker"=>{"host"=>"10.x.x.94",
> > > > > "port"=>9092}, "topic"=>"mcommits"}
> > > > >
> > > > > Using the MX4J interface, I poll the
> > > > > bean "kafka:type=kafka.logs.mcommits-2" on host 10.x.x.94 and get
> the
> > > > > attribute values:
> > > > >
> > > > > {"CurrentOffset"=>506171524, "Name"=>"mcommits-2",
> > > > > "NumAppendedMessages"=>10526508, "NumberOfSegments"=>4,
> > > > "Size"=>2116784530}
> > > > >
> > > > > At the time both of these values were snapshotted, this consumer
> was
> > > > close
> > > > > to the end of the log file. In that case, I would expect both
> offsets
> > > to
> > > > be
> > > > > fairly similar, however the consumer offset is >> the producer log
> > > > offset,
> > > > > which doesn't make sense.
> > > > >
> > > > > Clearly there is something I'm not understanding. How do I use the
> > JMX
> > > > > attributes to calculate how far behind the consumer is from the end
> > of
> > > > the
> > > > > log file? In this scenario the consumer offset is >> both the
+
Mike Heffner 2012-11-21, 15:57
+
Joel Koshy 2012-11-21, 18:47
+
Mike Heffner 2012-11-21, 22:01
+
Joel Koshy 2012-11-21, 22:16