Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Is there a way to get the offset of a consumer of a topic?


Copy link to this message
-
Re: Is there a way to get the offset of a consumer of a topic?
Simple tool I wrote to monitor 0.7 consumers.

https://github.com/otoolep/stormkafkamon
On Wed, Dec 4, 2013 at 12:49 PM, David DeMaagd <[EMAIL PROTECTED]>wrote:

> You can use either the MaxLag MBean (0.8):
>
> http://kafka.apache.org/documentation.html#monitoring
>
> Or the ConsumerOffsetChecker (0.7 or 0.8, can't seem to find a doc
> reference for it):
>
> ./kafka-run-class.sh kafka.tools.ConsumerOffsetChecker ...
>
> --
> Dave DeMaagd | S'aite Reliability Engineering, Y'all
> [EMAIL PROTECTED] | 818 262 7958
>
> ([EMAIL PROTECTED] - Wed, Dec 04, 2013 at 12:40:59PM -0800)
> > Hi,
> > I was wondering if there's an easy way to get the current offset of a
> > specified consumer?  What we need is something to check the consumer
> offset
> > every 5 min.  If it's changing at a certain rate every time we check,
> good.
> >  If not, then we have a problem.  I was going to write a monitoring
> program
> > to check the broker or consumer for this offset to see if it changed and
> > how much it changed since the last time I checked it.
> >
> > I ran into this pg on the wiki which looks like it's what I need.
> >
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetCommit%2FFetchAPI
> >
> > However, it says the API calls are not functional yet.  Are there some
> > other methods or APIs or JMX bean/object/etc that we can use in the
> > meantime?  If I were to ping zookeeper, where do I look for this offset?
> >  Thanks in advance for your help.
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB