Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Is there a way to get the offset of a consumer of a topic?


Copy link to this message
-
Re: Is there a way to get the offset of a consumer of a topic?
You can use either the MaxLag MBean (0.8):

http://kafka.apache.org/documentation.html#monitoring

Or the ConsumerOffsetChecker (0.7 or 0.8, can't seem to find a doc
reference for it):

./kafka-run-class.sh kafka.tools.ConsumerOffsetChecker ...

--
Dave DeMaagd | S'aite Reliability Engineering, Y'all
[EMAIL PROTECTED] | 818 262 7958

([EMAIL PROTECTED] - Wed, Dec 04, 2013 at 12:40:59PM -0800)
> Hi,
> I was wondering if there's an easy way to get the current offset of a
> specified consumer?  What we need is something to check the consumer offset
> every 5 min.  If it's changing at a certain rate every time we check, good.
>  If not, then we have a problem.  I was going to write a monitoring program
> to check the broker or consumer for this offset to see if it changed and
> how much it changed since the last time I checked it.
>
> I ran into this pg on the wiki which looks like it's what I need.
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetCommit%2FFetchAPI
>
> However, it says the API calls are not functional yet.  Are there some
> other methods or APIs or JMX bean/object/etc that we can use in the
> meantime?  If I were to ping zookeeper, where do I look for this offset?
>  Thanks in advance for your help.

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB