Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Is there a way to get the offset of a consumer of a topic?


Copy link to this message
-
Re: Is there a way to get the offset of a consumer of a topic?
Simple tool I wrote to monitor 0.7 consumers.

https://github.com/otoolep/stormkafkamon
On Wed, Dec 4, 2013 at 12:49 PM, David DeMaagd <[EMAIL PROTECTED]>wrote:

> You can use either the MaxLag MBean (0.8):
>
> http://kafka.apache.org/documentation.html#monitoring
>
> Or the ConsumerOffsetChecker (0.7 or 0.8, can't seem to find a doc
> reference for it):
>
> ./kafka-run-class.sh kafka.tools.ConsumerOffsetChecker ...
>
> --
> Dave DeMaagd | S'aite Reliability Engineering, Y'all
> [EMAIL PROTECTED] | 818 262 7958
>
> ([EMAIL PROTECTED] - Wed, Dec 04, 2013 at 12:40:59PM -0800)
> > Hi,
> > I was wondering if there's an easy way to get the current offset of a
> > specified consumer?  What we need is something to check the consumer
> offset
> > every 5 min.  If it's changing at a certain rate every time we check,
> good.
> >  If not, then we have a problem.  I was going to write a monitoring
> program
> > to check the broker or consumer for this offset to see if it changed and
> > how much it changed since the last time I checked it.
> >
> > I ran into this pg on the wiki which looks like it's what I need.
> >
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetCommit%2FFetchAPI
> >
> > However, it says the API calls are not functional yet.  Are there some
> > other methods or APIs or JMX bean/object/etc that we can use in the
> > meantime?  If I were to ping zookeeper, where do I look for this offset?
> >  Thanks in advance for your help.
>