Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Broker does not publish offset in zookeeper


Copy link to this message
-
Re: Broker does not publish offset in zookeeper
Are you trying to monitor the lag in the consumer? If so, you can use the
maxLag/minFetch JMX described in
http://kafka.apache.org/documentation.html#monitoring

Thanks,

Jun
On Thu, Aug 22, 2013 at 7:41 AM, Pablo Barrera González <
[EMAIL PROTECTED]> wrote:

> Hello
>
> I am trying to monitor our kafka cluster. For that I use a zookeeper client
> that reads the information of all the consumer and a jmx agent that
> connects to the brokers. I try to look for discrepancies between the data
> being produced and the data being consumed.
>
> I dislike the idea of using two different connect types (jmx and zookeeper)
> for this so I wonder why broker does not publish the offset into zookeeper?
> I don't want the last offset, just and update every X seconds or minutes.
> Is there any reason for not doing that?
>
> Other option is to use only jmx connections but then discovering the
> consumers is impossible.
>
> Thanks
>
> Pablo
>