Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Broker does not publish offset in zookeeper


Copy link to this message
-
Broker does not publish offset in zookeeper
Hello

I am trying to monitor our kafka cluster. For that I use a zookeeper client
that reads the information of all the consumer and a jmx agent that
connects to the brokers. I try to look for discrepancies between the data
being produced and the data being consumed.

I dislike the idea of using two different connect types (jmx and zookeeper)
for this so I wonder why broker does not publish the offset into zookeeper?
I don't want the last offset, just and update every X seconds or minutes.
Is there any reason for not doing that?

Other option is to use only jmx connections but then discovering the
consumers is impossible.

Thanks

Pablo

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB