Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Broker does not publish offset in zookeeper


Copy link to this message
-
Re: Broker does not publish offset in zookeeper
Is that 0.8 specific?

Sorry, I can't find the documentation you mention (there are no references
to the variable maxLag).

I still find more problems in having a dual connection (jmx and zookeeper)
instead of only one (zookeeper).
2013/8/22 Jun Rao <[EMAIL PROTECTED]>

> Are you trying to monitor the lag in the consumer? If so, you can use the
> maxLag/minFetch JMX described in
> http://kafka.apache.org/documentation.html#monitoring
>
> Thanks,
>
> Jun
>
>
> On Thu, Aug 22, 2013 at 7:41 AM, Pablo Barrera González <
> [EMAIL PROTECTED]> wrote:
>
> > Hello
> >
> > I am trying to monitor our kafka cluster. For that I use a zookeeper
> client
> > that reads the information of all the consumer and a jmx agent that
> > connects to the brokers. I try to look for discrepancies between the data
> > being produced and the data being consumed.
> >
> > I dislike the idea of using two different connect types (jmx and
> zookeeper)
> > for this so I wonder why broker does not publish the offset into
> zookeeper?
> > I don't want the last offset, just and update every X seconds or minutes.
> > Is there any reason for not doing that?
> >
> > Other option is to use only jmx connections but then discovering the
> > consumers is impossible.
> >
> > Thanks
> >
> > Pablo
> >
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB