Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Broker does not publish offset in zookeeper


+
Pablo Barrera González 2013-08-22, 14:42
+
Jun Rao 2013-08-22, 14:48
+
Pablo Barrera González 2013-08-22, 15:42
Copy link to this message
-
Re: Broker does not publish offset in zookeeper
Yes, that's 0.8 specific. You can find the docs in
http://kafka.apache.org/documentation.html#monitoring

Thanks,

Jun
On Thu, Aug 22, 2013 at 8:42 AM, Pablo Barrera González <
[EMAIL PROTECTED]> wrote:

> Is that 0.8 specific?
>
> Sorry, I can't find the documentation you mention (there are no references
> to the variable maxLag).
>
> I still find more problems in having a dual connection (jmx and zookeeper)
> instead of only one (zookeeper).
>
>
>
>
> 2013/8/22 Jun Rao <[EMAIL PROTECTED]>
>
> > Are you trying to monitor the lag in the consumer? If so, you can use the
> > maxLag/minFetch JMX described in
> > http://kafka.apache.org/documentation.html#monitoring
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Aug 22, 2013 at 7:41 AM, Pablo Barrera González <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hello
> > >
> > > I am trying to monitor our kafka cluster. For that I use a zookeeper
> > client
> > > that reads the information of all the consumer and a jmx agent that
> > > connects to the brokers. I try to look for discrepancies between the
> data
> > > being produced and the data being consumed.
> > >
> > > I dislike the idea of using two different connect types (jmx and
> > zookeeper)
> > > for this so I wonder why broker does not publish the offset into
> > zookeeper?
> > > I don't want the last offset, just and update every X seconds or
> minutes.
> > > Is there any reason for not doing that?
> > >
> > > Other option is to use only jmx connections but then discovering the
> > > consumers is impossible.
> > >
> > > Thanks
> > >
> > > Pablo
> > >
> >
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB