Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> API to to query messages amount under one topic


Copy link to this message
-
Re: API to to query messages amount under one topic
For monitoring, we have jmxs on the broker for both the message and the
byte rate.

Thanks,

Jun
On Thu, May 16, 2013 at 10:04 PM, Rob Withers <[EMAIL PROTECTED]> wrote:

> Immediately monitoring.  Later, possible thresholding to evoke a
> reconfiguration of the number of partitions into a new topic and migrate
> message index/logs and redirect pubs/subs to the new topic, during a
> traffic
> spike.
>
> > -----Original Message-----
> > From: Jun Rao [mailto:[EMAIL PROTECTED]]
> > Sent: Thursday, May 16, 2013 10:00 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: API to to query messages amount under one topic
> >
> > What do you plan to use this information for?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, May 16, 2013 at 5:57 AM, Withers, Robert
> > <[EMAIL PROTECTED]>wrote:
> >
> > > Any possibility for a queuedMessageCount(topic, partitionNumber)
> > protocol?
> > >
> > > thanks,
> > > rob
> > > ________________________________________
> > > From: Jun Rao [[EMAIL PROTECTED]]
> > > Sent: Wednesday, May 15, 2013 10:59 PM
> > > To: [EMAIL PROTECTED]
> > > Subject: Re: API to to query messages amount under one topic
> > >
> > > In 0.8, you can get the earliest and the latest offset using the
> > > getOffsetBefore api. The difference btw the two gives the number of
> > > messages on the broker.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Wed, May 15, 2013 at 6:05 PM, Sining Ma <[EMAIL PROTECTED]> wrote:
> > >
> > > > Hi,
> > > > Is there any APIs in kafka that I can use to query how many messages
> > > there
> > > > are under one topic?
> > > > This topic already exist in Kafka server, and producer is sending
> > > messages
> > > > to one topic in kafka server
> > > >
> > > >
> > > > -- Regards
> > > >    Sining Ma
> > > >
> > > >
> > > >
> > >
>
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB