Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Measuring remaining data


+
Jonathan Davey 2014-01-02, 21:15
Copy link to this message
-
Re: Measuring remaining data
Hi Jonathan,

Have you checked out the ConsumerOffsetChecker tool?

Guozhang
On Thu, Jan 2, 2014 at 1:15 PM, Jonathan Davey <[EMAIL PROTECTED]> wrote:

> I need to provide a report that shows the total amount of unread data
> available to a consumer group for a particular set of topics.
>
> Right now I do this by adding custom offset tracking logic to the
> producers and consumers. This works but I think I can do better with
> one of the following ideas:
>
> * Running something alongside each broker that grabs the data from the
> log dir and zookeeper
> * Extending Kafka to do the same as above but expose it as a request
> through the protocol
>
> If I wanted to do these things, where would be a good place to start
> looking?
>

--
-- Guozhang

 
+
Jonathan Davey 2014-01-02, 22:27
+
Guozhang Wang 2014-01-02, 22:43