Hi Neha, Thanks for the quick reply. Could you tell me if there is some way of determining the offset for a consumer from a high level Java consumer class apart from ConsumerOffsetChecker tool? This tool can be run only from the command line. Is it possible to use this in a Java class?
I write streaming data from a website into Kafka topic and then read the same data using a servlet and serve it to a Java client. The problem is that the servlet sometimes does not consume data from the topic as quickly as it is produced. In order to give warning to the user saying data is lost, I will need some way of determining the consumer group offset from the servlet.
The servlet is a high level Kafka consumer and it is sufficient for our needs, so I don't want to use a simple consumer.
Any pointers will be of great help. Thanks Arathi On Wed, May 22, 2013 at 7:24 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
The other option is jmx bean that exposes the lag. Also Kafka provides at least once guarantees so even if your consumer lags occasionally, you will eventually receive all mesaages. You need to provision enough consumers so that they don't fall behind.
Thanks, Neha On May 23, 2013 5:30 AM, "arathi maddula" <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation projects and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext