Have a very quick question and need your help in designing a consumer. I am trying to keep the email short and simple.
Scenario, Lets say:
1) Have a Kafka with 20 messages(from Offset 0 - 19)
2) Pulling the 1st 10 message from 0th to 9th Offset(assuming the desired size pulls 1st 10 message)
3) Manually commit the Last offset - which is Offset 9 using OffsetCommit API Now, when I want to going into the next iteration to pull the next batch of messages(in this case from 10th to 19th Offsets), I will do a Current Offset Fetch.
In this case, the offset fetch will give me 9th.
Since I have already processed the message @ 9th Offset, I will end up processing it again.
Ideally, I want to get 10 as Offset when I want to start the next iteration of starting the message processing.
I navigated through the latest Kafka documents, but I dont see any method to find out the Next Offset from Kafka rather than the current manually committed offset.
*Simply put: I processed "Nth" Offset and committed "N". I want the NEXT OFFSET FROM the Nth OFFSET. I dont want to assume it as (N+1).*
*Or is it correct if I assume its N+1 ?* Thanks for your time ! Krishna Raj
Thanks, Jun. Greatly helps and that was simple solution which I overlooked
Regards, KR On Tue, Mar 25, 2014 at 9:03 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext