Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Consume from X messages ago


+
James Englert 2013-03-19, 15:05
Copy link to this message
-
Re: Consume from X messages ago
Using the Offsets API, you can get the latest offset by setting time to
-1. Then you subtract 10000

There is no guarantee that 10k prior messages exist of course, so you'd
need to handle that case.

-David

On 3/19/13 11:04 AM, James Englert wrote:
> Hi,
>
> I'm using Kafka 0.8.  I would like to setup a consumer to fetch the last
> 10,000 messages and then start consuming messages.
>
> I see the configuration autooffset.reset, but that isn't quite what I
> want.  I want only the last 10,000 messages.
>
> Is there a good way to achieve this in 0.8, besides just hacking the data
> in ZK?
>
> Thanks,
> Jim
>
 
+
James Englert 2013-03-19, 15:26
+
David Arthur 2013-03-19, 15:49
+
Neha Narkhede 2013-03-19, 16:16
+
S Ahmed 2013-03-19, 18:01
+
Neha Narkhede 2013-03-19, 18:05
+
James Englert 2013-03-22, 15:13