Home | About | Sematext search-lucene.com search-hadoop.com search-devops.com
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Consume from X messages ago


+
James Englert 2013-03-19, 15:05
+
David Arthur 2013-03-19, 15:17
+
James Englert 2013-03-19, 15:26
+
David Arthur 2013-03-19, 15:49
Copy link to this message
-
Re: Consume from X messages ago
Jim,

You can leverage the ExportZkOffsets/ImportZkOffsets tools to do this.
ExportZkOffsets exports the consumer offsets for your group to a file in a
certain format. You can then place the desired offset per partition you
want to reset your consumer to in the exported file.

1. Shutdown the consumer
2. Export current offsets
3. Get the desired offset (current offset - 10K). As David mentions, this
is approximate and might get you more than 10K messages if the data is
compressed.
4. Replace the exported offsets with these offsets
5. Restart the consumer.

HTH,
Neha
On Tue, Mar 19, 2013 at 8:49 AM, David Arthur <[EMAIL PROTECTED]> wrote:
 
+
S Ahmed 2013-03-19, 18:01
+
Neha Narkhede 2013-03-19, 18:05
+
James Englert 2013-03-22, 15:13
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB