Kafka, mail # user - Re: High Level Consumer error handling and clean exit - 2013-07-09, 15:22
 Search Hadoop and all its subprojects:

Switch to Plain View
+
Chris Curtin 2013-07-09, 15:16
Copy link to this message
-
Re: High Level Consumer error handling and clean exit
It seems like you're not explicitly controlling the offsets. Is that
correct?

If so, the moment you pull a message from the stream, the client framework
considers it processed. So if your app subsequently crashes before the
message is fully processed, and "auto-commit" updates the offsets in
Zookeeper, you will drop that message.

The solution to this to call commitOffsets() explicitly.

Philip

On Tue, Jul 9, 2013 at 11:16 AM, Chris Curtin <[EMAIL PROTECTED]>wrote:
 
+
Chris Curtin 2013-07-09, 18:23
+
Philip OToole 2013-07-09, 19:24
+
Chris Curtin 2013-07-09, 19:54
+
Chris Curtin 2013-07-09, 20:09
+
Ian Friedman 2013-07-09, 21:51
+
Chris Curtin 2013-07-10, 11:14
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB