Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Proper use of ConsumerConnector

Copy link to this message
Re: Proper use of ConsumerConnector
In our project we use senseidb to consume kafka data. Senseidb will process the message immediately but won't flush to disk immeidately. So if senseidb crash then all result not flushed will be lost£¬ we want to rewind kafka. The offset we want to rewind to is the flush checkpoint.
In this case, we will lost some data

Sent from my iPad

ÔÚ 2012-12-22£¬1:37£¬Neha Narkhede <[EMAIL PROTECTED]> дµÀ£º

>  But if crash happens just after offset committed, then unprocessed message in consumer will be skipped after reconnected.
> If the consumer crashes, you will get duplicates, not lose any data.
> Thanks,
> Neha