Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Proper use of ConsumerConnector


Copy link to this message
-
Re: Proper use of ConsumerConnector
In our project we use senseidb to consume kafka data. Senseidb will process the message immediately but won't flush to disk immeidately. So if senseidb crash then all result not flushed will be lost£¬ we want to rewind kafka. The offset we want to rewind to is the flush checkpoint.
In this case, we will lost some data

Sent from my iPad

ÔÚ 2012-12-22£¬1:37£¬Neha Narkhede <[EMAIL PROTECTED]> дµÀ£º

>
>  But if crash happens just after offset committed, then unprocessed message in consumer will be skipped after reconnected.
>
> If the consumer crashes, you will get duplicates, not lose any data.
>
> Thanks,
> Neha
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB