Home | About | Sematext search-lucene.com search-hadoop.com search-devops.com
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Duplicate Messages on the Consumer


Copy link to this message
-
Re: Duplicate Messages on the Consumer
In 0.7.x, if the messages are compressed, there could be duplicated
messages during consumer rebalance. This is because we can only checkpoint
consumer offset at the compressed unit boundary. You may want to see if you
have unnecessary rebalances (see
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyaretheremanyrebalancesinmyconsumerlog%3F).
In 0.8, there won't be duplicated messages even when compression is enabled.

Thanks,

Jun
On Fri, Jul 19, 2013 at 1:16 PM, Sybrandy, Casey <
[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB