In 0.7.x, if the messages are compressed, there could be duplicated
messages during consumer rebalance. This is because we can only checkpoint
consumer offset at the compressed unit boundary. You may want to see if you
have unnecessary rebalances (see
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyaretheremanyrebalancesinmyconsumerlog%3F).
In 0.8, there won't be duplicated messages even when compression is enabled.

Thanks,

Jun
On Fri, Jul 19, 2013 at 1:16 PM, Sybrandy, Casey <
[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB