Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: duplicat and lost message in Kafka


Copy link to this message
-
Re: duplicat and lost message in Kafka
In 0.8, a producer can choose to receive an ack after all replicas have
received the message. In this mode, a published message won't be lost on
individual broker failure.

Thanks,

Jun
On Mon, Apr 22, 2013 at 12:03 PM, Yu, Libo <[EMAIL PROTECTED]> wrote:

> Hi,
> I have read both the Kafka design document at
> http://kafka.apache.org/design.html
> and the paper. And I have two questions about lost and duplicate message.
>
> "Without acknowledging the producer, there is no guarantee that every
> published
> message is actually received by the broker." Is this still true as of now?
> Is there a
> solution to this issue in release 0.8?
>
> "However, in the case when a consumer process crashes without a clean
> shutdown,
> the consumer process that takes over those partitions owned by the failed
> consumer
> may get some duplicate messages that are after the last offset
> successfully committed
> to zookeeper."  Does this issue have to be fixed on the consumer side as
> well? Why
> not fixed in Kafka?
>
> Thanks.
>
> Regards,
>
> Libo Yu
>
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB