Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Re: duplicat and lost message in Kafka

Scott Carey 2013-04-22, 21:31
Copy link to this message
Re: duplicat and lost message in Kafka
In 0.8, a producer can choose to receive an ack after all replicas have
received the message. In this mode, a published message won't be lost on
individual broker failure.


On Mon, Apr 22, 2013 at 12:03 PM, Yu, Libo <[EMAIL PROTECTED]> wrote:

> Hi,
> I have read both the Kafka design document at
> http://kafka.apache.org/design.html
> and the paper. And I have two questions about lost and duplicate message.
> "Without acknowledging the producer, there is no guarantee that every
> published
> message is actually received by the broker." Is this still true as of now?
> Is there a
> solution to this issue in release 0.8?
> "However, in the case when a consumer process crashes without a clean
> shutdown,
> the consumer process that takes over those partitions owned by the failed
> consumer
> may get some duplicate messages that are after the last offset
> successfully committed
> to zookeeper."  Does this issue have to be fixed on the consumer side as
> well? Why
> not fixed in Kafka?
> Thanks.
> Regards,
> Libo Yu