Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Re: duplicat and lost message in Kafka


Copy link to this message
-
Re: duplicat and lost message in Kafka
Jun Rao 2013-04-23, 04:10
In 0.8, a producer can choose to receive an ack after all replicas have
received the message. In this mode, a published message won't be lost on
individual broker failure.

Thanks,

Jun
On Mon, Apr 22, 2013 at 12:03 PM, Yu, Libo <[EMAIL PROTECTED]> wrote:

> Hi,
> I have read both the Kafka design document at
> http://kafka.apache.org/design.html
> and the paper. And I have two questions about lost and duplicate message.
>
> "Without acknowledging the producer, there is no guarantee that every
> published
> message is actually received by the broker." Is this still true as of now?
> Is there a
> solution to this issue in release 0.8?
>
> "However, in the case when a consumer process crashes without a clean
> shutdown,
> the consumer process that takes over those partitions owned by the failed
> consumer
> may get some duplicate messages that are after the last offset
> successfully committed
> to zookeeper."  Does this issue have to be fixed on the consumer side as
> well? Why
> not fixed in Kafka?
>
> Thanks.
>
> Regards,
>
> Libo Yu
>
>