Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Persistence guarantees with Kafka 0.8

Copy link to this message
Re: Persistence guarantees with Kafka 0.8
Joel Koshy 2012-09-18, 20:33

The producer can specify the number of acks required - if it is set to the
replication factor, then the guarantee is that an ack will be sent only
after the message has been committed (i.e., when all followers have
received the message). If the required acks < replication factor then it is
possible for the message to be acknowledged before a leader failure and the
message will be "lost".



On Tue, Sep 18, 2012 at 12:40 PM, Rohit Prasad <[EMAIL PROTECTED]>wrote:

> Hi,
> I have gone through the replication documentation of 0.8, but have gone
> though the code. It seems that Kafka 0.8 is willing to take some message
> loss (when master fails while replicas are not in sync, and when log buffer
> is not yet persisted) to trade for good performance. It implies that
> Producers will think that a message has been committed, but there is no
> strong guarantee that it actually has. Does it mean that kafka should not
> be used in use-cases where message loss can not be tolerated? Please
> correct me if my conclusion after reading the docs (and psuedocode) is
> wrong.
> Thanks,
> Rohit