Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - kafka producer - retry messages


Copy link to this message
-
Re: kafka producer - retry messages
Otis Gospodnetic 2013-11-28, 23:58
Hi,

We've done this at Sematext, where we use Kafka in all 3 products/services
you see in my signature.  When we fail to push a message into Kafka we
store it in the FS and from there we can process it later.

Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
On Thu, Nov 28, 2013 at 9:37 AM, Demian Berjman <[EMAIL PROTECTED]>wrote:

> Hi.
>
> Anyone has build a retry system (durable) for the producer in case the
> kakfa cluster is down? We have certain messages that must be sent because
> there are after a transaction that cannot be undo.
>
> We could set the property "message.send.max.retries", but if the producer
> goes down, we lost the messages.
>
> Thanks,
>