Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> kafka producer - retry messages


Copy link to this message
-
Re: kafka producer - retry messages
Demian,
  I have been looking at building that into Sparkngin (
https://github.com/DemandCube/Sparkngin)  What kind of window are you
looking for?
-Steve

On Thu, Nov 28, 2013 at 7:23 AM, Demian Berjman <[EMAIL PROTECTED]>wrote:

> Joe, i meant that all the kafka cluster is down, even the replicas of that
> topic.
>
> Thanks,
>
>
> On Thu, Nov 28, 2013 at 12:10 PM, Joe Stein <[EMAIL PROTECTED]> wrote:
>
> > Have you tried the replication feature added to 0.8
> > http://kafka.apache.org/documentation.html#replication
> >
> >
> > /*******************************************
> >  Joe Stein
> >  Founder, Principal Consultant
> >  Big Data Open Source Security LLC
> >  http://www.stealth.ly
> >  Twitter: @allthingshadoop
> > ********************************************/
> >
> >
> > On Nov 28, 2013, at 9:37 AM, Demian Berjman <[EMAIL PROTECTED]>
> wrote:
> >
> > > Hi.
> > >
> > > Anyone has build a retry system (durable) for the producer in case the
> > > kakfa cluster is down? We have certain messages that must be sent
> because
> > > there are after a transaction that cannot be undo.
> > >
> > > We could set the property "message.send.max.retries", but if the
> producer
> > > goes down, we lost the messages.
> > >
> > > Thanks,
> >
>