Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Killing broker leader


Copy link to this message
-
Re: Killing broker leader
Hi Francois,

Probably Kafka-1193 is not due to any misconfiguration, there may be
something else is missing. I also tried both (max retries 10 and producer
acks -1) together, that was also causing data loss.
On Wed, Jan 15, 2014 at 9:40 PM, François Langelier
<[EMAIL PROTECTED]>wrote:

> Yeah, that's what i found... sorry about that...
>
> Well, i tested it out!
> At first, my command was :
>
> bin/kafka-console-producer.sh --broker-list localhost:9092 --topic
> my-replicated-topic --message-send-max-retries 10 --retry-backoff-ms 1000
>
>
> Then i noticed that only the second message i sent got lost... So i tough
> it could be because my producer get the message but didn't have the time to
> replicate it (?) before it get killed, so i tried that because i have 3
> brokers :
>
> bin/kafka-console-producer.sh --broker-list localhost:9092 --topic
> my-replicated-topic --message-send-max-retries 10 --retry-backoff-ms 1000
> --request-required-acks 2
>
> And now that's working...
>
> So since the beginning I had no "bug" just a misconfiguration...
>
> @Hanish: maybe https://issues.apache.org/jira/browse/KAFKA-1193 is a
> misconfiguration too? I know you tried  --message-send-max-retries 10 and
>  --request-required-acks -1 but have you tried both together?
>
> Thank for your help guys!
>
>
>
> François Langelier
> Étudiant en génie Logiciel - École de Technologie
> Supérieure<http://www.etsmtl.ca/>
> Capitaine Club Capra <http://capra.etsmtl.ca/>
> VP-Communication - CS Games <http://csgames.org> 2014
> Jeux de Génies <http://www.jdgets.com/> 2011 à 2014
> Argentier Fraternité du Piranhas <http://fraternitedupiranha.com/>
> 2012-2014
> Comité Organisateur Olympiades ÉTS 2012
> Compétition Québécoise d'Ingénierie 2012 - Compétition Senior
>
>
> On Wed, Jan 15, 2014 at 10:44 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Those are actually producer side configs.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Wed, Jan 15, 2014 at 6:51 AM, François Langelier
> > <[EMAIL PROTECTED]>wrote:
> >
> > > Nope, it's the "3 tries" messages... maybe I did something wrong...
> > >
> > > i put
> > >
> > > message.send.max.retries=10
> > > retry.backoff.ms=1000
> > >
> > > in my server.properties of each broker
> > >
> > > I'm checking it right now!
> > >
> > >
> > >
> > >
> > > François Langelier
> > > Étudiant en génie Logiciel - École de Technologie
> > > Supérieure<http://www.etsmtl.ca/>
> > > Capitaine Club Capra <http://capra.etsmtl.ca/>
> > > VP-Communication - CS Games <http://csgames.org> 2014
> > > Jeux de Génies <http://www.jdgets.com/> 2011 à 2014
> > > Argentier Fraternité du Piranhas <http://fraternitedupiranha.com/>
> > > 2012-2014
> > > Comité Organisateur Olympiades ÉTS 2012
> > > Compétition Québécoise d'Ingénierie 2012 - Compétition Senior
> > >
> > >
> > > On Tue, Jan 14, 2014 at 7:32 PM, Guozhang Wang <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > Hi Francois, just a quick question, when you set the number of
> retries
> > to
> > > > 10, does its log still have "ailed to send messages after 10 tries."
> > > entry?
> > > >
> > > >
> > > > On Tue, Jan 14, 2014 at 11:41 AM, Francois Langelier <
> > > > [EMAIL PROTECTED]> wrote:
> > > >
> > > > > Of course! As soon as it will close, i'll try!
> > > > >
> > > > >
> > > > > On Tue, Jan 14, 2014 at 2:29 PM, Guozhang Wang <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > > >
> > > > > > Hanish and Francois,
> > > > > >
> > > > > > The current patch of 1193 still have something missing and I am
> > > > currently
> > > > > > working on it to be closed soon. Could you retry the scenario
> after
> > > it
> > > > is
> > > > > > checked in?
> > > > > >
> > > > > > Guozhang
> > > > > >
> > > > > >
> > > > > > On Tue, Jan 14, 2014 at 8:03 AM, Francois Langelier <
> > > > > > [EMAIL PROTECTED]> wrote:
> > > > > >
> > > > > > > @Guozhang Wang: I set the max retries to 10 and the backoof at
> > 1000
> > > > ms
> > > > > > but
> > > > > > > the bug still there and some messages don't reach my

*Thanks & Regards*
*Hanish Bansal*