Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Killing broker leader


Copy link to this message
-
Re: Killing broker leader
Hi Francois,

Probably Kafka-1193 is not due to any misconfiguration, there may be
something else is missing. I also tried both (max retries 10 and producer
acks -1) together, that was also causing data loss.
On Wed, Jan 15, 2014 at 9:40 PM, François Langelier
<[EMAIL PROTECTED]>wrote:

> Yeah, that's what i found... sorry about that...
>
> Well, i tested it out!
> At first, my command was :
>
> bin/kafka-console-producer.sh --broker-list localhost:9092 --topic
> my-replicated-topic --message-send-max-retries 10 --retry-backoff-ms 1000
>
>
> Then i noticed that only the second message i sent got lost... So i tough
> it could be because my producer get the message but didn't have the time to
> replicate it (?) before it get killed, so i tried that because i have 3
> brokers :
>
> bin/kafka-console-producer.sh --broker-list localhost:9092 --topic
> my-replicated-topic --message-send-max-retries 10 --retry-backoff-ms 1000
> --request-required-acks 2
>
> And now that's working...
>
> So since the beginning I had no "bug" just a misconfiguration...
>
> @Hanish: maybe https://issues.apache.org/jira/browse/KAFKA-1193 is a
> misconfiguration too? I know you tried  --message-send-max-retries 10 and
>  --request-required-acks -1 but have you tried both together?
>
> Thank for your help guys!
>
>
>
> François Langelier
> Étudiant en génie Logiciel - École de Technologie
> Supérieure<http://www.etsmtl.ca/>
> Capitaine Club Capra <http://capra.etsmtl.ca/>
> VP-Communication - CS Games <http://csgames.org> 2014
> Jeux de Génies <http://www.jdgets.com/> 2011 à 2014
> Argentier Fraternité du Piranhas <http://fraternitedupiranha.com/>
> 2012-2014
> Comité Organisateur Olympiades ÉTS 2012
> Compétition Québécoise d'Ingénierie 2012 - Compétition Senior
>
>
> On Wed, Jan 15, 2014 at 10:44 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Those are actually producer side configs.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Wed, Jan 15, 2014 at 6:51 AM, François Langelier
> > <[EMAIL PROTECTED]>wrote:
> >
> > > Nope, it's the "3 tries" messages... maybe I did something wrong...
> > >
> > > i put
> > >
> > > message.send.max.retries=10
> > > retry.backoff.ms=1000
> > >
> > > in my server.properties of each broker
> > >
> > > I'm checking it right now!
> > >
> > >
> > >
> > >
> > > François Langelier
> > > Étudiant en génie Logiciel - École de Technologie
> > > Supérieure<http://www.etsmtl.ca/>
> > > Capitaine Club Capra <http://capra.etsmtl.ca/>
> > > VP-Communication - CS Games <http://csgames.org> 2014
> > > Jeux de Génies <http://www.jdgets.com/> 2011 à 2014
> > > Argentier Fraternité du Piranhas <http://fraternitedupiranha.com/>
> > > 2012-2014
> > > Comité Organisateur Olympiades ÉTS 2012
> > > Compétition Québécoise d'Ingénierie 2012 - Compétition Senior
> > >
> > >
> > > On Tue, Jan 14, 2014 at 7:32 PM, Guozhang Wang <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > Hi Francois, just a quick question, when you set the number of
> retries
> > to
> > > > 10, does its log still have "ailed to send messages after 10 tries."
> > > entry?
> > > >
> > > >
> > > > On Tue, Jan 14, 2014 at 11:41 AM, Francois Langelier <
> > > > [EMAIL PROTECTED]> wrote:
> > > >
> > > > > Of course! As soon as it will close, i'll try!
> > > > >
> > > > >
> > > > > On Tue, Jan 14, 2014 at 2:29 PM, Guozhang Wang <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > > >
> > > > > > Hanish and Francois,
> > > > > >
> > > > > > The current patch of 1193 still have something missing and I am
> > > > currently
> > > > > > working on it to be closed soon. Could you retry the scenario
> after
> > > it
> > > > is
> > > > > > checked in?
> > > > > >
> > > > > > Guozhang
> > > > > >
> > > > > >
> > > > > > On Tue, Jan 14, 2014 at 8:03 AM, Francois Langelier <
> > > > > > [EMAIL PROTECTED]> wrote:
> > > > > >
> > > > > > > @Guozhang Wang: I set the max retries to 10 and the backoof at
> > 1000
> > > > ms
> > > > > > but
> > > > > > > the bug still there and some messages don't reach my

*Thanks & Regards*
*Hanish Bansal*

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB