Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> 0.8.0 HEAD 3/4/2013 performance jump?

Copy link to this message
Re: 0.8.0 HEAD 3/4/2013 performance jump?
Hi Jun,

I wasn't explicitly setting the ack anywhere.

Am I reading the code correctly that in SyncProducerConfig.scala the
DefaultRequiredAcks is 0? Thus not waiting on the leader?

Setting:  props.put("request.required.acks", "1"); causes the writes to go
back to the performance I was seeing before yesterday.

Are you guys open to changing the default to be 1? The MongoDB Java-driver
guys made a similar default change at the end of last year because many
people didn't understand the risk that the default value of no-ack was
putting them in until they had a node failure. So they default to 'safe'
and let you decide what your risk level is vs. assuming you can lose data.



On Tue, Mar 5, 2013 at 1:00 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Chris,
> On the producer side, are you using ack=0? Earlier, ack=0 is the same as
> ack=1, which means that the producer has to wait for the message to be
> received by the leader. More recently, we did the actual implementation of
> ack=0, which means the producer doesn't wait for the message to reach the
> leader and therefore it is much faster.
> Thanks,
> Jun
> On Mon, Mar 4, 2013 at 12:01 PM, Chris Curtin <[EMAIL PROTECTED]
> >wrote:
> > Hi,
> >
> > I'm definitely not complaining, but after upgrading to HEAD today my
> > producers are running much, much faster.
> >
> > Don't have any measurements, but last release I was able to tab windows
> to
> > stop a Broker before I could generate 500 partitioned messages. Now it
> > completes before I can get the Broker shutdown!
> >
> > Anything in particular you guys fixed?
> >
> > (I did remove all the files on disk per the email thread last week and
> > reset the ZooKeeper meta, but that shouldn't matter right?)
> >
> > Very impressive!
> >
> > Thanks,
> >
> > Chris
> >