Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - 0.8.0 HEAD 3/4/2013 performance jump?


Copy link to this message
-
Re: 0.8.0 HEAD 3/4/2013 performance jump?
Neha Narkhede 2013-03-05, 16:46
My take on this is that since 0.8 is very new, most people are going to be
on 0.7 for a while. When those people try out Kafka 0.8, it is best if they
see performance/guarantees similar to 0.7. Gradually, people are going to
want to move to 0.8 which is when we can revisit changing the default num
acks to 1.

Thanks,
Neha
On Tue, Mar 5, 2013 at 8:30 AM, Chris Curtin <[EMAIL PROTECTED]> wrote:

> What about making it explicit in the Producer Constructor? So in addition
> to passing the Config object you set the ACK rule?
>
> Someone with a working 0.7x application is going to have to make a number
> of changes anyway so this shouldn't significantly impact the upgrade
> process.
>
> I know you're pushing for 0.8 stability but it would make it obvious to
> everyone the impact of this important new feature.
>
> Chris
>
>
> On Tue, Mar 5, 2013 at 11:13 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Chris, Joe,
> >
> > Yes, the default ack is currently 0. Let me explain the ack mode a bit
> more
> > so that we are on the same page (details are covered in my ApachCon
> > presentation
> > http://www.slideshare.net/junrao/kafka-replication-apachecon2013) .
> There
> > are only 3 ack modes that make sense.
> >
> > ack=0: producer waits until the message is in the producer's socket
> buffer
> > ack=1: producer waits until the message is received by the leader
> > ack=-1: producer waits until the message is committed
> >
> > The tradeoffs are:
> >
> > ack=0: lowest latency; some data loss during broker failure
> > ack=1: lower latency; a few data loss during broker failure
> > ack=-1: low latency; no data loss during broker failure
> >
> > All cases work with replication factor 1, which is the default setting
> out
> > of box. With ack=1/-1, the producer may see some error when the leader
> > hasn't been being elected. However, the number of errors should be small
> > since typically leaders are elected very quickly.
> >
> > The argument for making the default ack 0 is that (1) this is the same
> > behavior you get in 0.7 and (2) the producer runs fastest in this mode.
> >
> > The argument for making the default ack 1 or -1 is that they gave you
> > better reliability.
> >
> > I am not sure what's the best thing to do that here since correct setting
> > really depends on the application. What do people feel?
> >
> > Thanks,
> >
> > Jun
> >
> >
> >
>