Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - 0.8.0 HEAD 3/4/2013 performance jump?


+
Chris Curtin 2013-03-04, 20:01
+
Neha Narkhede 2013-03-04, 20:07
+
Jun Rao 2013-03-05, 06:01
+
Chris Curtin 2013-03-05, 13:30
+
Joe Stein 2013-03-05, 14:37
+
Chris Curtin 2013-03-05, 15:56
+
Jun Rao 2013-03-05, 16:14
+
Colin Blower 2013-03-05, 16:19
Copy link to this message
-
Re: 0.8.0 HEAD 3/4/2013 performance jump?
Chris Curtin 2013-03-05, 16:30
What about making it explicit in the Producer Constructor? So in addition
to passing the Config object you set the ACK rule?

Someone with a working 0.7x application is going to have to make a number
of changes anyway so this shouldn't significantly impact the upgrade
process.

I know you're pushing for 0.8 stability but it would make it obvious to
everyone the impact of this important new feature.

Chris
On Tue, Mar 5, 2013 at 11:13 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Chris, Joe,
>
> Yes, the default ack is currently 0. Let me explain the ack mode a bit more
> so that we are on the same page (details are covered in my ApachCon
> presentation
> http://www.slideshare.net/junrao/kafka-replication-apachecon2013) . There
> are only 3 ack modes that make sense.
>
> ack=0: producer waits until the message is in the producer's socket buffer
> ack=1: producer waits until the message is received by the leader
> ack=-1: producer waits until the message is committed
>
> The tradeoffs are:
>
> ack=0: lowest latency; some data loss during broker failure
> ack=1: lower latency; a few data loss during broker failure
> ack=-1: low latency; no data loss during broker failure
>
> All cases work with replication factor 1, which is the default setting out
> of box. With ack=1/-1, the producer may see some error when the leader
> hasn't been being elected. However, the number of errors should be small
> since typically leaders are elected very quickly.
>
> The argument for making the default ack 0 is that (1) this is the same
> behavior you get in 0.7 and (2) the producer runs fastest in this mode.
>
> The argument for making the default ack 1 or -1 is that they gave you
> better reliability.
>
> I am not sure what's the best thing to do that here since correct setting
> really depends on the application. What do people feel?
>
> Thanks,
>
> Jun
>
>
>

 
+
Neha Narkhede 2013-03-05, 16:46