Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> 0.8.0 HEAD 3/4/2013 performance jump?


+
Chris Curtin 2013-03-04, 20:01
+
Neha Narkhede 2013-03-04, 20:07
+
Jun Rao 2013-03-05, 06:01
+
Chris Curtin 2013-03-05, 13:30
+
Joe Stein 2013-03-05, 14:37
+
Chris Curtin 2013-03-05, 15:56
+
Jun Rao 2013-03-05, 16:14
Copy link to this message
-
Re: 0.8.0 HEAD 3/4/2013 performance jump?
I vote for ack=1.

It is a reasonable tradeoff between performance and reliability.

On 03/05/2013 08:13 AM, Jun Rao wrote:
> Chris, Joe,
>
> Yes, the default ack is currently 0. Let me explain the ack mode a bit more
> so that we are on the same page (details are covered in my ApachCon
> presentation
> http://www.slideshare.net/junrao/kafka-replication-apachecon2013) . There
> are only 3 ack modes that make sense.
>
> ack=0: producer waits until the message is in the producer's socket buffer
> ack=1: producer waits until the message is received by the leader
> ack=-1: producer waits until the message is committed
>
> The tradeoffs are:
>
> ack=0: lowest latency; some data loss during broker failure
> ack=1: lower latency; a few data loss during broker failure
> ack=-1: low latency; no data loss during broker failure
>
> All cases work with replication factor 1, which is the default setting out
> of box. With ack=1/-1, the producer may see some error when the leader
> hasn't been being elected. However, the number of errors should be small
> since typically leaders are elected very quickly.
>
> The argument for making the default ack 0 is that (1) this is the same
> behavior you get in 0.7 and (2) the producer runs fastest in this mode.
>
> The argument for making the default ack 1 or -1 is that they gave you
> better reliability.
>
> I am not sure what's the best thing to do that here since correct setting
> really depends on the application. What do people feel?
>
> Thanks,
>
> Jun
>
>
>

--
*Colin Blower*
/Software Engineer/
Barracuda Networks Inc.
+1 408-342-5576 (o)

Copy, by Barracuda, helps you store, protect, and share all your amazing
things. Start today: www.copy.com.

 
+
Chris Curtin 2013-03-05, 16:30
+
Neha Narkhede 2013-03-05, 16:46
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB