Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Analysis of producer performance


+
Piotr Kozikowski 2013-04-08, 23:43
+
Jun Rao 2013-04-09, 04:49
+
Guy Doulberg 2013-04-09, 06:34
+
Piotr Kozikowski 2013-04-09, 17:23
+
Otis Gospodnetic 2013-04-10, 19:05
+
Piotr Kozikowski 2013-04-10, 20:11
+
Yiu Wing TSANG 2013-04-11, 02:47
+
Jun Rao 2013-04-11, 05:18
+
Piotr Kozikowski 2013-04-12, 00:46
+
Jun Rao 2013-04-12, 14:54
+
Philip OToole 2013-04-12, 15:22
+
S Ahmed 2013-04-12, 15:28
+
Philip OToole 2013-04-12, 15:59
+
Philip OToole 2013-04-12, 17:04
+
Piotr Kozikowski 2013-04-12, 23:09
Copy link to this message
-
Re: Analysis of producer performance
Piotr,

Not sure where the updated numbers are. but what you described may make
sense. In the no ack mode, if the broker is saturated, it will put back
pressure to the producer. Eventually, the producer will slow down because
the socket buffer is full. One big difference btw 0.8 and 0.7 is that the
0.8 broker has the overhead of recompressing compressed messages. If you
only have one partition in your test, all producers have to synchronize on
the log when doing recompressing, which could limit the throughput. To
improve the throughput, you can try using more partitions, turning off
compression or using a cheaper compression codec like snappy.

Thanks,

Jun
On Fri, Apr 12, 2013 at 4:08 PM, Piotr Kozikowski <[EMAIL PROTECTED]>wrote:
 
+
Piotr Kozikowski 2013-04-15, 18:19
+
David Arthur 2013-04-23, 12:22
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB