Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Analysis of producer performance

Copy link to this message
Re: Analysis of producer performance
Hi all,

I posted an update on the post (
https://blog.liveramp.com/2013/04/08/kafka-0-8-producer-performance-2/) to
test the effect of disabling ack messages from brokers. It appears this
only makes a big difference (~2x improvement ) when using synthetic log
messages, but only a modest 12% improvement when using real production
messages. This is using GZIP compression. The way I interpret this is that
just turning acks off is not enough to mimic the 0.7 behavior because GZIP
consumes significant CPU time and since the brokers now need to decompress
data, there is a hit on throughput even without acks. Does this sound



On Mon, Apr 8, 2013 at 4:42 PM, Piotr Kozikowski <[EMAIL PROTECTED]> wrote:

> Hi,
> At LiveRamp we are considering replacing Scribe with Kafka, and as a first
> step we run some tests to evaluate producer performance. You can find our
> preliminary results here:
> https://blog.liveramp.com/2013/04/08/kafka-0-8-producer-performance-2/.
> We hope this will be useful for some folks, and If anyone has comments or
> suggestions about what to do differently to obtain better results your
> feedback will be very welcome.
> Thanks,
> Piotr