Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Analysis of producer performance


+
Piotr Kozikowski 2013-04-08, 23:43
+
Jun Rao 2013-04-09, 04:49
Copy link to this message
-
Re: Analysis of producer performance
Hi Jun

Great presentation,  great feature
On 04/09/2013 07:48 AM, Jun Rao wrote:
> Piotr,
>
> Thanks for sharing this. Very interesting and useful study. A few comments:
>
> 1. For existing 0.7 users, we have a migration tool that mirrors data from
> an 0.7 cluster to an 0.8 cluster. Applications can upgrade to 0.8 by
> upgrading consumers first, followed by producers.
>
> 2. Have you looked at the Kafka ApacheCon slides (
> http://www.slideshare.net/junrao/kafka-replication-apachecon2013)? Towards
> the end, there are some performance numbers too. The figure for throughput
> vs #producer is different from what you have. Not sure if this is because
> that you have turned on compression.
>
> 3. Not sure that I understand the difference btw the first 2 graphs in the
> latency section. What's different btw the 2 tests?
>
> 4. Post 0.8, we plan to improve the producer side throughput by
> implementing non-blocking socket on the client side.
>
> Jun
>
>
> On Mon, Apr 8, 2013 at 4:42 PM, Piotr Kozikowski <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> At LiveRamp we are considering replacing Scribe with Kafka, and as a first
>> step we run some tests to evaluate producer performance. You can find our
>> preliminary results here:
>> https://blog.liveramp.com/2013/04/08/kafka-0-8-producer-performance-2/. We
>> hope this will be useful for some folks, and If anyone has comments or
>> suggestions about what to do differently to obtain better results your
>> feedback will be very welcome.
>>
>> Thanks,
>>
>> Piotr
>>
 
+
Piotr Kozikowski 2013-04-09, 17:23
+
Otis Gospodnetic 2013-04-10, 19:05
+
Piotr Kozikowski 2013-04-10, 20:11
+
Yiu Wing TSANG 2013-04-11, 02:47
+
Jun Rao 2013-04-11, 05:18
+
Piotr Kozikowski 2013-04-12, 00:46
+
Jun Rao 2013-04-12, 14:54
+
Piotr Kozikowski 2013-04-12, 23:09
+
Jun Rao 2013-04-15, 01:06
+
Philip OToole 2013-04-12, 15:22
+
S Ahmed 2013-04-12, 15:28
+
Philip OToole 2013-04-12, 15:59
+
Philip OToole 2013-04-12, 17:04
+
Piotr Kozikowski 2013-04-15, 18:19
+
David Arthur 2013-04-23, 12:22
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB