Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Analysis of producer performance


Copy link to this message
-
Re: Analysis of producer performance
Hi Jun

Great presentation,  great feature
On 04/09/2013 07:48 AM, Jun Rao wrote:
> Piotr,
>
> Thanks for sharing this. Very interesting and useful study. A few comments:
>
> 1. For existing 0.7 users, we have a migration tool that mirrors data from
> an 0.7 cluster to an 0.8 cluster. Applications can upgrade to 0.8 by
> upgrading consumers first, followed by producers.
>
> 2. Have you looked at the Kafka ApacheCon slides (
> http://www.slideshare.net/junrao/kafka-replication-apachecon2013)? Towards
> the end, there are some performance numbers too. The figure for throughput
> vs #producer is different from what you have. Not sure if this is because
> that you have turned on compression.
>
> 3. Not sure that I understand the difference btw the first 2 graphs in the
> latency section. What's different btw the 2 tests?
>
> 4. Post 0.8, we plan to improve the producer side throughput by
> implementing non-blocking socket on the client side.
>
> Jun
>
>
> On Mon, Apr 8, 2013 at 4:42 PM, Piotr Kozikowski <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> At LiveRamp we are considering replacing Scribe with Kafka, and as a first
>> step we run some tests to evaluate producer performance. You can find our
>> preliminary results here:
>> https://blog.liveramp.com/2013/04/08/kafka-0-8-producer-performance-2/. We
>> hope this will be useful for some folks, and If anyone has comments or
>> suggestions about what to do differently to obtain better results your
>> feedback will be very welcome.
>>
>> Thanks,
>>
>> Piotr
>>