Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> librdkafka 0.8.0 released


Copy link to this message
-
Re: librdkafka 0.8.0 released
So, a single producer'a throughput is 80 MB/s? That seems pretty high. What
was the number of acks setting? Thanks for sharing these numbers.

On Sunday, November 24, 2013, Magnus Edenhill wrote:

> Hi Neha,
>
> these tests were done using 100 byte messages. More information about the
> producer performance tests can be found here:
>
> https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md#performance-numbers
>
> The tests are indicative at best and in no way scientific, but I must say
> that the Kafka broker performance is impressive.
>
> Regards,
> Magnus
>
>
>
> 2013/11/22 Neha Narkhede <[EMAIL PROTECTED] <javascript:;>>
>
> > Thanks for sharing this! What is the message size for the throughput
> > numbers stated below?
> >
> > Thanks,
> > Neha
> > On Nov 22, 2013 6:59 AM, "Magnus Edenhill" <[EMAIL PROTECTED]<javascript:;>>
> wrote:
> >
> > > This announces the 0.8.0 release of librdkafka - The Apache Kafka
> client
> > C
> > > library - now with 0.8 protocol support.
> > >
> > > Features:
> > > * Producer (~800K msgs/s)
> > > * Consumer  (~3M msgs/s)
> > > * Compression (Snappy, gzip)
> > > * Proper failover and leader re-election support - no message is ever
> > lost.
> > > * Configuration properties compatible with official Apache Kafka.
> > > * Stabilized ABI-safe API
> > > * Mainline Debian package submitted
> > > * Production quality
> > >
> > >
> > > Home:
> > > https://github.com/edenhill/librdkafka
> > >
> > > Introduction and performance numbers:
> > > https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md
> > >
> > > Have fun.
> > >
> > > Regards,
> > > Magnus
> > >
> > > P.S.
> > > Check out Wikimedia Foundation's varnishkafka daemon for a use case -
> > > varnish log forwarding over Kafka:
> > > https://github.com/wikimedia/varnishkafka
> > >
> >
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB