Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Velocity on local machine


+
Klaus Schaefers 2014-01-10, 09:44
+
Magnus Edenhill 2014-01-10, 11:33
Copy link to this message
-
Re: Velocity on local machine
Klaus Schaefers 2014-01-10, 12:07
Hi,

I have close to 2k messages per second. My machine is just a (BG 4-core i5
but I would expect more messages. I ran Kafka in the default settings.
On Fri, Jan 10, 2014 at 12:31 PM, Magnus Edenhill <[EMAIL PROTECTED]>wrote:

> What performance numbers did you see?
>
> For reference you can check the following tests that were also run on the
> same machine as the broker:
>
> https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md#performance-numbers
>
> Do they correspond to your numbers?
>
> Consumer thruput is not included in that document but typically peaks at
> around 3 million msgs/s when run on the same host and with hot disk caches.
>
> Regards,
> Magnus
>
>
> 2014/1/10 Klaus Schaefers <[EMAIL PROTECTED]>
>
> > Hi,
> >
> > I am currently benchmarking Kafka against ActiveMQ and I got some results
> > the surprised my quite a lot. ActiveMQ managed to deliver 4x more
> messages
> > when running locally. But from all what I was reading I am a little bit
> > surprised. Honestly I expected Kafka to outperform ActiveMQ. Some I am
> left
> > with the feeling that I configured some wrong or did any kind of other
> > mistake.
> >
> > My setup looks like this:
> >
> > - One local broker
> > - 10 Topic / Queues
> > - 1 producer that dispatches messages randomly to the topics
> > - 1 consumer per topic
> >
> > I basically used the example on the Kafka web page.
> >
> > Also I encountered some issues when increasing the number of topics, lets
> > say 100. In this case the consumer cannot connect the Zookeeper...
> >
> > Does anybody has an idea how to improve the performance?
> >
> >
> > Thx,
> >
> > Klaus
> >
> >
> > --
> >
> > --
> >
> > Klaus Schaefers
> > Senior Optimization Manager
> >
> > Ligatus GmbH
> > Hohenstaufenring 30-32
> > D-50674 Köln
> >
> > Tel.:  +49 (0) 221 / 56939 -784
> > Fax:  +49 (0) 221 / 56 939 - 599
> > E-Mail: [EMAIL PROTECTED]
> > Web: www.ligatus.de
> >
> > HRB Köln 56003
> > Geschäftsführung:
> > Dipl.-Kaufmann Lars Hasselbach, Dipl.-Kaufmann Klaus Ludemann,
> > Dipl.-Wirtschaftsingenieur Arne Wolter
> >
>

--

--

Klaus Schaefers
Senior Optimization Manager

Ligatus GmbH
Hohenstaufenring 30-32
D-50674 Köln

Tel.:  +49 (0) 221 / 56939 -784
Fax:  +49 (0) 221 / 56 939 - 599
E-Mail: [EMAIL PROTECTED]
Web: www.ligatus.de

HRB Köln 56003
Geschäftsführung:
Dipl.-Kaufmann Lars Hasselbach, Dipl.-Kaufmann Klaus Ludemann,
Dipl.-Wirtschaftsingenieur Arne Wolter

 
+
Gerrit Jansen van Vuuren 2014-01-10, 13:27
+
Magnus Edenhill 2014-01-10, 13:39
+
Klaus Schaefers 2014-01-10, 16:37
+
Neha Narkhede 2014-01-10, 19:48
+
Klaus Schaefers 2014-01-13, 08:12