Home | About | Sematext search-lucene.com search-hadoop.com search-devops.com metrics + logs = try SPM and Logsene for free
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Velocity on local machine


+
Klaus Schaefers 2014-01-10, 09:44
+
Magnus Edenhill 2014-01-10, 11:33
+
Klaus Schaefers 2014-01-10, 12:07
+
Gerrit Jansen van Vuuren 2014-01-10, 13:27
Copy link to this message
-
Re: Velocity on local machine
2k msgs/s is silly, unless your messages are 10MB each, so something is
clearly wrong.

* What is the CPU usage and IO load when running your performance tests?
* Are you using the sync producer per chance? Maybe combined with an
aggressive log.flush.interval?
* For reference, and to find out where the bottleneck is, try running
rdkafka_performance [1] according to the previous link I posted:
  if performance does not increase then the broker is the problem,
otherwise it is the producer/consumers.

Regards,
Magnus

[1]: https://github.com/edenhill/librdkafka/tree/master/examples

2014/1/10 Klaus Schaefers <[EMAIL PROTECTED]>
 
+
Klaus Schaefers 2014-01-10, 16:37
+
Neha Narkhede 2014-01-10, 19:48
+
Klaus Schaefers 2014-01-13, 08:12
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB