We observed below correlation between kafka configuration and performance data:
1. producer insertion rate drops as compression enabled, especially gzip; 2. when the producer batch size is below 200, we got high CPU/network IO on brokers, got high network IO on producers/consumers;
If the data is compressed, the broker has to recompress the messages in order to assign offsets. So, there is some CPU overhead. However, it shouldn't be too high. How high of the CPU load did you observe?
Jun On Mon, Oct 21, 2013 at 12:01 PM, Lu Xuechao <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext