Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Please help find the reason behind performance test results.


Copy link to this message
-
Re: Please help find the reason behind performance test results.
If the data is compressed, the broker has to recompress the messages in
order to assign offsets. So, there is some CPU overhead. However, it
shouldn't be too high. How high of the CPU load did you observe?

Thanks,

Jun
On Mon, Oct 21, 2013 at 12:01 PM, Lu Xuechao <[EMAIL PROTECTED]> wrote:

> Hi,
>
> We observed below correlation between kafka configuration and performance
> data:
>
> 1. producer insertion rate drops as compression enabled, especially gzip;
> 2. when the producer batch size is below 200, we got high CPU/network IO on
> brokers, got high network IO on producers/consumers;
>
> What's the reasons behind?
>
> thanks.
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB