Home | About | Sematext search-lucene.com search-hadoop.com search-devops.com metrics + logs = try SPM and Logsene for free
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> Abou Kafka 0.8 producer throughput test


Copy link to this message
-
Re: Abou Kafka 0.8 producer throughput test
We also did some perf test on 0.8 using the following command. All configs
on the broker are the defaults.
bin/kafka-run-class.sh kafka.perf.ProducerPerformance --broker-list
localhost:9092 --initial-message-id 0 --messages 2000000 --topics topic_001

The following is our preliminary result. Could you try this on your
environment? For replication factor larger than 1, we will try ack=1 and
report the numbers later. It should provide better throughput. Thanks,

*No. of Brokers = 1 / Replication Factor = 1 (Partition = 1)**Producer
threads**comp**msg size**Acks**batch**Thru Put (MB/s)*101024-115.49201024-11
9.38501024-1116.611001024-1119.54101024-15025.72201024-15039.25501024-150
54.171001024-15056.71101024-110027.97201024-110045.05501024-110058.011001024
-110059.82*No. of Brokers = 2 / Replication Factor = 2 (Partitions =
1)**Producer
threads**comp**msg size**Acks**batch**Thru Put (MB/s)*101024-110.58201024-11
1.17501024-111.601001024-113.15101024-1507.48201024-15013.89501024-15018.11
1001024-15020.91101024-11008.72201024-110016.84501024-110020.661001024-1100
23.82*No. of Brokers = 3 / Replication Factor = 3 (Partitions = 1)**Producer
threads**comp**msg size**Acks**batch**Thru Put (MB/s)*101024-110.53201024-11
0.94501024-111.721001024-112.78101024-1507.08201024-15013.40501024-15018.11
1001024-15021.01101024-11008.09201024-110014.88501024-110019.931001024-1100
23.22
Thanks,

Jun

On Wed, Jan 16, 2013 at 8:33 PM, Jun Guo -X (jungu - CIIC at Cisco) <
[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB