Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # dev - Re: kafka 0.8 producer throughput


Copy link to this message
-
Re: kafka 0.8 producer throughput
Jun Rao 2013-01-09, 05:41
The 50MB/s number is for 0.7. We haven't carefully measured the performance
in 0.8 yet. We do expect the throughput that a single producer can drive in
0.8 to be less. This is because the 0.8 producer needs to wait for an RPC
response from the broker while in 0.7, there is no ack for the producer.
Nevertheless, 2MB/s seems low. Could you try increasing flush interval to
sth bigger, like 20000?

Thanks,

Jun

On Tue, Jan 8, 2013 at 8:32 PM, Jun Guo -X (jungu - CIIC at Cisco) <
[EMAIL PROTECTED]> wrote:

> According to Kafka official document, the producer throughput is about
> 50MB/S. But I do some test, the producer throughout is only about 2MB/S.
> The test environment is the same with document says. One producer, One
> broker, One Zookeeper are in independent machine. Message size is 100
> bytes, batch size is 200, flush interval is 600 messages. The test
> environment is the same, the configuration is the same. The why there is
> such big gap the my test result and the document says?
>

 
+
Jay Kreps 2013-01-25, 17:42