Kafka, mail # user - Re: only one ProducerSendThread thread when running with multiple brokers (kafka 0.8) - 2014-01-01, 17:57
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: only one ProducerSendThread thread when running with multiple brokers (kafka 0.8)
In 0.7, we have 1 producer send thread per broker. This is changed in 0.8,
where there is only 1 producer send thread per producer. If a producer
needs to send messages to multiple brokers, the send thread will do that
serially, which will reduce the throughput. We plan to improve that in 0.9
through client rewrites. For now, you can improve the throughput by either
using a larger batch size or using more producer instances.

As for degraded performance with more partitions, are you specifying a key
for each message?

Thanks,

Jun

On Wed, Jan 1, 2014 at 4:17 AM, yosi botzer <[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB