Kafka, mail # user - Re: Producer not distributing across all partitions - 2013-09-14, 03:47
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: Producer not distributing across all partitions
Without fixing KAFKA-1017, the issue is that the producer will maintain a
socket connection per min(#partitions, #brokers). If you have lots of
producers, the open file handlers on the broker could be an issue.

So, what KAFKA-1017 fixes is to pick a random partition and stick to it for
a configurable amount of time, and then switch to another random partition.
This is the behavior in 0.7 when a load balancer is used and reduces #
socket connections significantly.

The issue you are reporting seems like a bug though. Which revision in 0.8
are you using?

Thanks,

Jun
On Fri, Sep 13, 2013 at 8:28 PM, prashant amar <[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB