Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> How can a producer avoid slow brokers?


Copy link to this message
-
Re: How can a producer avoid slow brokers?
Do you know why some of the brokers are much slower than others?

Thanks,

Jun

On Fri, Jan 4, 2013 at 12:24 AM, Raghu Angadi <[EMAIL PROTECTED]> wrote:

> Producer distributes messages uniformly across the partitions.
>
> This does not work very well when some of the brokers are much slower than
> others. Is there a way to temporarily avoid such slow brokers?
>
> While async producers, I could avoid producers that have lot more messages
> in their internal queue compared to others (through my own Partitioners).
> But the queue size is not available. tried to maintain my own estimate of
> queue size using 'CallbackHandler', but API does not seem to provide enough
> info (it provides partition id, but not broker id. plus, CallbackHandler
> seems to be removed in 0.8).
>
> any suggestions?
>
> Kafka version : 0.7.1
>
> thanks,
> Raghu.
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB