Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Consumers can't connect while broker is under load

Copy link to this message
Re: Consumers can't connect while broker is under load
Can you post the consumer logs for the long-time-to-connect scenarios?

On Fri, Jan 10, 2014 at 1:20 PM, Tim Kellogg <[EMAIL PROTECTED]> wrote:

> Hi,
> I have a cluster of 12 brokers receiving 10,000 msg/s from producers where
> each message is roughly 2.5KB. We also have 12 ZooKeepers and everything is
> on AWS. Under these conditions, top (the Linux utility) reports around
> 10-15 out of 32 for system load, so we’re at less than half capacity.
> When under this load consumers take a very long time, often more than 30
> minutes, to connect to the brokers. When under no load they connect
> immediately. Why is this happening?
> Thanks,
> Tim Kellogg
> Sr. Software Engineer, Protocols
> 2lemetry
> @kellogh
-- Guozhang