Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Consumers can't connect while broker is under load


Copy link to this message
-
Re: Consumers can't connect while broker is under load
Guozhang Wang 2014-01-11, 02:21
Can you post the consumer logs for the long-time-to-connect scenarios?

Guozhang
On Fri, Jan 10, 2014 at 1:20 PM, Tim Kellogg <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I have a cluster of 12 brokers receiving 10,000 msg/s from producers where
> each message is roughly 2.5KB. We also have 12 ZooKeepers and everything is
> on AWS. Under these conditions, top (the Linux utility) reports around
> 10-15 out of 32 for system load, so we’re at less than half capacity.
>
> When under this load consumers take a very long time, often more than 30
> minutes, to connect to the brokers. When under no load they connect
> immediately. Why is this happening?
>
> Thanks,
>
> Tim Kellogg
> Sr. Software Engineer, Protocols
> 2lemetry
> @kellogh
--
-- Guozhang