Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Consumers can't connect while broker is under load


Copy link to this message
-
Re: Consumers can't connect while broker is under load
Can you post the consumer logs for the long-time-to-connect scenarios?

Guozhang
On Fri, Jan 10, 2014 at 1:20 PM, Tim Kellogg <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I have a cluster of 12 brokers receiving 10,000 msg/s from producers where
> each message is roughly 2.5KB. We also have 12 ZooKeepers and everything is
> on AWS. Under these conditions, top (the Linux utility) reports around
> 10-15 out of 32 for system load, so we’re at less than half capacity.
>
> When under this load consumers take a very long time, often more than 30
> minutes, to connect to the brokers. When under no load they connect
> immediately. Why is this happening?
>
> Thanks,
>
> Tim Kellogg
> Sr. Software Engineer, Protocols
> 2lemetry
> @kellogh
--
-- Guozhang

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB