Tim Kellogg 2014-01-11, 02:03
Guozhang Wang 2014-01-11, 02:21
-Re: Consumers can't connect while broker is under load
Jason Rosenberg 2014-01-14, 10:48
12 zookeepers seems like a lot......and you should always, by default,
prefer an odd number of zookeepers. Consumers negotiate with each
other for partition ownership, via zookeeper.....
On Fri, Jan 10, 2014 at 9:20 PM, Guozhang Wang <[EMAIL PROTECTED]> wrote:
> Can you post the consumer logs for the long-time-to-connect scenarios?
> On Fri, Jan 10, 2014 at 1:20 PM, Tim Kellogg <[EMAIL PROTECTED]> wrote:
>> I have a cluster of 12 brokers receiving 10,000 msg/s from producers where
>> each message is roughly 2.5KB. We also have 12 ZooKeepers and everything is
>> on AWS. Under these conditions, top (the Linux utility) reports around
>> 10-15 out of 32 for system load, so we’re at less than half capacity.
>> When under this load consumers take a very long time, often more than 30
>> minutes, to connect to the brokers. When under no load they connect
>> immediately. Why is this happening?
>> Tim Kellogg
>> Sr. Software Engineer, Protocols
> -- Guozhang