Karl Kirch 2013-04-23, 16:23
-Re: Clustered Kafka/Zookeeper setup not rebalancing all partitions
Jun Rao 2013-04-24, 01:02
Could you run the tool kafka.tools.ConsumerOffsetChecker? It will show you
which consumer instance owns which partition.
On Tue, Apr 23, 2013 at 12:12 AM, Karl Kirch <[EMAIL PROTECTED]> wrote:
> Our setup is a three node cluster, we have a Kafka 0.7.2 (with 10
> partitions per node) and Zookeeper 3.3.4 on each node. When we first start
> up the cluster, there are no issues and things work just fine, but after a
> while things start acting a bit haywire. It looks like we start to miss
> messages that should be coming through the system. However on client
> restart, logging shows that only the partitions on broker 2 and 3 are being
> re-balanced (for the particular topic I'm connecting to). I'm confidant
> that there is nothing else connected to the cluster (as we've seen this a
> couple of times now). To make it more interesting, if I switch topics, the
> client will rebalance the partitions on all three brokers.
> Anyone know of any reason why this could occur?