Our setup is a three node cluster, we have a Kafka 0.7.2 (with 10 partitions per node) and Zookeeper 3.3.4 on each node. I'm seeing that only one broker is being used to actually send messages through. We're seeing this behavior after a restart and deletion of the kafka data (/tmp/kafka-logs). In the past its used all three and worked fine.
Is this normal? (i.e. it takes some time to spread across the cluster) Or should it distribute stuff immediately?
Now to make things even more interesting. I restarted 2 and now it sees all 3 nodes. I think I've got some sort of weirdness happening with how I'm bringing everything online.
I've tried starting up all the zookeeper instances and then each of the brokers. I've tried starting up all of the kafka brokers and then the zookeeper instances. I've also tried bringing up each machine (zookeeper and kafka) up individually. All seem to give inconsistent results.
Is there a recommended way to bring nodes online?
On Apr 23, 2013, at 1:36 PM, Karl Kirch <[EMAIL PROTECTED]> wrote:
I'll try to keep an eye on that and see if that fixes the problem. Also, how will that affect things if I have a node completely go away (i.e. zookeeper and kafka both go offline for a single node). Should I still bring up zookeeper on that node first or does it matter at that point? Karl On Apr 23, 2013, at 2:11 PM, Xavier Stevens <[EMAIL PROTECTED]> wrote: