-Re: Producer only finding partition on 1 of 2 Brokers, even though ZK shows 1 partition exists on both Brokers?
Neha Narkhede 2013-06-04, 23:24
You are probably hitting https://issues.apache.org/jira/browse/KAFKA-278.
Can you please try the workaround mentioned in the JIRA description?
On Tue, Jun 4, 2013 at 4:21 PM, Brett Hoerner <[EMAIL PROTECTED]>wrote:
> (version 0.7.2)
> For some reason, my producers are only picking up the partition on 1 of my
> 2 brokers. I've been digging through the code, and I don't see any issues
> given the state of my ZK nodes. The producer never seems to locate a
> partition on Broker0, even though ZK clearly states that it has 1 (just
> like Broker1 does).
> The result of this is that the producer works, but only sends data to one
> Has anyone seen something like this before? I'm stumped. Thanks.
> # zk information follows:
> [zk: localhost:2181(CONNECTED) 5] ls /kafka/brokers/ids
> [1, 0]
> [zk: localhost:2181(CONNECTED) 6] get /kafka/brokers/ids/0
> [zk: localhost:2181(CONNECTED) 7] get /kafka/brokers/ids/1
> [zk: localhost:2181(CONNECTED) 10] ls /kafka/brokers/topics/test
> [1, 0]
> # this is the most confusing one
> [zk: localhost:2181(CONNECTED) 11] get /kafka/brokers/topics/test/0
> [zk: localhost:2181(CONNECTED) 12] get /kafka/brokers/topics/test/1
> # kafka producer ZK information DEBUG log, as you can see it finds 0
> partitions on Broker0:
> Broker Topic Path => /brokers/topics
> DEBUG [2013-06-04 23:14:30,689] kafka.producer.ZKBrokerPartitionInfo:
> Broker ids and # of partitions on each for topic: test = ArrayBuffer((0,0),
> DEBUG [2013-06-04 23:14:30,690] kafka.producer.ZKBrokerPartitionInfo:
> Sorted list of broker ids and partition ids on each for topic: test =
> DEBUG [2013-06-04 23:14:30,819]
> [BrokerTopicsListener] Creating broker topics listener to watch the
> following paths -
> /broker/topics, /broker/topics/topic, /broker/ids
> DEBUG [2013-06-04 23:14:30,823]
> [BrokerTopicsListener] Initialized this broker topics listener with initial
> mapping of broker id to partition id per topic with Map(test ->
> DEBUG [2013-06-04 23:14:30,904] kafka.producer.ZKBrokerPartitionInfo:
> Registering listener on path: /brokers/topics/test