-Re: Not balancing across multiple brokers
Neha Narkhede 2013-04-16, 16:10
I think you maybe hitting
https://issues.apache.org/jira/browse/KAFKA-278. A workaround is to
create the topic-partition directories on the newly added broker and
restart the broker. After that the zookeeper based producer will load
On Fri, Apr 12, 2013 at 4:02 PM, Tom Brown <[EMAIL PROTECTED]> wrote:
> In the producer config, we use the zk connect string:
> Both brokers have registered themselves with zookeeper. Because only the
> first broker has ever received any writes, only the first broker is
> registered for the topic in question.
> On Fri, Apr 12, 2013 at 3:32 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
>> Do you use a VIP or zookeeper for producer side load balancing ? In
>> other words, what are the values you override for "broker.list" and
>> "zk.connect" in the producer config ?
>> On Fri, Apr 12, 2013 at 12:16 PM, Tom Brown <[EMAIL PROTECTED]> wrote:
>> > We have recently setup a new kafka (0.7.1) cluster with two brokers. Each
>> > topic has 2 partitions per server. We have a two processes that that
>> > to the cluster using the class: kafka.javaapi.producer.Producer.Producer.
>> > The problem is that the first process only writes to the first broker.
>> > second process (using the exact same code to perform the write)
>> > successfully writes to both brokers.
>> > How can I identify the cause of the imbalance in the first process? How
>> > does the Producer decide which broker is the recipient of each message?
>> > Thanks!
>> > --Tom