Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Not balancing across multiple brokers


+
Tom Brown 2013-04-12, 19:17
Copy link to this message
-
Re: Not balancing across multiple brokers
Do you use a VIP or zookeeper for producer side load balancing ? In
other words, what are the values you override for "broker.list" and
"zk.connect" in the producer config ?

Thanks,
Neha

On Fri, Apr 12, 2013 at 12:16 PM, Tom Brown <[EMAIL PROTECTED]> wrote:
> We have recently setup a new kafka (0.7.1) cluster with two brokers. Each
> topic has 2 partitions per server. We have a two processes that that write
> to the cluster using the class: kafka.javaapi.producer.Producer.Producer.
>
> The problem is that the first process only writes to the first broker. The
> second process (using the exact same code to perform the write)
> successfully writes to both brokers.
>
> How can I identify the cause of the imbalance in the first process? How
> does the Producer decide which broker is the recipient of each message?
>
> Thanks!
>
> --Tom

 
+
Tom Brown 2013-04-12, 23:03
+
Neha Narkhede 2013-04-16, 16:10