Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Not balancing across multiple brokers


+
Tom Brown 2013-04-12, 19:17
Copy link to this message
-
Re: Not balancing across multiple brokers
Do you use a VIP or zookeeper for producer side load balancing ? In
other words, what are the values you override for "broker.list" and
"zk.connect" in the producer config ?

Thanks,
Neha

On Fri, Apr 12, 2013 at 12:16 PM, Tom Brown <[EMAIL PROTECTED]> wrote:
> We have recently setup a new kafka (0.7.1) cluster with two brokers. Each
> topic has 2 partitions per server. We have a two processes that that write
> to the cluster using the class: kafka.javaapi.producer.Producer.Producer.
>
> The problem is that the first process only writes to the first broker. The
> second process (using the exact same code to perform the write)
> successfully writes to both brokers.
>
> How can I identify the cause of the imbalance in the first process? How
> does the Producer decide which broker is the recipient of each message?
>
> Thanks!
>
> --Tom

 
+
Tom Brown 2013-04-12, 23:03
+
Neha Narkhede 2013-04-16, 16:10
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB