Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Not balancing across multiple brokers


Copy link to this message
-
Re: Not balancing across multiple brokers
Do you use a VIP or zookeeper for producer side load balancing ? In
other words, what are the values you override for "broker.list" and
"zk.connect" in the producer config ?

Thanks,
Neha

On Fri, Apr 12, 2013 at 12:16 PM, Tom Brown <[EMAIL PROTECTED]> wrote:
> We have recently setup a new kafka (0.7.1) cluster with two brokers. Each
> topic has 2 partitions per server. We have a two processes that that write
> to the cluster using the class: kafka.javaapi.producer.Producer.Producer.
>
> The problem is that the first process only writes to the first broker. The
> second process (using the exact same code to perform the write)
> successfully writes to both brokers.
>
> How can I identify the cause of the imbalance in the first process? How
> does the Producer decide which broker is the recipient of each message?
>
> Thanks!
>
> --Tom

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB