Kafka, mail # dev - Kafka Cluster issue.. - 2013-02-08, 15:57
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Kafka Cluster issue..
Hi

I am using Kafka 0.7.2...the setup have two Kafka brokers running on two
different machines both pointing to a single Zookeper server which is on
one of the machines hosting Kafka....Now if I run the Producer, either one
provided in getting started guide or custom one, although it states that
Creating async producer for broker 1 and broker 2....but the entire load is
being processed by single Kafka broker...the second instance have no impact
and nor does it creates any logs....I have num.partition=6 for both
brokers....

I would like to understand that wouldn't Zookeeper discover the two brokers
and distribute the load.....am I missing anything here....

Also for supporting a load of 2 million messages (say 200KB each) what is
kind of topology and configuration is recommended...

Thanks,

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB