Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # dev >> Kafka Cluster issue..


Copy link to this message
-
Kafka Cluster issue..
Hi

I am using Kafka 0.7.2...the setup have two Kafka brokers running on two
different machines both pointing to a single Zookeper server which is on
one of the machines hosting Kafka....Now if I run the Producer, either one
provided in getting started guide or custom one, although it states that
Creating async producer for broker 1 and broker 2....but the entire load is
being processed by single Kafka broker...the second instance have no impact
and nor does it creates any logs....I have num.partition=6 for both
brokers....

I would like to understand that wouldn't Zookeeper discover the two brokers
and distribute the load.....am I missing anything here....

Also for supporting a load of 2 million messages (say 200KB each) what is
kind of topology and configuration is recommended...

Thanks,

 
+
Jun Rao 2013-02-11, 17:15
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB