Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> changing broker hosts with 0.7.2


Copy link to this message
-
Re: changing broker hosts with 0.7.2
> No worries Philip, I'll assume you you mispoke at first when talking about
> a load-balancer between the consumers and brokers.  Kafka, unfortunately,
> doesn't allow consumers to connect to kafka via a load balancer.

Ah yes, I misspoke. I meant an LB between Producers and Kafka Brokers.

> For
> producers, also, you can't really use a load-balancer to connect to brokers
> (you can use zk, or you can use a broker list, in 0.7.2, and in 0.8, you
> can use an LB for the initial meta data connection, but then you still have
> to have direct connections to each broker from each producer).

Huh? Sure you can, if the Producers are simple. Producers just need a
destination IP address and port. They have no way of knowing if that
IP is an LB or real Kafka broker. Set the partition to -1 (i.e.
random) in all messages destined for Kafka, and it all just works.
Granted, this is a simple type of 0.72 deployment. Perhaps it doesn't
work for more complex Producers or 0.8 deployments -- I am not yet
that familiar with 0.8.

Philip

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB