Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> changing broker hosts with 0.7.2

Copy link to this message
Re: changing broker hosts with 0.7.2
> No worries Philip, I'll assume you you mispoke at first when talking about
> a load-balancer between the consumers and brokers.  Kafka, unfortunately,
> doesn't allow consumers to connect to kafka via a load balancer.

Ah yes, I misspoke. I meant an LB between Producers and Kafka Brokers.

> For
> producers, also, you can't really use a load-balancer to connect to brokers
> (you can use zk, or you can use a broker list, in 0.7.2, and in 0.8, you
> can use an LB for the initial meta data connection, but then you still have
> to have direct connections to each broker from each producer).

Huh? Sure you can, if the Producers are simple. Producers just need a
destination IP address and port. They have no way of knowing if that
IP is an LB or real Kafka broker. Set the partition to -1 (i.e.
random) in all messages destined for Kafka, and it all just works.
Granted, this is a simple type of 0.72 deployment. Perhaps it doesn't
work for more complex Producers or 0.8 deployments -- I am not yet
that familiar with 0.8.