I'm getting started experimenting with Kafka and ran into a configuration issue.
Currently, in server.properties, you can configure host.name which gets used for two purposes: 1) to bind the socket 2) to publish the broker details to ZK for clients to use.
There are times when these two settings need to be different. Here's an example. I want to setup Kafka brokers on OpenStack virtual machines in a private cloud but I need producers to connect from elsewhere on the internal corporate network. With OpenStack, the virtual machines are only exposed to DHCP addresses (typically RFC 1918 private addresses). You can assign "floating ips" to a virtual machine but it's forwarded using Network Address Translation and not exposed directly to the VM. Also, there's typically no DNS to provide hostname lookup. Hosts have names like "fubar.novalocal" that are not externally routable.
Here's what I want. I want the broker to bind to the VM's private network IP but I want it to publish it's floating IP to ZooKeeper so that producers can publish to it.
I propose a new optional parameter, "listen", which would allow you to specify the socket address to listen on. If not set, the parameter would default to host.name, which is the current behavior.
#Publish the externally routable IP in ZK host.name = <floating ip> #Accept connections from any interface the VM knows about listen = *
I'm assuming others will eventually have the same requirement so I've added a JIRA ticket.
OK. I've submitted a patch along with unit tests. This change is backward compatible with two new optional parameters called advertise.host.name and advertise.port. The reason I called them that instead of zkHost.name and zkHost.port is that the zkHost.* names sound like they represent the host and port of a ZooKeeper server rather than the host and port info of the broker that gets published to ZK.
# Hostname the broker will advertise to producers and consumers. If not set, it uses the # value for "host.name" if configured. Otherwise, it will use the value returned from # java.net.InetAddress.getCanonicalHostName(). #advertise.host.name=<hostname routable by clients>
# The port to publish to ZooKeeper for clients to use. If this is not set, # it will publish the same port that the broker binds to. #advertise.port=<port accessible by clients>
Please take a look and let me know if anything else is needed.
On Fri, Oct 25, 2013 at 4:05 PM, Roger Hoover <[EMAIL PROTECTED]>wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation projects and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext