Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Broker bind address versus published hostname in ZooKeeper


Copy link to this message
-
Re: Broker bind address versus published hostname in ZooKeeper
Tim,

This seems like a reasonable requirement. Would you be interested in
providing a patch to the jira?

Thanks,

Jun
On Thu, Oct 17, 2013 at 3:20 PM, Timothy Chen <[EMAIL PROTECTED]> wrote:

> Hi Roger,
>
> That's exactly what I need in my end, and actually internally created a new
> property called zkHost.name to publish a different host to zk. This is also
> needed for deploying Kafka into Azure.
>
> I also created zkHost.port since the internal and external ports that's
> exposed might be different as well.
>
> Tim
>
>
> On Thu, Oct 17, 2013 at 3:13 PM, Roger Hoover <[EMAIL PROTECTED]
> >wrote:
>
> > Hi all,
> >
> > I'm getting started experimenting with Kafka and ran into a configuration
> > issue.
> >
> > Currently, in server.properties, you can configure host.name which gets
> > used for two purposes: 1) to bind the socket 2) to publish the broker
> > details to ZK for clients to use.
> >
> > There are times when these two settings need to be different. Here's an
> > example. I want to setup Kafka brokers on OpenStack virtual machines in a
> > private cloud but I need producers to connect from elsewhere on the
> > internal corporate network. With OpenStack, the virtual machines are only
> > exposed to DHCP addresses (typically RFC 1918 private addresses). You can
> > assign "floating ips" to a virtual machine but it's forwarded using
> Network
> > Address Translation and not exposed directly to the VM. Also, there's
> > typically no DNS to provide hostname lookup. Hosts have names like
> > "fubar.novalocal" that are not externally routable.
> >
> > Here's what I want. I want the broker to bind to the VM's private network
> > IP but I want it to publish it's floating IP to ZooKeeper so that
> producers
> > can publish to it.
> >
> > I propose a new optional parameter, "listen", which would allow you to
> > specify the socket address to listen on. If not set, the parameter would
> > default to host.name, which is the current behavior.
> >
> > #Publish the externally routable IP in ZK
> > host.name = <floating ip>
> > #Accept connections from any interface the VM knows about
> > listen = *
> >
> > I'm assuming others will eventually have the same requirement so I've
> added
> > a JIRA ticket.
> >
> > https://issues.apache.org/jira/browse/KAFKA-1092
> >
> > Thanks for your consideration.
> >
> > Cheers,
> >
> > Roger
> >
>