Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Broker bind address versus published hostname in ZooKeeper


Copy link to this message
-
Re: Broker bind address versus published hostname in ZooKeeper
Hi Folks/Roger,

Unfortunately I don't have legal clearance to contribute patches yet back
to Kafka for code done at work, so Roger it will be great if you can
provide this patch.

Thanks!

Tim
On Mon, Oct 21, 2013 at 11:17 AM, Roger Hoover <[EMAIL PROTECTED]>wrote:

> Agreed.  Tim, it would be very helpful is you could provide a patch.
>  Otherwise, I may be willing to create one.
>
>
> On Thu, Oct 17, 2013 at 8:15 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Tim,
> >
> > This seems like a reasonable requirement. Would you be interested in
> > providing a patch to the jira?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Oct 17, 2013 at 3:20 PM, Timothy Chen <[EMAIL PROTECTED]> wrote:
> >
> > > Hi Roger,
> > >
> > > That's exactly what I need in my end, and actually internally created a
> > new
> > > property called zkHost.name to publish a different host to zk. This is
> > also
> > > needed for deploying Kafka into Azure.
> > >
> > > I also created zkHost.port since the internal and external ports that's
> > > exposed might be different as well.
> > >
> > > Tim
> > >
> > >
> > > On Thu, Oct 17, 2013 at 3:13 PM, Roger Hoover <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'm getting started experimenting with Kafka and ran into a
> > configuration
> > > > issue.
> > > >
> > > > Currently, in server.properties, you can configure host.name which
> > gets
> > > > used for two purposes: 1) to bind the socket 2) to publish the broker
> > > > details to ZK for clients to use.
> > > >
> > > > There are times when these two settings need to be different. Here's
> an
> > > > example. I want to setup Kafka brokers on OpenStack virtual machines
> > in a
> > > > private cloud but I need producers to connect from elsewhere on the
> > > > internal corporate network. With OpenStack, the virtual machines are
> > only
> > > > exposed to DHCP addresses (typically RFC 1918 private addresses). You
> > can
> > > > assign "floating ips" to a virtual machine but it's forwarded using
> > > Network
> > > > Address Translation and not exposed directly to the VM. Also, there's
> > > > typically no DNS to provide hostname lookup. Hosts have names like
> > > > "fubar.novalocal" that are not externally routable.
> > > >
> > > > Here's what I want. I want the broker to bind to the VM's private
> > network
> > > > IP but I want it to publish it's floating IP to ZooKeeper so that
> > > producers
> > > > can publish to it.
> > > >
> > > > I propose a new optional parameter, "listen", which would allow you
> to
> > > > specify the socket address to listen on. If not set, the parameter
> > would
> > > > default to host.name, which is the current behavior.
> > > >
> > > > #Publish the externally routable IP in ZK
> > > > host.name = <floating ip>
> > > > #Accept connections from any interface the VM knows about
> > > > listen = *
> > > >
> > > > I'm assuming others will eventually have the same requirement so I've
> > > added
> > > > a JIRA ticket.
> > > >
> > > > https://issues.apache.org/jira/browse/KAFKA-1092
> > > >
> > > > Thanks for your consideration.
> > > >
> > > > Cheers,
> > > >
> > > > Roger
> > > >
> > >
> >
>