Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> 0.72 Kafka - ZK over VPN

Copy link to this message
Re: 0.72 Kafka - ZK over VPN
Actually, we are using custom producers, which recover fully after any
disconnect-and-reconnect from Kafka. It is the High-Level consumers and
Kafka itself than concerned me.

I have seen good behaviour from the system in these conditions before, but
wanted to confirm.

On Wed, Nov 27, 2013 at 7:46 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> I assume that you are using a ZK-based producer. If brokers don't change in
> the window where the VPN connection is down, this may not matter. When the
> VPN connection is back, the ZK session will expire and the producer will
> establish establish a new ZK session and  new connections to the brokers.
> Thanks,
> Jun
> On Tue, Nov 26, 2013 at 9:33 PM, Philip O'Toole <[EMAIL PROTECTED]> wrote:
> > I want to use a ZK cluster for my Kafka cluster, which is only available
> > over a cross-country VPN tunnel. The VPN tunnel is prone to resets, every
> > other day or so, perhaps down for a couple of  minutes at a time.
> >
> > Is this a concern? Any setting changes I should make to mitigate any
> > potential issues? Will my simple producers be affected at all, while
> > writing to a Kafka broker that experiences this?
> >
> > Thanks,
> >
> > Philip