Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> 0.72 Kafka - ZK over VPN


Copy link to this message
-
Re: 0.72 Kafka - ZK over VPN
Actually, we are using custom producers, which recover fully after any
disconnect-and-reconnect from Kafka. It is the High-Level consumers and
Kafka itself than concerned me.

I have seen good behaviour from the system in these conditions before, but
wanted to confirm.

Philip
On Wed, Nov 27, 2013 at 7:46 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> I assume that you are using a ZK-based producer. If brokers don't change in
> the window where the VPN connection is down, this may not matter. When the
> VPN connection is back, the ZK session will expire and the producer will
> establish establish a new ZK session and  new connections to the brokers.
>
> Thanks,
>
> Jun
>
>
> On Tue, Nov 26, 2013 at 9:33 PM, Philip O'Toole <[EMAIL PROTECTED]> wrote:
>
> > I want to use a ZK cluster for my Kafka cluster, which is only available
> > over a cross-country VPN tunnel. The VPN tunnel is prone to resets, every
> > other day or so, perhaps down for a couple of  minutes at a time.
> >
> > Is this a concern? Any setting changes I should make to mitigate any
> > potential issues? Will my simple producers be affected at all, while
> > writing to a Kafka broker that experiences this?
> >
> > Thanks,
> >
> > Philip
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB