Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Re: Failover for Zookeeper and Kafka


Copy link to this message
-
Re: Failover for Zookeeper and Kafka
You want to setup a Zookeeper ensemble ( always an odd number of servers, three is often acceptable )
http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_zkMulitServerSetup

And use Kafka 0.8 replication http://kafka.apache.org/documentation.html#replication in addition if you want to mirror the cluster like in another data center you could also use https://cwiki.apache.org/confluence/display/KAFKA/Kafka+mirroring+(MirrorMaker)

/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop
********************************************/
On Sep 9, 2013, at 8:39 PM, Xuyen On <[EMAIL PROTECTED]> wrote:

> Hi everyone,
>
> I am trying to setup a Kafka cluster and have a couple of questions about failover.
>
> Has anyone deployed more than one zookeeper for a single Kafka cluster and have high availability so if one zookeeper node goes down, the cluster automatically fails over to a backup zookeeper node? If so, how is this done?
>
> My second question is how can I set up for automatic failover if I have a mirror secondary Kafka cluster. So, if the main Kafka cluster goes down, what do I need to do in order for my producers and consumers to automatically fail over to the backup mirror Kafka cluster. Do I need to code this into my producers and consumers, should I setup a DNS redirect in case the main Kafka cluster goes down to point to the mirror cluster, or is there some other configuration that I can do?
>
> Thanks,
>
> Xuyen
>
>

 
+
Jun Rao 2013-09-10, 14:50
+
Arjun 2013-11-25, 15:26